Using the DrmCursor Layer in Rockchip

Recently, RK has implemented a hardware cursor layer processed from the upper layer, with the repository located at https://github.com/JeffyCN/drm-cursor. Previously, hardware cursors like those on the RK3399 used the generic interface of libdrm, but on new platforms such as RK3568 and RK3588, it seems that the kernel does not set up a separate mouse layer.

If you need to use a hardware cursor on the platform, you can use this library to implement it.

The drm cursor mouse pointer implemented here is achieved by hooking the relevant functions for setting the mouse in drm through the EGL interface, avoiding the need to go through libdrm to the kernel.

Here is an explanation of what a cursor is:

Cursors

Similar to planes, many hardware also supports cursors. A cursor is a very small buffer with an image that is blended over the CRTC framebuffer. You can set a different cursor for each CRTC with drmModeSetCursor(3) and move it on the screen with drmModeMoveCursor(3). This allows to move the cursor on the screen without rerendering. If no hardware cursors are supported, you need to rerender for each frame the cursor is moved.

1: Standard Cursor Function Implementation

1.1 Implementation of libdrm Functions

drmModeSetCursor
xf86drmMode.c +412

drmpublic int drmModeSetCursor(int fd, uint32_t crtcId, uint32_t bo_handle,
uint32_t width, uint32_t height)
{
struct drm_mode_cursor arg;

memclear(arg);
arg.flags = DRM_MODE_CURSOR_BO;
arg.crtc_id = crtcId;
arg.width = width;
arg.height = height;
arg.handle = bo_handle;

return DRM_IOCTL(fd, DRM_IOCTL_MODE_CURSOR, &arg);
}

DRM_IOCTL_DEF(DRM_IOCTL_MODE_CURSOR, drm_mode_cursor_ioctl, DRM_MASTER),
drmModeMoveCursor
drmpublic int drmModeMoveCursor(int fd, uint32_t crtcId, int x, int y)
{
 struct drm_mode_cursor arg;
memclear(arg);
 arg.flags = DRM_MODE_CURSOR_MOVE;
 arg.crtc_id = crtcId;
 arg.x = x;
 arg.y = y;
return DRM_IOCTL(fd, DRM_IOCTL_MODE_CURSOR, &arg);
}

DRM_IOCTL_DEF(DRM_IOCTL_MODE_CURSOR, drm_mode_cursor_ioctl, DRM_MASTER),
drmModeSetCursor2
drmpublic int drmModeSetCursor2(int fd, uint32_t crtcId, uint32_t bo_handle,
 uint32_t width, uint32_t height, int32_t hot_x,
 int32_t hot_y)
{
 struct drm_mode_cursor2 arg;
 memclear(arg);
 arg.flags = DRM_MODE_CURSOR_BO;
 arg.crtc_id = crtcId;
 arg.width = width;
 arg.height = height;
 arg.handle = bo_handle;
 arg.hot_x = hot_x;
 arg.hot_y = hot_y;
return DRM_IOCTL(fd, DRM_IOCTL_MODE_CURSOR2, &arg);
}

DRM_IOCTL_DEF(DRM_IOCTL_MODE_CURSOR2, drm_mode_cursor2_ioctl, DRM_MASTER|DRM_UNLOCKED)

1.2 Implementation of Xorg Functions

The implementation of the cursor in Xorg is as follows:

hw/xfree86/drivers/modesetting/drmmode_display.c
drmmode_set_cursor_position
static void
drmmode_set_cursor_position(xf86CrtcPtr crtc, int x, int y)
{
 drmmode_crtc_private_ptr drmmode_crtc = crtc->driver_private;
 drmmode_ptr drmmode = drmmode_crtc->drmmode;
drmModeMoveCursor(drmmode->fd, drmmode_crtc->mode_crtc->crtc_id, x, y);
}

drmmode_show_cursor
static Bool
drmmode_show_cursor(xf86CrtcPtr crtc)
{
 drmmode_crtc_private_ptr drmmode_crtc = crtc->driver_private;
 drmmode_crtc->cursor_up = TRUE;
 return drmmode_set_cursor(crtc);
}

1.3 Whether to Use Hardware Cursor

ret = drmModeSetCursor2(drmmode->fd, drmmode_crtc->mode_crtc->crtc_id,
 handle, ms->cursor_width, ms->cursor_height,
 cursor->bits->xhot, cursor->bits->yhot);
/* -EINVAL can mean that an old kernel supports drmModeSetCursor but
 * not drmModeSetCursor2, though it can mean other things too. */
if (ret == -EINVAL)
 ret = drmModeSetCursor(drmmode->fd, drmmode_crtc->mode_crtc->crtc_id,
 handle, ms->cursor_width, ms->cursor_height);
/* -ENXIO normally means that the current drm driver supports neither
 * cursor_set nor cursor_set2. Disable hardware cursor support for
 * the rest of the session in that case. */
if (ret == -ENXIO) {
 xf86CrtcConfigPtr xf86_config = XF86_CRTC_CONFIG_PTR(crtc->scrn);
 xf86CursorInfoPtr cursor_info = xf86_config->cursor_info;
 cursor_info->MaxWidth = cursor_info->MaxHeight = 0;
 drmmode_crtc->drmmode->sw_cursor = TRUE;
 }

If the kernel’s IOCTL returns -ENXIO, then Xorg implements a software cursor.

drmmode_crtc->drmmode->sw_cursor = TRUE;

2: RK’s Cursor Hook Implementation

2.1 Function Implementation

int drmModeSetCursor(int fd, uint32_t crtcId, uint32_t bo_handle,
 uint32_t width, uint32_t height)
{
 /* Init log file */
 drm_get_ctx(fd);
 return drm_set_cursor(fd, crtcId, bo_handle, width, height);
}
int drmModeMoveCursor(int fd, uint32_t crtcId, int x, int y)
{
 return drm_move_cursor(fd, crtcId, x, y);
}
int drmModeSetCursor2(int fd, uint32_t crtcId, uint32_t bo_handle,
 uint32_t width, uint32_t height,
 int32_t hot_x, int32_t hot_y)
{
 return -EINVAL;
}

2.2 Handling of Hooks

This repository ultimately generates the library file libdrm-cursor.so.1.0.0, but the cursor implementation functions are in the libdrm so. Here, RK uses the ld preload configuration.

The method is to add the following in debian/libdrm-cursor.postinst:

echo /usr/lib/*/libdrm-cursor.so.1 >> /etc/ld.so.preload

The linking situation of Xorg is as follows, indicating that the hook has taken effect:

kylin@kylin:~$ ldd /usr/lib/xorg/Xorg | grep drm
 /usr/lib/aarch64-linux-gnu/libdrm-cursor.so.1 (0x0000007fb9a31000)
 libdrm.so.2 => /lib/aarch64-linux-gnu/libdrm.so.2 (0x0000007fb97d1000)

2.3 drm-cursor Runtime Log

 drm_get_ctx(485) atomicdrmAPIenabled
drm_get_ctx(518) maxfps: 60
drm_get_ctx(559) found 0 CRTC: 64(0) (1920x1200) preferplane: 0
drm_get_ctx(559) found 1 CRTC: 83(1) (0x0) preferplane: 0
drm_get_ctx(567) found 2 CRTCs
drm_get_ctx(616) foundplane: 58[primary]crtcs: 0x1 (ARGB)
drm_get_ctx(616) foundplane: 61[cursor ]crtcs: 0x1 (ARGB)
drm_get_ctx(616) foundplane: 65[overlay]crtcs: 0x1 (ARGB)
drm_get_ctx(616) foundplane: 68[overlay]crtcs: 0x1 (ARGB)
drm_get_ctx(616) foundplane: 80[primary]crtcs: 0x2 (ARGB)
drm_get_ctx(616) foundplane: 84[overlay]crtcs: 0x2 (ARGB)
drm_get_ctx(625) usinglibdrm-cursor (1.3.0~20211201)
drmModeSetCursor(1185) fd: 11 crtc: 64 handle: 0 size: 0x0
drm_crtc_bind_plane(692) CRTC[64]: usingcursorplane
drm_crtc_bind_plane(700) CRTC[64]: bindplane: 61
drm_set_cursor(1097) CRTC[64]: requestsettingnewcursor 0 (0x0)
drm_crtc_thread_fn(866) CRTC[64]: threadstarted
drm_crtc_thread_fn(924) CRTC[64]: setnewcursor 0 (0x0)
drmModeMoveCursor(1192) fd: 11 crtc: 64 position: 956,596
drm_move_cursor(1154) CRTC[64]: requestmovingcursorto (956,596) in (1920x1200)

The information indicates: 1. Parsing the configuration file, 2. Finding the CRTC supported by drm, 3. Finding the corresponding plane, 4. Configuring the specified plane as the cursor plane.

2.4 Kernel Runtime Log

To view the kernel drm cursor logs, you need to enable the relevant configuration macros for drm.

The sys node for drm is:
/sys/module/drm/parameters/debug

The corresponding optional values are as follows:

Bit 0 (0x01) will enable CORE messages (drm core code)
Bit 1 (0x02) will enable DRIVER messages (drm controller code)
Bit 2 (0x04) will enable KMS messages (modesetting code)
Bit 3 (0x08) will enable PRIME messages (prime code)
Bit 4 (0x10) will enable ATOMIC messages (atomic code)
Bit 5 (0x20) will enable VBL messages (vblank code) (int)

Since the cursor is implemented through the atomic API, leveraging the debugging of atomic can provide some useful information.

echo 0x10 > /sys/module/drm/parameters/debug # Enable drm ATOMIC debug information

The logs are as follows:

[drm:drm_atomic_state_init] Allocatedatomicstate 000000009ea89924
[drm:drm_atomic_get_plane_state]Added[PLANE:71:Cluster0-win0] 000000001d2516b5stateto 000000009ea89924
[drm:drm_atomic_get_crtc_state]Added[CRTC:85:video_port0] 000000002526233cstateto 000000009ea89924
[drm:drm_atomic_set_fb_for_plane]Set[FB:202]for[PLANE:71:Cluster0-win0]state 000000001d2516b5
[drm:drm_atomic_check_only]checking 000000009ea89924
[drm:drm_atomic_commit]committing 000000009ea89924
[drm:drm_atomic_state_default_clear]Clearingatomicstate 000000009ea89924
[drm:__drm_atomic_state_free]Freeingatomicstate 000000009ea89924

It can be seen that the default uses plane 71, crtc 85, fb202.

Check the information under sys to see if it corresponds normally:

1.cat /sys/kernel/debug/dri/0/framebuffer  | grep drm-cursor -B 1

framebuffer[202]:
 allocated by = drm-cursor[85]
2.cat /sys/kernel/debug/dri/0/state  | grep drm-cursor -B 3

plane[71]: Cluster0-win0
 crtc=video_port0
 fb=208
 allocated by = drm-cursor[85]
3.cat /sys/kernel/debug/dri/0/summary
Cluster0-win0: ACTIVE
win_id: 4
format: AB24little-endian (0x34324241)[AFBC]SDR[0]color_space[0]glb_alpha[0xff]
rotate: xmirror: 0 ymirror: 0 rotate_90: 0 rotate_270: 0
csc: y2r[0]r2y[1]cscmode[1]
zpos: 7
src: pos[0, 0]rect[64 x 64]
dst: pos[0, 0]rect[64 x 64]
buf[0]: addr: 0x0000000001825000pitch: 256 offset: 0

Thus, it can be confirmed that the hardware cursor is started normally, with a size of 64×64.

3: Testing and Debugging

3.1 Configuration File /etc/drm-cursor.conf

# debug=1
# log-file=
# hide=1 # hide cursors
# atomic=0 # disable atomic drm API
# max-fps=60
# allow-overlay=1 # allowing overlay planes
# prefer-afbc=0 # prefer plane with AFBC modifier supported
# num-surfaces=8 # num of egl surfaces to avoid edge moving corruption
# prefer-plane=65
# prefer-planes=61,65
# crtc-blocklist=64,83

debug=1  Default will output logs to /var/log/drm-cursor.log

log-file=/tmp/drm-cursor.log  Manually specify log file

hide=1  Whether to hide the mouse

atomic=1  This item is not useful, drm cursor defaults to use atomic API

max-fps=60  The display frame rate of the cursor pointer.

allow-overlay=1 Allow the use of overlay layers. If not set, it defaults to not using overlay, which means only using the cursor layer

prefer-afbc=0  Whether to use the arm's afbc function overlay layer

num-surfaces=8 Number of off-screen buffers created by eglCreateWindowSurface

prefer-plane=65  Default set plane

prefer-planes=61,65 Optional multiple planes

crtc-blocklist=83  Prevent which crtc from allowing the cursor layer

3.2 How to Choose Plane and CRTC

Install libdrm-tests

modeprint rockchip

You can see that the current display is hanging on crtc 64.

modetest -M rockchip -p  Check plane

You can see that plane 61 is mounted on the cursor layer of crtc 64.

Test hardware cursor verification:

systemctl stop lightdm

modetest -M rockchip -s 91@64:1920x1200 -C

If a color map appears, it indicates that libdrm can normally use the hardware cursor.

Specify plane:

modetest -M rockchip -P 61@64:1920x1200

If a color map appears, it indicates that libdrm can normally specify the plane.

4: Specific Applications

First, you need to enable the hardware cursor layer in the kernel, find the vp where you need to configure the hardware cursor, and add cursor-win-id. For example, my display is on vp2.

Using the DrmCursor Layer in Rockchip
Using the DrmCursor Layer in Rockchip

Then you need to install the libdrm-cursor software and libdrm-tests software.

First, use the libdrm-tests software to check the actual CRTC layer in use.

Command: modetest

Using the DrmCursor Layer in Rockchip
Using the DrmCursor Layer in Rockchip
Using the DrmCursor Layer in Rockchip
Using the DrmCursor Layer in Rockchip

Here you can see that 68, 85, and 102 are unused CRTCs, and the actual CRTC connected to fb is 133.

So we need to configure the software to block the first three CRTC layers so that the mouse layer can correctly match to 133.

Command: <span>vim /etc/drm-cursor.conf</span>

Before modification:

Using the DrmCursor Layer in RockchipAfter modification:

Using the DrmCursor Layer in Rockchip

Here we have enabled the hardware cursor layer, so there is no need for the overlay layer, which we commented out, and opened the crtc-blocklist option to block the other unused crtc layers. Restarting the machine will allow the hardware cursor to function normally. The specific id of the crtc layer needs to be determined based on the actual situation using the modetest command.

5: Reference Links

《Linux DRM Developer's Guide》https://landley.net/kdocs/htmldocs/drm.html

《Linux Kernel Doc》 https://www.kernel.org/doc/

《DRM-wikipedia》 https://en.wikipedia.org/wiki/Direct_Rendering_Manager
《Linux GPU Driver Developer's Guide》https://01.org/linuxgraphics/gfx-docs/drm/gpu/index.html

《DRM-KMS Doc》 https://kernel.readthedocs.io/en/latest/gpu/drm-kms.html#mode-setting
《KMS简介》 https://events.static.linuxfound.org/sites/events/files/slides/brezillon-drm-kms.pdf

《EGL Reference Pages》https://www.khronos.org/registry/EGL/sdk/docs/man/

《Linux User and Programmer's Manual》https://www.systutorials.com/docs/linux/man/3-drmModeGetResources/

《Nvidia drm api参考》https://docs.nvidia.com/jetson/l4t-graphics/group__direct__rendering__manager.html

《Linux How to的KMS简介》https://www.linuxhowtos.org/manpages/7/drm-kms.htm

《freedesktop.org的kms文档》https://dri.freedesktop.org/docs/drm/gpu/drm-kms.html

For more articles, please follow the public account: Kirin Embedded

Related posts

  1. BLE DTM Testing: The “Health Check Expert” for Bluetooth RF Performance
  2. RK3588 Course Series: LPDDR4 Memory Control Circuit Design and Parameter Optimization
  3. Differences Between RK3576 and RK3588
  4. RK3568 Security Backend Solution Enables Comprehensive Hardware Upgrade for NVR/XVR
  5. Introduction to Rockchip Electronics Co., Ltd. and Its Chips RK3288 and RK3588
  6. RK3288 Vertical Face Recognition Management Module
  7. RK3188 Network HD Decoder Board Customization
  8. Detailed Advantages of Rockchip’s New RK3506 Chip Platform
  9. How to Implement Facial Fatigue Detection Using RK3576
  10. In-Depth Review of Rockchip RK3328 Ultra HD TV Box: Nearly 3x Speed Boost
  11. RK3566 Harmony Core Board/Development Board Overview
  12. Comprehensive Upgrade of Power Applications: All-Round Solutions from Rockchip
  13. A Brief History of PCB Development
  14. Fully Independent and Controllable Supply Chain: Dongfeng’s Automotive Grade MCU Chip DF30 Set for Mass Production Next Year
  15. NTT Launches AI Inference Chip for Low-Power Video Processing at the Edge
  16. Understanding the Tg Value of PCB Materials
  17. Troubleshooting EDP Display Connection Issues with modetest -M rockchip
  18. YOLO v8.3.107 Released! Rockchip RKNN Fixes, OpenVINO Compatibility Upgrades, Enhancing Developer Efficiency!
  19. RKLLama: LLM Server and Client for Rockchip 3588/3576 Chips

Leave a Comment