LLSM: Low Latency and Low Bandwidth Streaming Solutions for Embedded Systems

With the rapid development of the Internet of Things and artificial intelligence, real-time video transmission has become increasingly important in embedded systems. Whether it is smart cameras, drones, or industrial monitoring devices, efficient and low-latency streaming solutions are required.The LLSM low-latency, low-bandwidth streaming transmission module launched by Huishi can achieve stable streaming transmission in a 500K bandwidth environment, with a latency of around 50ms (excluding data link).This result is based on our team’s comprehensive and meticulous latency testing!Currently, the typical transmission link for most streaming video is as follows:Camera imaging output -> Streaming transmission module processing -> Pull stream end displayThus, we approached this from these three aspects.

The testing equipment is as follows:

• Acquisition end: SDI camera, resolution 1080p@60HZ

• Processing end: Rockchip RK3588, with built-in hardware codec

• Pull stream end:

1. CPU: Intel Xeon E5-2650 v2 @ 2.6GHZ

2. GPU: Nvidia GeForce RTX 2060

3. Display: 2k@120HZ monitor, supports PiP (can receive two input sources simultaneously and display them in picture-in-picture mode)

4. Network environment: Local area network

Imaging latency testingTesting purpose: To compare the imaging latency of different cameras under the same resolution and frame rate conditions.Testing steps:

1. Connect the monitor to two input sources simultaneously and enable picture-in-picture mode.
2. One input source connects to the computer output
3. One input source connects to the camera, which is connected via an SDI to HDMI converter (this converter has a latency of about 10ms).
4. Run a high-frame-rate stopwatch program on the computer, with the camera aimed at the stopwatch program.
5. Use a high-frame-rate camera to take a photo of the screen, and the difference between the two stopwatches in the photo is the camera’s imaging latency.

Testing results:The test result for Camera A is shown in the figure; subtracting the converter latency, the camera latency is 23ms.The test result for Camera B is shown in the figure; subtracting the converter latency, the camera latency is 61ms.Overall latency testing under different protocolsTesting purpose: To compare the overall latency of the entire link when using different protocols under the same overall link conditions.Testing steps:

1. Use the camera with a latency of 23ms, connected to the RK3588 streaming module.
2. Run a high-frame-rate stopwatch program on the computer, with the camera aimed at the stopwatch program.
3. Run dedicated pull stream software on the computer for display.
4. Record the entire screen at high frame rate.
5. Use VLC to view the video frame by frame; the difference between the two stopwatches in the video is the overall latency.
6. Change the streaming protocol of the RK3588 streaming module and repeat the above steps.

Testing results:First, confirm that the recorded video is indeed 120Hz!The overall latency using the RTSP protocol is 100ms.The overall latency using the RTMP protocol is 108ms.The overall latency using the custom protocol is 60ms.Pull stream end latency testingTesting purpose: To compare the overall latency of the entire link when using different pull stream software under the same overall link conditions.Testing steps:

1. Same as the testing steps for the overall latency testing under different protocols.
2. Fix the streaming protocol to RTSP and use different pull stream software.

Testing results:Using dedicated pull stream software, the latency is 100ms.Using VLC for pull stream playback, the latency is 192ms.Most streaming video transmissions have a demand for low bandwidth and high quality. Common streaming protocols (RTSP, RTMP) transmit H264/H265 bitstreams, and the bandwidth required for transmission is related to the size of the encoded bitstream.Therefore, we compared the imaging quality by setting different bitrates and observed bandwidth fluctuations.RTSP 2M bitrateFor a 1080p@60HZ video stream, using the RTSP protocol, encoded in H264, and set to a bitrate of 2M, the effect and network bandwidth usage are shown in the following figures:From the above figures, we can see:Some parts of the image have become blurry; if the bitrate is lowered further, it will be unwatchable.The bitrate fluctuates significantly.Custom protocol 0.5M bitrateFor a 1080p@60HZ video stream, using a custom protocol and set to a bitrate of 0.5M, the effect and network bandwidth usage are shown in the following figures:From the above figures, we can see:The image is generally clear and viewable.The bitrate fluctuates little and is generally stable.FAQ

• How to reduce latency in streaming video transmission:

1. Use cameras with low imaging latency.
2. Use powerful CPUs and MCUs with strong ISP processing capabilities.
3. Use low-latency streaming protocols, or even proprietary protocols.
4. Use high-performance hardware codecs.
5. Use dedicated pull stream software.
6. Use a good network environment.
7. Use high-refresh screens, with a refresh rate at least twice that of the video frame rate.

• How to reduce bandwidth in streaming video transmission:

1. Use high-compression encoding standards (such as H265), or even proprietary protocols.
2. Downscale the video before encoding and transmission.

● Exploring the low-altitude economy: Seeking new growth momentum in low bandwidth and low latency

● RK3588 low-latency, low-bandwidth image transmission solution enhances remote unmanned device control capabilities

● The humanoid robot “dancing” at the Spring Festival Gala: The hard power of multiple unmanned devices collaborating

Click here for likes and views!