Background
IPC (Internet Protocol Camera) is a type of camera that transmits video data over the network, widely used in monitoring, security, and video conferencing scenarios. Acquiring the video stream from an IPC camera is an important step in achieving remote monitoring and video management. By correctly configuring the network connection, protocol selection, and URL construction, real-time video data can be effectively obtained from IPC cameras.
Currently, the RK3588’s VPU (Video Processing Unit) supports up to 8K@30fps H.264 decoding and 8K@60fps H.265 decoding. This means the RK3588 can handle multiple video streams simultaneously, making it very suitable for video surveillance systems that require multi-channel video decoding.
Using MPP for Hardware Decoding
The RK3588’s VPU supports MPP (Media Process Platform), which is a general media processing software platform from Rockchip, supporting various mainstream chip platforms from Rockchip, including:
RK3288 series, RK3368 series, RK3399 series, RK3588 series
RK30xx series, RK312x series, RK322x series, RK332x series
RV1109 / RV1126 series
MPP abstracts the complexity of low-level processing for application software, providing a unified video media processing interface (Media Process Interface, abbreviated as MPI), supporting various video codec formats.
Taking H.264 as an example, the basic steps for H.264 hardware decoding using MPP include:
1. Initialize the MPP environment: create an MPP context and initialize it.
2. Configure decoding parameters: set decoder parameters such as resolution, encoding format, etc., through the MPI interface.
3. Input video stream: input the H.264 encoded video stream into the decoder.
4. Obtain decoded frames: retrieve the decoded video frames from the decoder for subsequent processing or display.
5. Release resources: release MPP resources after decoding is complete.
Using FFmpeg to Pull IPC Camera Data
FFmpeg is a very powerful multimedia processing tool that supports processing video and audio in various formats. By combining FFmpeg with RK3588’s MPP, efficient video stream processing can be achieved. For example, FFmpeg can be used to pull RTSP[2] video streams and then perform hardware decoding through MPP.
This combined approach can significantly improve the efficiency and performance of video stream processing, especially in scenarios requiring multiple video streams. Through the flexibility of FFmpeg and the hardware acceleration capabilities of MPP, a complete process from video acquisition, decoding to display or further processing can be achieved.
Performance Verification
Verification Scenario One
We conducted streaming tests based on the Rockchip RK3588 platform with multiple IPC cameras. This platform includes multiple IPC cameras with a resolution and frame rate of 4K@25fps, which output H.264 data via the RTSP protocol. We then used the RK3588 to acquire data from multiple IPC cameras and perform hardware decoding, resulting in the final BGR image. Test results indicate that four streams of 4K@25fps H.264 streaming decoding can be achieved on the RK3588.
Figure 1 Streaming decoding performance of one channel
Figure 2 Streaming decoding performance of four channels
Figure 3 Streaming decoding performance of six channels
Table 1 Performance testing of IPC camera hardware decoding on RK3588
Verification Scenario Two
We conducted hardware decoding tests on the RK3588 platform in our research group, using locally stored H.264 and H.265 video files with resolutions of 1920*1080, 3840*2160, and 7680*4320, testing both decoding alone and decoding to BGR image performance.
Figure 4 Performance test of H.264 decoding only
Figure 5 Performance test of H.264 decoding to BGR image
Figure 6 Performance test of H.265 decoding only
Figure 7 Performance test of H.265 decoding to BGR image
Table 2 Hardware decoding performance testing of H.264 and H.265 video files on RK3588
Conclusion
Based on the Rockchip RK3588, we conducted a simple evaluation of the hardware decoding performance in two scenarios. We primarily tested the decoding performance for H.264 and H.265 formats, and verified that four streams of 4K@25fps H.264 streaming decoding can be achieved on the RK3588, effectively meeting the demands of practical applications.
References
[1] Qin Ling, Wang Yu Jian, Li Dong Xin, et al. Major technical characteristics and application prospects of the video coding standard H.264[D]. 2004.
[2] Sullivan G J, Ohm J R, Han W J, et al. Overview of the High Efficiency Video Coding (HEVC) Standard[J]. IEEE Transactions on Circuits & Systems for Video Technology, 2013, 22(12):1649-1668.
[3] Schulzrinne H, Rao A, Lanphier R. Real-time streaming protocol (RTSP)[R]. 1998.
Disclaimer：Copyright belongs to the original author. Please contact us for reprint permissions.
ABOUT
The Southeast University Spark Lab research group focuses on edge artificial intelligence, conducting “practical” research to enable a hundred models and thousands of states, assisting various industries.
Communication & Cooperation: [email protected]
