Continuing from the previous article
Can the NVIDIA Jetson Nano Handle 4K Cameras? Let’s Validate the Encoding Performance (Part 1)
Can the NVIDIA Jetson Nano Handle 4K Cameras? Let’s Validate the Encoding Performance (Part 2)
4K Video Measurement Results
Finally, here are the measurement results for the 4K video of the main subject.
① H.264 Encoding Performance
■ 4K (3840 x 2160) x264 (Software Encoding: 4 CPUs)
This is the measurement result of x264 video compression (software encoding) using the CPU (4 cores) of the Jetson Nano. The result is fps = 3.7. After all, software encoding is painful.
fps = 3.678 CPU = 100％, 98％, 86％, 100％
$gst-launch-1.0 -e v4l2src device=/dev/video1 ! 'image/jpeg,width=3840,height=2160,framerate=30/1' ! jpegdec ! queue ! videoconvert ! queue ! x264enc threads=4 ! h264parse ! qtmux ! filesink location=4K_h264SW4x.mp4 -e$mediainfo 4K_h264SW4x.mp4---省略---Bit rate : 193 kb/sNominal bit rate : 2 048 kb/sMaximum bit rate : 2 097 kb/s---省略---Frame rate : 3.678 FPSMinimum frame rate : 0.128 FPSMaximum frame rate : 31.250 FPS(Swipe left to see the complete code)
4K (3840×2160) nvv4l2h264enc (Hardware Encoding)
This is the measurement result of h264 video compression using the hardware encoder installed in the Jetson Nano. It hardly uses the CPU, and the hardware encoder runs at “396MHz”, but10.8fps!
So? Not 30fps? Yes, it won’t reach 30fps, the reason is that the “jpegdec” part decodes the MJPG received from the camera. This seems to be the bottleneck.
fps = 10.783 CPU = 30％, 40％, 15％, 54％
$gst-launch-1.0 -e v4l2src device=/dev/video1 ! 'image/jpeg,width=3840,height=2160,framerate=30/1' ! jpegdec ! nvvidconv ! nvv4l2h264enc ! h264parse ! queue2 ! qtmux ! filesink location=4K_h264HW.mp4 -e$mediainfo 4K_h264HW.mp4---省略---Bit rate : 1 438 kb/sMaximum bit rate : 20.7 Mb/s---省略---Frame rate : 10.783 FPSMinimum frame rate : 2.686 FPSMaximum frame rate : 31.579 FPS(Swipe left to see the complete code)
■ 4K (3840×2160) nvv4l2h264enc with nvjpegdec (Hardware Encoding)
In fact, the Jetson Nano is equipped with a hardware decoder for JPEG (NVJPG), although it is not mentioned in the specifications. It is used to decode the JPEG images received from the camera. I spent 6 days noticing this, but it didn’t work, and it took me 2 days to discover that the “io-mode = 2” option was used.
I did my best, but the measurement result was 17.8fps on the JTOP screen. I can confirm that I am using NVJPG correctly, but I cannot achieve 30fps.
fps = 17.849 CPU = 17％, 72％, 10％, 6％
$gst-launch-1.0 -e v4l2src device=/dev/video1 io-mode=2 ! 'image/jpeg,width=3840,height=2160,framerate=30/1' ! nvjpegdec ! 'video/x-raw' ! nvvidconv ! nvv4l2h264enc ! h264parse ! queue2 ! qtmux ! filesink location=4K_h264HW_nvjpeg.mp4 -e$mediainfo 4K_h264HW_nvjpeg.mp4---省略---Bit rate : 2 374 kb/sMaximum bit rate : 25.3 Mb/s---省略---Frame rate : 17.849 FPSMinimum frame rate : 2.686 FPSMaximum frame rate : 31.250 FPS(Swipe left to see the complete code)
■ 4K (3840×2160) nvv4l2h264enc with nvv4l2decoder (Hardware Encoding)
Shooting 4K video frustrated me, but when I asked a question on the NVIDIA forum, a kind person told me to treat the input from the camera as MJPG video instead of JPEG images, and then decode it using the NVJPG decoder!
The measurement result was 29.6fps!!️ I was close to 30fps😍
fps = 29.556 CPU = 31％, 14％, 24％, 18％
$gst-launch-1.0 -e v4l2src device=/dev/video0 io-mode=2 ! image/jpeg, width=3840, height=2160, framerate=30/1 ! nvv4l2decoder mjpeg=1 ! 'video/x-raw(memory:NVMM)' ! nvv4l2h264enc ! h264parse ! queue2 ! qtmux ! filesink location=4K_h264HW_nvv4l2decoder.mp4$mediainfo 4K_h264HW_nvv4l2decoder.mp4---省略---Bit rate : 3 915 kb/sMaximum bit rate : 30.4 Mb/s---省略---Frame rate : 29.566 FPSMinimum frame rate : 2.688 FPSMaximum frame rate : 31.579 FPS(Swipe left to see the complete code)
② H.265 Encoding Performance
■ 4K (3840 x 2160) x265 (Software Encoding: 1 CPU)
Similarly, in the case of H.265, the first measurement result is from x265 video compression (software encoding) using the CPU (1 core) of the Jetson Nano. The result is fps = 9.781. Well, this is tough on the CPU, isn’t it?
fps = 9.781 CPU = 100％, 4％, 11％, 6％
$gst-launch-1.0 -e v4l2src device=/dev/video1 ! 'image/jpeg,width=3840,height=2160,framerate=30/1' ! jpegdec ! queue ! videoconvert ! queue ! x265enc ! h265parse ! qtmux ! filesink location=4K_h265SW.mp4 -e$mediainfo 4K_h265SW.mp4---省略---Bit rate : 535 kb/sMaximum bit rate : 7 394 kb/s---省略---Frame rate : 9.781 FPSMinimum frame rate : 1.453 FPSMaximum frame rate : 31.250 FPS
■ 4K (3840×2160) nvv4l2h265enc (Hardware Encoding)
This is the measurement result when using the H.265 hardware encoder. The hardware encoding is effective, but the speed is 10fps. The JPEG decoding is the bottleneck.
fps = 10.716 CPU = 22％, 12％, 76％, 30％
$gst-launch-1.0 -e v4l2src device=/dev/video1 ! 'image/jpeg,width=3840,height=2160,framerate=30/1' ! jpegdec ! nvvidconv ! nvv4l2h265enc ! h265parse ! queue2 ! qtmux ! filesink location=4K_h265HW.mp4$mediainfo 4K_h265HW.mp4---省略---Bit rate : 1 438 kb/sMaximum bit rate : 14.9 Mb/s---省略---Frame rate : 10.716 FPSMinimum frame rate : 2.688 FPSMaximum frame rate : 31.250 FPS
■ 4K (3840×2160) nvv4l2h265enc with nvjpegdec (Hardware Encoding)
This is the measurement result of H.265 compression performance using both the NVJPG decoder and the H.265 encoder. The result is fps = 17.6. After all, nvjpegdec does not reach 30fps😭
fps = 17.551 CPU = 46％, 48％, 18％, 19％
$gst-launch-1.0 -e v4l2src device=/dev/video1 io-mode=2 ! 'image/jpeg,width=3840,height=2160,framerate=30/1' ! nvjpegdec ! 'video/x-raw' ! nvvidconv ! nvv4l2h265enc ! h265parse ! queue2 ! qtmux ! filesink location=4K_h265HW_nvjpeg.mp4 -e$mediainfo 4K_h265HW_nvjpeg.mp4---省略---Bit rate : 2 347 kb/sMaximum bit rate : 16.5 Mb/s---省略---Frame rate : 17.551 FPSMinimum frame rate : 2.941 FPSMaximum frame rate : 31.250 FPS
■ 4K (3840×2160) nvv4l2h265enc with nvv4l2decoder (Hardware Encoding)
This is the measurement result of using the nvv4l2 decoder with the NVJPG encoder and the H.265 encoder!️ The result is fps = 29.6!️ H.265 but correctly at 30fps😀
fps = 29.602 CPU = 19％, 16％, 9％, 10％ S
$gst-launch-1.0 -e v4l2src device=/dev/video0 io-mode=2 ! image/jpeg, width=3840, height=2160, framerate=30/1 ! nvv4l2decoder mjpeg=1 ! 'video/x-raw(memory:NVMM)' ! nvv4l2h265enc ! h265parse ! queue2 ! qtmux ! filesink location=4K_h265HW_nvv4l2decoder.mp4$mediainfo 4K_h265HW_nvv4l2decoder.mp4---省略---Bit rate : 3 949 kb/sMaximum bit rate : 19.1 Mb/s---省略---Frame rate : 29.602 FPSMinimum frame rate : 2.688 FPSMaximum frame rate : 37.975 FPS
Summary of Measurement Results
This is a summary of the measurement results for 4K video and full HD video verified on the actual machine under changed measurement conditions.
Conclusion
I tried to verify how much 4K video the NVIDIA Jetson Nano can handle using a 3840 x 2160 @ 30fps MJPG transmission and an actual machine camera. For full HD video, using the H.265 / H.265 hardware encoder is no problem!
In summary, for 4K video, if using both the H.264 / H.265 hardware encoder and the JPEG hardware decoder, there is no problem. However, for cameras transmitting MJPG, the process of decoding the obtained MJPG video is very cumbersome, and if you do not leave this process to the JPEG hardware decoder, you can only output about 17 fps.
However, any camera that supports RAW image transmission can stream directly to the H.264 / H.265 hardware encoder, so this issue will not occur. However, I could not find a camera that supports 3840 x 2160 @ 30fps and RAW transmission on Amazon Japan.
Therefore, if you use the Jetson Nano, you can encode 4K @ 30fps in H.264 / H.265!
