Live Streaming with Raspberry Pi 3: H.264 Hardware Encoding Implementation

Let’s briefly introduce the composition of the entire system: 1. Raspberry Pi captures camera video -> 2. Hard-encoded into FLV format -> 3. Streams via RTMP to the local NGINX-RTMP server -> 4. Player plays the live video.

In fact, most of the functionalities have already been implemented in previous posts, and we just need to make a few minor adjustments to connect everything, and our live streaming capability will be realized. Of course, we need better efficiency, so we will use hardware encoding, which will speed things up a lot, saving precious CPU resources on the Raspberry Pi and being more energy-efficient and environmentally friendly.

The previous posts have already compiled FFmpeg with H.264 hardware encoding and decoding. Here, for streaming, we need the hardware encoding feature. In the last post, “Playing with Raspberry Pi 3 + Using FFmpeg Command Line Tools and Video Recording,” we learned that the default FFmpeg encoding uses the x264 encoder. So how do we specify the use of the hardware encoder? We can directly specify it. In the previous FFmpeg compilation, we discussed that the Raspberry Pi’s VideoCore IV hardware encoding functionality has been exposed to us through the OMX interface, so we can directly specify the h264_omx encoder, and FFmpeg can call it itself. It’s really convenient!

In the last post, we already implemented recording files locally, but how do we stream to the NGINX-RTMP server? Here we have some new issues. Since the RTMP protocol is designed for Flash, if we want to stream to the RTMP server, we need to record the video in FLV format, and the output cannot be a filename anymore, but rather the RTMP server’s URL. Previously, we configured NGINX to realize RTMP live streaming functionality, and the live address follows the same logic as VOD: rtmp://ip/live/stream_name. FFmpeg’s command line functionality is very powerful, allowing us to achieve most functionalities without programming. Some people even directly integrate the FFmpeg executable into their projects and call FFmpeg to create processes through parameters to achieve the desired functionality. For instance, some servers need to transcode users’ uploaded videos in real-time, which can be done using FFmpeg command line.

Let’s directly look at the images: . The second red box highlights the difference between RTMP streaming and recording files. ‘picam’ is the stream name specified during streaming, and the player needs to match this link. The first red box specifies the OMX hardware encoding acceleration, and the green box shows that FFmpeg calls Broadcom’s OMX encode. The black box shows the real-time streaming status. From the image, we can see that it is streaming stably at 15fps. Due to the USB camera’s USB speed limitations, the fps may decrease at higher resolutions. At that time, 720p at 10fps was basically no problem. The Raspberry Pi 3 really surprised me. If we switch to a CSI interface camera, both fps and resolution can be set very high, and I believe that using hardware encoding would also be completely fine.

After completing the streaming, how do we watch the live stream? It’s very simple; just find a player that supports RTMP. FFplay itself also supports playback; simply execute ffplay -i rtmp://192.168.15.101/live/picam, which is very convenient. Of course, there are other more user-friendly players available, such as VLC, which can be used across platforms, whether on mobile, Windows, Linux, or Mac.

Here’s an effect image; note that the CPU usage on the Raspberry Pi desktop is only 4%! Although it looks a bit exaggerated, the average CPU usage is around 5%. This includes video capture, encoding, streaming, and the NGINX server working simultaneously, resulting in a very satisfying outcome. In fact, the performance of the Raspberry Pi 3 has exceeded my expectations!

After streaming, how do we access the RTMP server on the Raspberry Pi from outside? Don’t worry; there is a way, although it requires a bit more effort. There are a few prerequisites: telecom broadband, router dialing, and the router’s WAN IP must match the public IP. If these conditions are met, accessing the home video over the public network can be resolved in no time; you just need to set up a port mapping.

How to do it? Let your Raspberry Pi directly connect to the broadband dial-up router at home, then enter the router settings interface, find the port mapping settings, and add a mapping from WAN port 1935 TCP to the Raspberry Pi’s IP 1935 TCP port. Then you can access the Raspberry Pi’s camera using the link rtmp://public_ip/live/picam from anywhere.

Here’s a screenshot of my port mapping settings (I’m using the Pandora router firmware; please search for port mapping settings for other routers).

This concludes the article. ————————- Cutting here ————————– This article implements hardware-encoded RTMP live streaming through the FFmpeg command line. In most projects, it’s not necessarily done this way. For example, on a mobile phone, some functionalities cannot be directly used due to system restrictions, like camera capture, which cannot read the v4l2 driver file directly. In such cases, you would need to use a series of FFmpeg APIs for secondary development. After this evaluation, I will open another post to discuss some secondary development topics.

PS. To check whether the WAN IP and public IP are the same, see the image below. The left shows the router’s WAN status, and the right shows the public IP queried from ip.cn, which can be seen to be consistent.

PS+. Friendly reminder: If you find that the WAN IP is not a public IP and you are directly dialing the telecom broadband with the router, you can contact the telecom customer service at 10000 to have them change it to a public IP. Generally, it will take effect quickly. Putting aside the high cost of telecom broadband and the serious imbalance in upstream and downstream bandwidth, I still want to give a thumbs up for telecom’s service.

Welcome to click to read the original text and communicate with the author.

Follow EEWORLD (Electronic Engineering World) WeChat public account: reply “submission”, your original work will have the opportunity to appear in our WeChat push.

To communicate with more industry peers, please click to read the original text to log in to the EEWORLD forum.