How to Implement a Deep Learning Intelligent Gateway on Raspberry Pi

This project uses deep learning to turn the Raspberry Pi 3 into an intelligent gateway. The device does not need to be connected to the internet; all training and debugging are done locally on the Raspberry Pi 3. At DT42, we believe that applying deep learning to convenient devices is a big trend for the future. This not only saves costs for data transmission and storage but also allows devices to respond to events displayed in images or videos without needing to connect to the cloud.

Project address: https://github.com/DT42/BerryNet

Figure 1

Figure 1 shows the software architecture of the project, where we use Node.js, MQTT, and AI engines for deep learning on images or video frames. So far, the system is supported by two AI engines: the classification engine and the object detection engine. Figure 2 shows the difference between classification and object detection.

Figure 2

One application of this intelligent gateway is to monitor places of interest using a camera. As shown in Figure 3, it displays the analysis results from the DT42 office camera. These video frames are captured by an IP camera and sent to the AI engine. The output results from the AI engine will be displayed on the control panel. We are working on email and IM notification interfaces, so when the IP camera captures a target (like a dog), the system will notify the user via email or other means.

Figure 3

AI Engine

The currently supported AI engines utilize research from the following projects:

• TensorFlow

• Darkflow

• Darknet

The classification model currently supported by the system is Inception v3, and the object detection model is TinyYOLO.

Installation

$ git clone https://github.com/DT42/BerryNet.git

$ cd BerryNet

$ ./configure

Start and Stop BerryNet

BerryNet is controlled by the system (systemd), and you can control BerryNet via berry-manager:

$ berrynet-manager [start | stop | status | log]

Configuration

All configurations are in config.js:

• Select AI engine

• Currently, there are two types of AI engines: object classification and object detection.

• Configure the access interface for IP camera snapshots

• MQTT topics

Control Panel

Open the control panel on Raspberry Pi 3 (via touchscreen)

Open the browser and enter the URL: http://localhost:8080/index.html#source=dashboard.json

The default configuration file for the control panel will be loaded.

Open the control panel on any computer’s browser

Open the browser and enter the URL: http://:8080/index.html#source=dashboard.json

Click on the data source and change the MQTT broker’s IP address to the gateway’s IP address.

For more detailed control panel configuration information, please refer to the freeboard project: https://github.com/Freeboard/freeboard

Provide Image Input

Capture images through the Pi camera

$ mosquitto_pub -h localhost -t berrynet/event/camera -m snapshot_picam

Capture images through the configured IP camera

$ mosquitto_pub -h localhost -t berrynet/event/camera -m snapshot_ipcam

Provide local images

$ mosquitto_pub -h localhost -t berrynet/event/localImage -m <image_path>

Discussion

If you have any questions, suggestions, or ideas about this project, you can discuss it on the project’s Google forum: https://groups.google.com/a/dt42.io/d/forum/berrynet.

This article is compiled by Machine Heart, please contact this public account for authorization to reprint.

✄————————————————

Join Machine Heart (full-time reporter/intern): [email protected]

Submissions or seeking coverage: [email protected]

Advertising & Business Cooperation: [email protected]