The new year’s competition is in full swing~
To help everyone understand this event better, the editor has compiled excellent works from 2021 for learning and sharing.
We will share award-winning works every Monday, so remember to check back for the series~
Team Introduction
Participating Unit: Shenzhen University Team Name: Giants of Light Advisors: Zhong Shida, Yuan Tao Team Members: Feng Haogang, Pan Jiahao, Mu Haoze
Figure 1 Team Style
1. Project Overview
The COVID-19 pandemic has swept the globe, and effective mask-wearing can greatly reduce the risk of virus infection. This project developed a real-time face mask detection SoC based on the ARM Cortex-M3 processor and a deep learning accelerator, as shown in the figure. This SoC is aimed at crowded places such as shopping malls and subway stations, achieving high performance, high stability, and low power consumption for real-time face mask detection. Additionally, this SoC features social distance detection and Bluetooth wireless communication, which can assist the face mask detection system in specific application scenarios to enhance epidemic prevention and control.
Figure 2 System Communication Overview
2. System Architecture
This project uses the DesignStart Eval version of the Cortex-M3 processor provided by ARM as the central processing unit of the system. An AHB bus system was designed using the CMSDK tool. Based on a single-level AHB bus framework, the APB and AXI buses were extended through APB and AXI bridge connectors, forming an efficient bus framework for the SoC. By building an efficient bus system, the M3 processor is effectively interconnected with hardware accelerators, high-speed AHB peripherals, and low-speed APB peripherals, thereby realizing a SoC system with face detection, face mask detection, and social distance monitoring features, as shown in Figure 2. The operator can switch the system’s functional mode through the “function key.” The main functional components of the system include the ARM Cortex-M3 processor, two sets of hardware accelerators, AHB bus matrix, AXI interconnect module, APB bridge, DDR3 memory, camera module, on-chip memory module, HDMI display module, APB peripherals, GUI graphic interface, Bluetooth communication module, ultrasonic ranging module, etc. Among them, key modules such as the decision tree hardware accelerator, OV5640, and DDR3 are mounted on the AHB bus through the AXI bridge to achieve high-speed interconnection with the Cortex-M3 processor, while peripherals such as LED and Bluetooth serial modules are mounted on the APB bus. The overall system framework is shown in Figure 3.
Figure 3 System Bus Framework
3. Function Introduction
Control Function: In this project, the Cortex-M3 processor communicates collaboratively with two sets of hardware accelerators, high-speed AHB peripherals, and low-speed APB peripherals through a self-built bus system. The M3 processor uses a logic controller to invoke the camera, hardware accelerators, and other peripheral functional modules. During the detection of each frame of image, the M3 processor first controls the camera for image acquisition. Each frame of collected image is stored in DDR3, after which the processor wakes up the ‘face detection accelerator’ and ‘face mask detection accelerator’ to read and detect the image data stored in memory. The detection results are then processed by the processor for image post-processing (drawing boxes), and the final results are displayed in real-time through the monitor and GUI. Furthermore, to ensure the SoC’s operational efficiency, part of the control tasks is delegated to the logic controller on the APB bus, responsible for the distribution control of data flow information and address information.
Face Mask Detection Function: During the face mask detection process, the ‘face detection accelerator’ and ‘face mask detection accelerator’ will read the image data and weight parameter files that need to be detected from DDR3 and on-chip BRAM storage units, respectively. As shown in Figure 4, these two sets of parallel hardware accelerators will simultaneously perform window detection on the read images. As shown in Figure 5, the startup process of the two sets of accelerators does not affect each other and operates independently, both controlled by interrupt instructions from the M3 processor. Ultimately, the windows that successfully pass through all decision tree units will be output to the processor as objects of interest.
Figure 4 Parallel Window Detection
Figure 5 Parallel Detection of Two Sets of Accelerators
Detection Result Output Function: As shown in Figure 6, the images processed by the M3 processor and two sets of hardware accelerators can be output to the display through the HDMI output module. Among them, faces not wearing masks are identified with a ‘green box’, while faces wearing masks are identified with a ‘red box’. Additionally, considering situations where video display is unnecessary, detection results can also be output through peripherals such as serial ports, LED lights, and buzzers. To achieve more convenient human-computer interaction, this project also designed a dedicated graphic interface (GUI), as shown in Figure 7. This interface is designed using Python based on Qt to display real-time information such as the number of faces detected by the system, the number of faces not wearing masks, the system’s real-time average frame rate, and the distance measured by the ultrasonic ranging module.
Figure 6 Display Output
Figure 7 GUI Graphic Interface
Social Distance Check Function: Maintaining a safe social distance during the pandemic is an effective means to avoid infection risks. To address this scenario requirement, we simulated a situation where the product is miniaturized for personal wearing and added a social distance detection function, allowing the product to freely switch between face mask detection and social distance detection using the “function key.” In social distance detection mode, when the system detects a face not wearing a mask entering a dangerous social distance (less than 1 meter), the LED and buzzer in the system will be activated to alert the user to stay away from the unmasked individual. At the same time, this data will also be synchronized to the GUI for real-time display.
Bluetooth Wireless Communication Function: To avoid the inconvenience of wired data transmission, this project also added a Bluetooth wireless communication module. Through master-slave communication between two Bluetooth modules connected to an FPGA development board and a PC, the number of faces detected without masks, the number of faces wearing masks, the system’s average frame rate, and detection distance can be transmitted to the PC via the Bluetooth module, and then displayed in real-time through the GUI on the computer, enhancing the flexibility of system deployment and better adapting to market demands.
4. System Advantages
-
Unmanned face mask detection system
-
Social distance detection function for close range
-
DL-based parallel window accelerator
-
Low latency high-performance system bus architecture
-
Low energy consumption and high stability and efficiency
-
Triple reminders of display, buzzer, and LED
-
Synchronization for managers
-
Supports Bluetooth wireless data transmission*
5. Competition Experience and Sharing
With the development of the Internet of Things and edge intelligence, an increasing number of AI models will be deployed on edge devices. How to execute intelligent algorithms with low power consumption has become a hot topic in research and product development. By participating in this ARM Cup competition, we further increased our design experience with the on-chip system while personally experiencing the challenges of designing and deploying intelligent algorithms on edge systems. The entire project is divided into two parts: SoC construction and intelligent algorithm model design. The coordination and integration of these two routes require continuous communication and cooperation among team members. Under the guidance of our team instructors and through teamwork, we successfully completed the design of the entire system and achieved first place nationally. This competition will also serve as valuable experience for our future research and work.
Recommended Reading
【2021 Competition Work Sharing】Issue 1 | Moving Target Detection and Tracking System Based on Cortex-M3 Core SOC
END
About Anxin Education
Anxin Education is an innovative education platform focused on AIoT (Artificial Intelligence + Internet of Things), providing comprehensive AIoT education solutions from primary and secondary schools to higher education institutions. Anxin Education, relying on ARM technology, has developed the ASC (Arm Smart Connectivity) curriculum and talent training system, which has been widely applied in industry-university-research cooperation in higher education and STEM education in primary and secondary schools, aiming to cultivate talents in the field of smart connectivity that meet the demands of the times.