Student Xin Zhe from the research group successfully ported the deep learning gait recognition algorithm GaitSet to the Allwinner V853 development board as part of his undergraduate thesis titled “Research on Gait Recognition Based on Embedded Systems”. This study was tested on the CASIA-B dataset, achieving a gait recognition accuracy of 94.9% under normal walking conditions, and recognition accuracies of 87.9% and 71.0% under conditions of walking with a backpack and wearing a coat, respectively.
Gait Recognition
Research Background
Gait recognition, as an emerging biometric method, has advantages over facial recognition and fingerprint recognition, such as adaptability to environments and resistance to disguise. Moreover, gait recognition can be applied not only in ordinary scenarios but also to fill application gaps in special situations. Currently, researchers both domestically and internationally have gradually applied gait recognition to identity verification, security monitoring, and disease prediction.
At present, the application of embedded systems in the field of gait recognition mostly involves using embedded systems for image acquisition, with image data transmitted to a PC or server for model inference. The NPU, or Neural Processing Unit, is a specialized processor designed for neural network computations. The NPU can quickly complete data processing, transmission, and backpropagation operations for neural networks, thereby accelerating the computation process of neural network models. Porting the aforementioned gait recognition technology to the embedded system’s NPU and using the embedded system’s camera for real-time image acquisition and gait recognition will become an important application of gait recognition.
System Architecture
The gait recognition system designed in this paper is built on the Allwinner V853 development board, fully utilizing the onboard peripherals, CPU, and NPU to achieve a real-time gait recognition system on an embedded system.
The V853 is a next-generation high-performance, low-power SoC launched by Allwinner for the intelligent vision field. It integrates dual CPUs: Arm Cortex-A7 and RISC-V E907, and features an NPU with a maximum computing power of 1T. It uses Allwinner’s self-developed Smart video engine, supporting up to 5M@25fps H.265 encoding and 5M@25fps H.264 encoding and decoding, while also integrating a high-performance ISP image processor to provide professional-grade image quality.
Figure 1-1 V853 Chip
Figure 1-2 V853 Development Board
Specifically, the deep learning algorithm used in the system is trained on a PC, and the resulting Pytorch model is converted into an NB model that can run on the V853 NPU using a model conversion tool. The model inference is performed on the NPU. The overall operation process of the system is divided into four main parts: preprocessing, model inference, post-processing, and UI display. Most of the operators of the deep learning algorithm used in this system are inferred on the onboard NPU, while a small number of operators are computed on the onboard CPU. Both the preprocessing and post-processing processes are performed on the onboard CPU, using the OpenCV and Eigen computation libraries, respectively. The preprocessing extracts the gait contour from the video captured by the onboard camera, crops and stitches it to serve as input data for the NPU model, while the post-processing performs supplementary calculations on the output data obtained from the NPU model and conducts feature comparisons for identity verification. The UI display is implemented through an application generated by Qt.
Figure 1-3 Gait Recognition System Construction Process
Figure 1-4 Overall Process of Gait Recognition Algorithm
Test Results
This study tested the gait recognition accuracy of the NB model on the CASIA-B dataset. CASIA-B is a large-scale, multi-view gait recognition dataset that contains 124 samples, each with 10 gait sequences, divided into 6 normal walking sequences (NM), 2 sequences of walking in a long coat (CL), and 2 sequences of walking with a backpack (BG). CASIA-B emphasizes changes in perspective, with each walking sequence containing 11 different angles. 74 samples from the dataset were used as training samples, while the remaining 50 samples were used as test samples. In the test set, the first 4 normal walking sequences of each sample were used as the gallery set. To study the system’s performance under different human silhouettes, three probe sets were defined: the last 2 sequences of normal walking, 2 sequences of walking in a long coat, and 2 sequences of walking with a backpack. Considering the impact of angles on recognition performance, this study conducted separate tests at each angle to verify the recognition accuracy under different perspectives.
The following table was created based on the above test data, containing the recognition accuracy data of the NB model used in this paper’s gait recognition system GaitCircle and the GaitSet model under the same conditions. Here, NM indicates normal walking, BG indicates walking with a backpack, and CL indicates walking in a coat.
In addition to testing the gait recognition accuracy, this study also tested the real-time performance of gait recognition. For single-person gait recognition, the preprocessing speed reached 58ms per frame, the model inference runtime was only 77ms, and the post-processing runtime was 0.73s.
Finally, this study also conducted real-time recognition tests, which involved using the camera on the V853 development board to record video and perform real-time gait recognition to output pedestrian identities. Before testing, the gait recognition program was set to auto-start in the V853 Tina Linux environment, mainly by adding the command to run the gait recognition program in the /etc/profile file. The following video shows the process of real-time gait recognition testing, where the gait features of 20 individuals have been pre-entered into the gait information database.
Previous Recommendations
01
Thanks to Mr. Liu Jianhua and Mr. Li Long for donating the embedded development board.
02
Student Yue Zhong’s reflections on teaching: sowing the seeds of dreams on the earth.
03
Can this style of code still be maintained? Can it be taken over?
Follow Us
WeChat ID | bitpmcrg
Beijing Institute of Technology Communication Research Group
Mobile Communication Research Group, Beijing Institute of Technology