Last issue recap: (Click here to jump to the previous issue)

This issue will introduce how to run the Mnist Demo (handwritten digit recognition) on the RT-Thread operating system, allowing for verification of your own handwritten digits.

• System: Windows | Ubuntu 18.04
• Board: STM32 H743ZI NUCLEO
• RT-Thread runtime environment
• MDK 5
• Github: https://github.com/Lebhoryi/Edge_AI/tree/master/Project2-Mnist

How to run Mnist on RT-Thread:

1. Pull Mnist_CMSIS or Mnist_CMSIS.7z from Github to local, Mnist_CMSIS is 520M, it is recommended to download the compressed package, only 66.4M

2. Run method, choose one:

• Scons
• MDK5 Compile

CMSIS + RT-Thread inference success interface

The folder on Github already contains the CMSIS packages required for the experiment, download to run, and the newly created project needs to enable the CMSIS package in RT-Thread’s Menuconfig.

1. PC Training Model

File: mnist.ipynb

1.1 PC Environment

• Tensorflow: 2.3.0-dev20200515
• Numpy: 1.16.4
• Keras: 2.2.4-tf

1.2 Dataset

File: ./data/mnist.npz

The MNIST dataset consists of 60,000 (training set) + 10,000 (test set) handwritten characters, each image size is , dataset manual download address http://yann.lecun.com/exdb/mnist/ .

1.3 Network Structure

• Two layers of convolution + one fully connected layer

1.4 Training Model & Validation Model

File: ./model/mnist.h5

• Training model

• Validate the accuracy of the training model

• Save weight file

 1# save weights
 2model.save_weights(model_path / 'model_weights.h5')
 3
 4# load weights
 5model.load_weights(model_path / 'model_weights.h5')
 6
 7model.compile(optimizer='adam',
 8             loss='sparse_categorical_crossentropy',
 9             metrics=['accuracy',])
10loss, acc = model.evaluate(x_test, y_test)
11print("Restored model, accuracy: {:5.2f}%".format(100*acc))

1313/313 [==============================] - 1s 2ms/step - loss: 0.1226 - accuracy: 0.9651
2Restored model, accuracy: 96.51%

Ok, the model has been trained, but the preparation work is not finished yet, please continue reading.

1.5 Others

The ultimate goal is to successfully infer (test) the trained model on the RT-Thread system.

The method adopted in this experiment is CMISIS + RT-Thread, (requires certain deep learning background knowledge), the steps are as follows:

1. Convert weight data to int type and save

2. Use the CMSIS NN library to reproduce the neural network, the file is a .c type file

3. Import weight files and test samples

4. Inference success

2. Select CMSIS Software Package

Prerequisite: The runtime environment required for RT-Thread has been installed.

 1# windows
 2> pkgs --upgrade
 3# Enable CMSIS
 4> menuconfig
 5> pkgs --update
 6> scons --target=mdk5
 7
 8# linux
 9# If unable to enable, please execute
10# (base) Mnist_CMSIS[master*] % source ~/.env/env.sh
11(base) Mnist_CMSIS[master] % pkgs --upgrade 
12(base) Mnist_CMSIS[master] % scons --menuconfig
13(base) Mnist_CMSIS[master*] % pkgs --update

3. Use CMSIS Software Package to Generate Network Structure

By calling the CMSIS API, implement the network structure, this step requires certain deep learning basics.

In addition, during the reconstruction process, int is used instead of float.

This project implements a method for passing data, currently a basic method, only defining an array of size 784 in the main.c file to store the 28*28 size handwritten digit image, the format requirement is Int type. The specific code for reading image data is implemented in mnist.ipynb.

Since the input is a simple 28*28 = 784 (one-dimensional) array, it can support custom handwritten digit recognition verification. It is recommended to first perform custom handwritten digit recognition verification in mnist.ipynb.

Those interested can read the source files, no changes were made to other files.

• ./Mnist_CMSIS/applications/main.c
• ./Mnist_CMSIS/applications/mnist_parameters.h

4. Compile & Flash

• Windows (recommended using MDK)

  MDK one-click compilation and flashing, observe output through Putty.

• Linux (recommended using Scons)

  Scons compilation, flash using STM32 Cube Programmer, observe output through minicom.

Success interface:

5. FAQ

5.1 CMSIS + RT-Thread Cannot Find arm_math.h

Solution:

Windows:

1. Check DSP switch

2. Add macro definitions

USE_STDPERIPH_DRIVER,ARM_MATH_CM4,__CC_ARM,__FPU_PRESENT, ARM_MATH_DSP

• Linux:

1. First solve the issue of not finding math.h

   In the ./Mnist_CMSIS/packages/CMSIS-latest/SConscript file, line 15, manually add DSP, add:

   CPPPATH = CPPPATH + [cwd + '/CMSIS_5/CMSIS/DSP/Include']

   

2. Scons will report an error like this afterwards:

   

   

   Solution is as follows:

   In the ./Mnist_CMSIS/board/SConscript file, line 22, change to:

   CPPDEFINES = ['STM32H743xx','ARM_MATH_CM7','__FPU_PRESENT']

   

5.2 Scons Error

But the files already exist

Solution:

In the ./Mnist_CMSIS/SConscript file, change it to as shown in the picture.