Embedded AI

AI implementation has always been a promising and emerging industry. My curiosity is quite strong, so I want to try anything related to embedded systems and AI. This series of articles will guide you step by step to deploy AI models on embedded platforms, porting them to the RT-Thread operating system, achieving your first step or even the nth step from a beginner!

Development Environment:

The subsequent development process will be based on the STM32H743ZI-Nucleo development board, and will use STM32CubeMX.AI tools. It can automatically generate embedded project engineering based on trained AI Models (limited to Keras/TF-Lite), including but not limited to MDK, STM32CubeIDE, etc. This tool is easy to use and suitable for embedded AI beginner development.

STM32CubeMX is a tool launched by ST to automatically create microcontroller projects and initialization code, suitable for all STM32 series products. Now its AI component can provide the function of converting AI models to embedded C code.

1. Preparation Work

1.1 Install Development Environment

The operating system I used is Ubuntu 18.04. The following development tools will be used in this experiment, and the installation process is very simple, with mature tutorials available online, which I will not elaborate on here. This tutorial is also applicable to the Windows environment, and the experimental steps are exactly the same.

• STM32CubeMx
• STM32CubeIDE
• STM32CubeProgrammer

Using STM32CubeProgrammer in the Ubuntu environment may encounter the following error:

After installation, executing the executable file in the bin folder under the installation package path in the terminal will report error: Could not find or load main class “com.st.app.Main”. At this point, just switch the default Open-JDK of Ubuntu to Oracle JDK, and below is a successful screenshot of switching to Oracle JDK:

1# Download JavaSE JDK compressed package from Oracle's official website
2$ sudo tar zxvf jdk-8u172-linux-x64.tar.gz -C /usr/lib/jvm
3# Register the downloaded JDK to the system
4$ sudo update-alternatives --install /usr/bin/java java /usr/lib/jvm/jdk1.8.0_172/bin/java 300
5# Switch JDK
6$ sudo update-alternatives --config java
7# Check JDK version
8$ java -version

1.2 Build a Minimal Neural Network on PC

First clone the following open-source repository to local:

• Github: https://github.com/Lebhoryi/Edge_AI/tree/master/Project1

• tf2_linear_regression.ipynb contains three different ways to build the network structure
• tf2_linear_regression_extension.ipynb contains different ways to train the model

• Sequence
• Functional API
• Subclassing

1INVALID MODEL: Couldn't load Keras model /home/lebhoryi/RT-Thread/Edge_AI/Project1/keras_model.h5, 
2error: Unknown layer: Functional

2. Use CubeMX AI to Generate Project

2.1 Open CubeMX

2.2 Install CUBE-AI Software Package

Open the Help menu, select Embedded Software Packages Manager, then select the latest version of the X-CUBE-AI plugin in the STMicroelectronics section, and after installation, click Close at the bottom right.

The following interface will appear:

Next, select the serial port for communication. Here, choose serial port 3, as this serial port is used for the virtual serial port of STlink.

2.3 Import AI Model into the Project

Next, we need to verify the converted embedded project on the development board. During this process, the CubeMX AI tool will automatically generate the embedded project based on the imported AI model, and burn the compiled executable file into the development board, and verify the running results through the virtual serial port of STlink. My system is Ubuntu, which does not support MDK, so here I choose to automatically generate the STM32CubeIDE project.

The successful verification interface is as follows:

2.4 Generate Project Engineering

In the previous step, we only verified the project results but did not generate the project source code. Next, we will generate the project engineering, as shown in the figure below:

The generated Project folder tree is as follows:

 1(base) #( 07/03/20@10:51上午 )( lebhoryi@RT-AI ):~/RT-Thread/Edge_AI@master✗✗✗
 2   tree -L 2 ./Project1 
 3./Project1
 4├── DNN  # CubeMX generated project path
 5│   ├── DNN.ioc  # CubeMX type file
 6│   ├── Drivers
 7│   ├── Inc
 8│   ├── Middlewares
 9│   ├── network_generate_report.txt
10│   ├── Src
11│   ├── Startup
12│   ├── STM32CubeIDE
13│   ├── STM32H743ZITX_FLASH.ld
14│   └── STM32H743ZITX_RAM.ld
15├── image  # Folder for saving related images
16│   ├── mymodel1.png   # model
17│   └── STM32H743.jpg  # H743
18├── model  # model save path
19│   └── keras_model.h5
20├── Readme.md
21├── tf2_linear_regression.ipynb
22└── tf2_linear_regression_extension.ipynb

3. Code Debugging

Initial understanding of STM32CubeIDE: Basic description and development process: https://blog.csdn.net/Naisu_kun/article/details/95935283

3.1 Import Project

Select the path of the previously exported project:

Next, you can use STM32Cube IDE to debug the generated project.

3.2 Generate .bin File

During the compilation process, the corresponding .bin file will also be automatically generated, which can later be burned onto the development board using the stm32cubeProgrammer tool.

3.3 Burn .bin File

The interface for successful burning is as follows:

3.4 Other

Before using cutecom to connect to the serial port, remember to disconnect the STM32Programer and the development board, otherwise, an error will occur when opening the serial port.

As you can see, our AI model is happily running on the development board, awesome!!!

4. Reference Articles

• STM32CubeMX Series Tutorials

• Three Ways to Build Models in Tensorflow 2.0:

• 👆 https://blog.csdn.net/weixin_42264234/article/d

• Common Pitfalls When Installing STM32CubeProgrammer on Ubuntu16.04 and Ubuntu18.04:

• 👆 https://blog.csdn.net/lu_embedded/article/details/103032083

• Basic Description and Development Process:

• 👆 https://blog.csdn.net/Naisu_kun/article/details/95935283