How to Convert Open Source Framework Models to Ascend Models Based on Orange Pi AIpro

How to Convert Open Source Framework Models to Ascend Models Based on Orange Pi AIpro

by

How to Convert Open Source Framework Models to Ascend Models Based on Orange Pi AIpro

In the previous introduction, we discussed how to develop AI inference applications based on Orange Pi AIpro, and learned that before inference, the original network model (which could be PyTorch/TensorFlow/Caffe, etc.) needs to be converted into an .om model. Only then can we call the Ascend aclmdlExecute and other model execution interfaces for model inference on the Orange Pi AIpro. This model conversion process requires the use of the ATC tool.

Currently, the ATC tool directly supports the conversion from Caffe, ONNX, TensorFlow, and MindSpore models. Therefore, if your training framework is PyTorch, you need to perform the torch.onnx.export operation to export it to an ONNX model before using the ATC tool.

Introduction to the ATC Tool

The Ascend Tensor Compiler (ATC) is the model conversion tool for Ascend. It can convert network models from open source frameworks (such as TensorFlow, ONNX, etc.) into model files (.om format) supported by Ascend AI processors for subsequent model inference.
During the model conversion process, ATC will perform operator scheduling optimization, weight data rearrangement, memory usage optimization, and other operations to further tune the open source framework’s network model, enabling it to execute efficiently on Ascend AI processors.

How to Convert Open Source Framework Models to Ascend Models Based on Orange Pi AIpro

After the open source framework network model is parsed by the Parser, it is converted into Ascend’s intermediate graph IR Graph. Then, through a series of operations such as graph preparation, graph splitting, graph optimization, and graph compilation, it is converted into a *.om model file compatible with Ascend AI processors. Finally, users can call the model loading, execution, and other interfaces provided by AscendCL to implement model inference.

Basic Usage of the ATC Tool

Below, we will take the Caffe framework ResNet-50 network model as an example to introduce how to use the ATC tool to convert the model.
1. Upload the model file *.prototxt and weight file *.caffemodel of the Caffe framework ResNet-50 network model to the Linux server where the ATC tool is located.
2. Execute the following command to perform the model conversion.
atc --framework=0 --soc_version=${soc_version} --model=$HOME/mod/resnet50.prototxt --weight=$HOME/mod/resnet50.caffemodel --output=$HOME/mod/out/caffe_resnet50

  • –framework: The type of the original network model framework, 0 indicates the Caffe framework.

  • –soc_version: Specify the version of the Ascend AI processor during model conversion. You can execute the npu-smi info command to query it. Add the Ascend information before the queried “Name”, for example, if the corresponding value of “Name” is xxxyy.
  • –model: The path of the original network model file, including the file name.
  • –weight: The path of the original network model weight file, including the file name, only needed when the original network model is Caffe.

  • –output: The path of the converted *.om model file, including the file name. After successful conversion, the model file name will automatically end with the .om suffix.

3. If the ATC run success message is displayed, it indicates that the model conversion was successful.

You can view the converted model file at the path specified by the –output parameter, for example, caffe_resnet50.om.

Advanced Usage of the ATC Tool

1. Convert the original model file or Ascend *.om model file to json format

If users find it inconvenient to view the parameter information of the original model or offline model, they can convert the original model or offline model to a json file for viewing:
• Original model file —> json file
atc --mode=1 --framework=0 --om=$HOME/mod/resnet50.prototxt --json=$HOME/mod/out/caffe_resnet50.json
• Ascend *.om model file —> json file
atc --mode=1 --om=$HOME/mod/out/caffe_resnet50.om  --json=$HOME/mod/out/caffe_resnet50.json
2. Customize the input and output data types of the *.om model
During model conversion, it is possible to specify the data type and format of the input or output nodes of the network model, and set the precision, etc. The example command scenario here is for the Caffe framework ResNet50 network model, where the converted model input is of FP16 type, and the Pooling operator is specified as output, with that output node also being of FP16 type.
atc --framework=0 --soc_version=${soc_version} --model=$HOME/mod/resnet50.prototxt --weight=$HOME/mod/resnet50.caffemodel --output=$HOME/mod/out/caffe_resnet50  --input_fp16_nodes="data" --out_nodes="pool1:0" --output_type="pool1:0:FP16"

3. Set dynamic BatchSize/dynamic resolution

For certain inference scenarios, such as executing the target recognition network after detecting a target, the number of targets is not fixed, leading to a variable BatchSize for the target recognition network input. If each inference is computed using the maximum BatchSize or maximum resolution, it will waste computational resources. Therefore, model conversion needs to support the setting of dynamic BatchSize and dynamic resolution, and during actual inference, the BatchSize and dynamic resolution required for this inference are set through the AscendCL interface.
• Dynamic BatchSize
atc --framework=0 --soc_version=${soc_version}  --model=$HOME/mod/resnet50.prototxt --weight=$HOME/mod/resnet50.caffemodel --output=$HOME/mod/out/caffe_resnet50 --input_shape="data:-1,3,224,224"   --dynamic_batch_size="1,2,4,8"
In this command, the “-1” in “–input_shape” indicates that dynamic BatchSize is set, and the specific supported BatchSize is determined by “–dynamic_batch_size”.
• Dynamic resolution
atc --framework=0 --soc_version=${soc_version}  --model=$HOME/mod/resnet50.prototxt --weight=$HOME/mod/resnet50.caffemodel --output=$HOME/mod/out/caffe_resnet50 --input_shape="data:1,3,-1,-1"   --dynamic_image_size="224,224;448,448"
In this command, the “-1,-1” in “–input_shape” indicates that dynamic resolution is set, and the specific supported resolutions are determined by “–dynamic_image_size”.
Get more learning resources for the Orange Pi AIpro development board

How to Convert Open Source Framework Models to Ascend Models Based on Orange Pi AIpro

Previous Recommendations

How to Convert Open Source Framework Models to Ascend Models Based on Orange Pi AIpro

1. Beginner’s Guide | Teach you how to quickly get started with the Orange Pi AIpro development board

2. A complete sample collection of peripheral interfaces for the Orange Pi AIpro (with source code)

3. How to develop AI inference applications based on Orange Pi AIpro

4. How to upgrade the CANN software package on the Orange Pi AIpro development board

How to Convert Open Source Framework Models to Ascend Models Based on Orange Pi AIpro

Leave a Comment