Deep Learning Model Inference on Raspberry Pi Zero W Using Python

Deep learning-based image segmentation tool https://github.com/AIDajiangtang

Deep learning-based image stitching tool https://github.com/AIDajiangtang/Superpoint-LightGlue-Image-Stiching

Large model-based segmentation tool https://github.com/AIDajiangtang/Segment-Anything-CPP https://github.com/AIDajiangtang/Segment-Anything-CSharp

ARM7: Time point: The ARM7 series processors were released in 1994. Characteristics: The ARM7 series processors are early versions of the ARM architecture, using a 32-bit RISC architecture, mainly for embedded systems and low-power devices. It has lower power consumption and cost, suitable for resource-constrained applications. CPU architecture: ARMv4T architecture.
ARM9: Time point: The ARM9 series processors were released in 1997. Characteristics: The ARM9 series processors are also based on a 32-bit RISC architecture, used in various applications including embedded systems, mobile devices, and digital audio. It has higher performance and flexibility, suitable for medium power and performance requirements. CPU architecture: ARMv5TE architecture.

ARM11: Time point: The ARM11 series processors were released in 2002. Characteristics: The ARM11 series processors also use a 32-bit RISC architecture, mainly applied in mobile devices and digital media fields. It has higher performance and computing power, supporting multimedia processing and floating-point operations. CPU architecture: ARMv6 architecture.
ARM Cortex-A: Time point: The ARM Cortex-A series processors were released in 2005. Characteristics: The Cortex-A series is aimed at high-performance applications, used in smartphones, tablets, servers, and other devices requiring higher processing performance. It has more powerful processing capabilities, higher frequencies, and more complex functions. CPU architecture: Based on ARMv7-A or ARMv8-A architecture.
ARM Cortex-R: Time point: The ARM Cortex-R series processors were released in 2004. Characteristics: The Cortex-R series is aimed at real-time applications, used in embedded systems, automotive electronics, industrial control, etc., requiring quick response and reliability. It has lower latency and more reliable real-time performance. CPU architecture: Based on ARMv7-R or ARMv8-R architecture.
ARM Cortex-M: Time point: The ARM Cortex-M series processors were released in 2004. Characteristics: The Cortex-M series is aimed at low-power embedded systems, used in microcontrollers, sensors, IoT devices, and other resource-constrained applications. It has lower power consumption, small size, and efficient real-time performance. CPU architecture: Based on ARMv6-M or ARMv7-M architecture.

Raspberry Pi 1 Model A/B/A+/B+: CPU series: ARM11 series CPU architecture: ARMv6 architecture
Raspberry Pi 2 Model B: CPU series: ARM Cortex-A series CPU architecture: ARMv7-A architecture
Raspberry Pi 3 Model B/B+: CPU series: ARM Cortex-A series CPU architecture: ARMv8-A architecture
Raspberry Pi 4 Model B: CPU series: ARM Cortex-A series CPU architecture: ARMv8-A architecture
Raspberry Pi Zero/Zero W: CPU series: ARM11 series CPU architecture: ARMv6 architecture
Raspberry Pi Compute Module 3/3+: CPU series: ARM Cortex-A series CPU architecture: ARMv8-A architecture
Raspberry Pi Compute Module 4: CPU series: ARM Cortex-A series CPU architecture: ARMv8-A architecture

python3 -m pip install tflite-runtime

https://www.tensorflow.org/lite/guide/build_cmake_pip

import tensorflow as tf
interpreter = tf.lite.Interpreter(model_path=args.model_file)

https://github.com/lhelontra/tensorflow-on-arm/releases/download/v2.4.0/tensorflow-2.4.0-cp37-none-linux_armv6l.whl

https://downloads.raspberrypi.org/raspios_lite_armhf/images/raspios_lite_armhf-2021-05-28/

https://www.raspberrypi.com/software/

wget https://github.com/lhelontra/tensorflow-on-arm/releases/download/v2.2.0/tensorflow-2.2.0-cp37-none-linux_armv6l.whl

sudo pip3 install tensorflow-2.2.0-cp37-none-linux_armv6l.whl

python -c "import tensorflow as tf;print(tf.reduce_sum(tf.random.normal([100, 100])))"

TypeError: Descriptors cannot not be created directly.

Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python

sudo apt-get -y install libjpeg-dev libtiff5-dev libjasper-dev libpng12-dev
sudo apt-get -y install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev
sudo apt-get -y install libxvidcore-dev libx264-dev
sudo apt-get -y install qt4-dev-tools libatlas-base-dev
sudo apt-get install libgstreamer1.0-dev
sudo apt-get install libopenexr-dev
sudo apt-get install libilmbase-dev

pip3 install opencv-python==3.4.6.27

sudo raspi-config

raspistill -t 2000 -o image.jpg

import tensorflow as tf
import numpy as np
import cv2

det = true  # is it classification or object detection
model_path = "cls_model.tflite"
if det:
    model_path = "det_model.tflite"
# Load classification tflite file
interpreter = tf.lite.Interpreter(model_path)
interpreter.allocate_tensors()
label_id_offset = 1

# Again, uncomment this decorator if you want to run inference eagerly
def detect(interpreter, input_tensor):
    """Run detection on an input image.
    Args:
        interpreter: tf.lite.Interpreter
        input_tensor: A [1, height, width, 3] Tensor of type tf.float32.
        Note that height and width can be anything since the image will be immediately resized according to the needs of the model within this function.
    Returns:
        A dict containing 3 Tensors (`detection_boxes`, `detection_classes`, and `detection_scores`).
    """
    input_details = interpreter.get_input_details()
    output_details = interpreter.get_output_details()
    # We use the original model for pre-processing, since the TFLite model doesn't
    # include pre-processing.
    preprocessed_image, shapes = detection_model.preprocess(input_tensor)
    interpreter.set_tensor(input_details[0]['index'], preprocessed_image.numpy())
    interpreter.invoke()
    boxes = interpreter.get_tensor(output_details[0]['index'])
    classes = interpreter.get_tensor(output_details[1]['index'])
    scores = interpreter.get_tensor(output_details[2]['index'])
    return boxes, classes, scores

# Define camera
capture = cv2.VideoCapture(0)
while True:
    # Take a picture and preprocess it
    ret, frame = capture.read()
    frame = cv2.flip(frame, 1)
    frame_rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
    if det:
        resized_image = cv2.resize(frame_rgb, (300, 300))
    else:
        resized_image = cv2.resize(frame_rgb, (224, 224))
    resized_image = resized_image.astype(np.float32)
    # Scale pixel values to range 0-1
    resized_image /= 255.0
    # Scale pixel values to range -1 to 1
    resized_image = (resized_image - 0.5) * 2.0
    test = np.expand_dims(resized_image, axis=0)
    input_tensor = tf.convert_to_tensor(test, dtype=tf.float32)
    # Object detection model detection
    boxes, classes, scores = detect(interpreter, input_tensor)
    viz_utils.visualize_boxes_and_labels_on_image_array(
        test[0],
        boxes[0],
        classes[0].astype(np.uint32) + label_id_offset,
        scores[0],
        category_index,
        use_normalized_coordinates=True,
        min_score_thresh=0.8)
    # Present detection results
    frame = cv2.cvtColor(test[0], cv2.COLOR_BGR2RGB)
    cv2.imshow("Object detector", frame)
    c = cv2.waitKey(20)
    # If q is pressed, terminate
    if c == 113:
        break
cv2.destroyAllWindows()