Video Processing SoC System Based on Arm Cortex-M0

Guide Map

1

System Function and Architecture Introduction

This system is developed based on the EG4S20 series FPGA development board from Anlu Technology, establishing a System on Chip (SoC) architecture based on Arm Cortex-M0, implementing various Image Signal Processing (ISP) algorithms. The system reads raw video data from a Secure Digital (SD) memory card, processes it through ISP hardware acceleration, and displays the processed images in real-time via a High Definition Multimedia Interface (HDMI).

The system includes a General Purpose Input Output (GPIO) interrupt control module, which bridges to the AHB bus via an APB bus, and connects to the Cortex-M0 for intelligent switching between different ISP algorithms. The article also sets up an Advanced Peripheral Bus (APB) and connects a BOARD keyboard, seven-segment display, buzzer module, Serial Peripheral Interface (SPI) module, and Universal Asynchronous Receiver/Transmitter (UART) serial module on the AHB bus. To achieve intelligent video processing capabilities and build the ISP algorithm on-chip system, this system also includes a Secure Digital Input and Output (SDIO) read module, Synchronous Dynamic Random Access Memory (SDRAM) buffer module, ISP algorithm switching module, and HDMI driver module.

The workflow of the system is as follows: after reading raw video data from the Secure Digital (SD) memory card, the SDIO driver module decodes the read video data, placing it in the SDRAM module on the FPGA, and after processing by the SDRAM driver program, enters the ISP hardware acceleration module for video signal processing, which is displayed on the screen via the HDMI interface. The combination of matrix keyboard, seven-segment display, and buzzer modules achieves video processing functions and allows for certain expansions, setting up key interrupts to control different modes of LED light flow, displaying the current mode on the seven-segment display through key controls. Different modes can process the video with different ISP algorithms, continue playback, pause playback, and reset operations. For real-time debugging, a Serial Wire Debug (SWD) interface is designed to connect the ARM Cortex-M0 with a Personal Computer (PC), using the Keil μVision tool to write and generate software programs for online real-time debugging of the software running on the SoC.

2

Peripheral Introduction

GPIO, short for General Purpose Input/Output, is one of the most important basic hardware resources in embedded systems. The working principle of GPIO mainly involves operations on registers. For input functionality, the voltage level of the pin can be determined by reading a certain register; for output functionality, a certain register can be written to make the pin output high or low voltage.

UART, short for Universal Asynchronous Receiver/Transmitter, allows data transmission between two devices via a serial interface. This transmission is asynchronous, meaning the clocks for sending and receiving data are not synchronized, but rely on signals such as start bits, data bits, parity bits, and stop bits to indicate the beginning, end, and perform error checking of the data.

APB-TIMER is a general-purpose timer with an APB slave interface, which can serve as a clock reference or for pulse detection and PWM generation.

In the EG4S20 series FPGA, there are two types of Memory resources: Block Memory and Distributed Memory. Block RAM consists of dedicated SRAM within the FPGA, with its output being register output, providing read data on the next clock rising edge after an address is given; while Distributed Memory consists of LUT logic resources within the FPGA, with its output being logic output, independent of the clock.

SPI (Serial Peripheral Interface) is a synchronous serial communication protocol, usually used for communication between microcontrollers (MCUs) and their peripheral devices (peripheral ICs), or between two microcontrollers (MCUs).

This system has designed 8 ISP algorithms, which are Demosaic, Automatic White Balance (AWB), Color Correction Matrix (CCM), Red Green Blue (RGB) to YUV conversion, Gamma correction, Image denoising, Image enhancement, and YUV to RGB conversion.