FPGA-Based MIPI Camera Development with XILINX MIPI D-PHY IP Core

FPGA-Based MIPI Camera Development with XILINX MIPI D-PHY IP Core

by

XILINX’s ULTRASCALE+ series FPGA comes with two MIPI IP cores, namely the MIPI D-PHY IP and MIPI CSI-2 IP. The D-PHY can be considered a low-level IP core that can parse data from each LANE, with data between LANE being relatively independent, requiring the user to compose a complete frame of image data. The CSI-2 is a high-level IP core that directly outputs a frame of image data.

The image of the following SENSOR is about the OV5640. Other MIPI interface SENSORS can also be used as long as the correct initialization configuration data is provided. The reason for choosing OV5640 for development is mainly that OV5640 is quite common, easy to purchase in the market, and the previous UVC camera development board based on CYUSB3065 also used OV5640, allowing for direct use of the configuration data without issues. Additionally, the OV5640 supports both DVP and MIPI interfaces, making it convenient for testing. Below is the first image, with the SENSOR configured for VGA@60 resolution, using only one LANE’s data.

FPGA-Based MIPI Camera Development with XILINX MIPI D-PHY IP Core

The information of this image is as follows:

A: The pixel clock is non-continuous, and there is no clock during line blanking and frame blanking. The frequency of the pixel clock is the line rate/8, converted from bit clock to byte clock.

B: The details after the sync code will be magnified for careful analysis of each byte of information. The above image has 3 sync codes in one scan line, with 2 before the valid image data starts and 1 after the valid image data ends.

C: The signal dl0_rxvalidhs goes high when the image data is valid.

Next, let’s study these three sync codes. Below is the information for the first sync code:

FPGA-Based MIPI Camera Development with XILINX MIPI D-PHY IP Core

There are a total of 4 bytes for the sync code, as shown above: 02 19 02 20. Users can refer to the MIPI protocol for interpretation.

The 1st byte 02 represents the start of the line. (Line Start Code)

The 2nd and 3rd bytes 0219 represent the line number (for the first line of data, it is 0001, followed by 0002, 0003, and the last line of data is 03F9).

The 4th byte 20 represents ECC.

Below is the information for the 2nd sync code:

FPGA-Based MIPI Camera Development with XILINX MIPI D-PHY IP Core

After the 2nd sync code, the signal dl0_rxvalidhs goes high, so the data behind the red box is valid image data. The 1st byte is 2a, representing that the image data format is RAW8. The 2nd and 3rd bytes 0x0280 represent that there are 640 pixels in one line. The 4th byte is ECC. These 4 bytes form a 32-bit PACKET HEADER, followed by the image data.

Below is the 3rd sync code:

FPGA-Based MIPI Camera Development with XILINX MIPI D-PHY IP Core

Only the first 4 bytes of the sync code need to be focused on, which are: 03 01 00 16. These 4 bytes need to be interpreted according to the MIPI protocol.

The 1st byte 03 represents the end of the line. (Line End Code)

The 2nd and 3rd bytes represent the line number, which is 0001 here.

The 4th byte is ECC.

With the above three sync codes, the information for one scan line is quite complete. Next, let’s capture the frame header and footer information related to a frame. The frame header is as follows:

FPGA-Based MIPI Camera Development with XILINX MIPI D-PHY IP Core

The frame starts with a separate short packet, as shown at the position of the red line on the far left of the image. The rightmost part is the first line of data. Enlarging the short packet that starts the frame, as shown in the red ellipse in the middle of the image, the 1st byte 00 represents the start of the frame (Frame Start Code). The 16-bit data e3a4 after 00 is the frame count value, which is cyclical. The 4th byte is the CRC value.

The waveform for the end of the frame is as follows:

FPGA-Based MIPI Camera Development with XILINX MIPI D-PHY IP Core

The end of the frame is also a separate short packet, as shown at the position of the red line on the far right of the image. Enlarging the short packet that ends the frame, as shown in the red ellipse in the image, the 1st byte 01 represents the end of the frame. (Frame End Code)

Author: Liangzi USB, Source: Breadboard Community
Link: https://mbb.eet-china.com/blog/3903348-426263.html
Copyright Notice: This article is original by the blogger, and reproduction without permission is prohibited!

END

Follow the Breadboard Community for daily selected electronic technology knowledge

▼

FPGA-Based MIPI Camera Development with XILINX MIPI D-PHY IP Core

  • Onboard antenna adjustment for wireless modules

  • A 5000-word HDMI knowledge guide!

  • [Show Board] Soldered nearly 40 boards to make a high-voltage power supply

  • Running neural network algorithms on STM32 microcontroller

  • Not using GD32? I replaced STM32 with domestic MM32

  • [Show Board] OneWatch watch (chip selection + schematic)

Leave a Comment

×