Using Ultrascal+ to efficiently implement an image processing pipeline system with image processing IP cores.
Platforms Used in This Project
Hardware Components
Digilent Genesys ZU × 1 (FPGA Platform) Digilent PCAM5 × 1 (MIPI Camera)
Software Components
AMD-Xilinx Vivado Design Suite
Introduction
From simple embedded vision to autonomous vehicles and drones, image processing is at the core of many applications. The Xilinx MPSoC features built-in DisplayPort (DP) capabilities and supports MIPI DPhy in programmable logic IO, making it an excellent embedded vision platform.
Simultaneously using artificial intelligence and machine learning capabilities allows us to create a complex embedded vision processing system.
In this project, we will explore establishing and running image processing using PCAM (FMC expansion board) and Display Port. We can then add image processing IP cores to further demonstrate the FPGA’s ability to process images.
Programmable Logic Design (PL Side Design)
In our design, we will include the following IP:
-
Zynq MPSoC Processing System – Enable DisplayPort, I2C, and GPIO EMIO -
MIPI CSI2 RX Sub System – Receive MIPI CSI2 stream from PCAM. -
Sensor Demosaic – Converts RAW pixel format to RGB pixel format. -
Gamma LUT – Gamma image correction -
VDMA – Writes images to DDR memory in PS -
Video Timing Generator – Generates output video timing -
AXIS to Video Out – Converts AXI stream to video stream -
Clock Wizards – Generates video pixel clock and MIPI CSI2 reference clock
In addition to the IP, we also need to consider the clock architecture. For this solution, we designed the following clock tree:
-
AXI Clock – 150 MHz – This provides the clock for AXI Stream and AXI lite interfaces -
DPHY Reference Clock – 200 MHz – Generated by Clock Wizards -
Pixel Clock – 74.25 MHz – for 1280 x 720 60 FPS – Generated by Clock Wizards
The completed block diagram should be as follows, and the kernel configuration is shown below.
To control and configure PCAM5 (FMC-MIPI expansion board), we use I2C and GPIO. The GPIO signal is used to enable and power the PCAM5. I2C is used to configure the PCAM5 itself.
We can use EMIO GPIO to extend processor GPIO to programmable logic. While I2C is provided by PS IIC0, we need to configure the enable in the application software.
To enable the DisplayPort in the MPSoC, we need to configure the DisplayPort peripherals under the I/O configuration page. The hardware uses the MGT of Bank 505.
We also need to configure auxiliary pins connected to PL through EMIO. This means we need to handle them differently from MIO in PS. The auxiliary output enable pin is active low, so we need to use inverters on signals when using EMIO.
Once the DisplayPort is configured, we can enable real-time video input for PL. This is available in PS-PL configuration.
The final configuration is to set the GPIO to provide a 1-bit EMIO, allowing us to turn on and off the PCAM5.
The Sensor will be configured via I2C, ultimately outputting 10-bit RAW video at a data rate of 280 Mbps through two MIPI channels.
Thus, we need to configure the MIPI CSI-2 RX subsystem.
Since we only have one MIPI interface, we will configure the MIPI kernel to include all shared logic. If we have multiple MIPI interfaces on the same bank, the next MIPI interface will be configured to use the shared logic in the example design.
The final setup of the MIPI interface is to assign IO bank and pins for the MIPI channels and clocks. As defined here, we do not need to add them to the XDC file that defines pin locations.
The next step to creating usable images is to convert the raw image, which contains information about one color channel per pixel.
In images containing red, green, and blue elements, this is called demosaicing, implemented by the Sensor Demosaic IP block.
For our application, PCAM5 will output pixels in the following format BGBG/GRGR.
Green is twice as much as red and blue because our eyes are more sensitive to green; if the visible spectrum is in the middle, green is more sensitive than red and blue at both ends.
The next step in processing is to implement the Gamma correction IP core.
The final stage is to insert a VDMA, which will allow video data to be written from PS DDR to memory.
To provide output video timing to the DisplayPort, we use VTC in generator mode.
The final stage is the AXIS to video output IP, which will take the video timing controller, timing signals, and AXI video stream to create output video with the correct timing.
Its output will be connected to the DisplayPort video real-time input.
This requires some thought, as real-time video input has a 36-bit video input, with each element of the pixel being 12 bits. We are using 10 bits per element, making it a total of 30 bits.
By padding the LSB, the AXIS stream to the video output will scale the output block from 30 bits per pixel to 36 bits per pixel.
Ensure that the color channels output by the AXIS to Video Out IP core are correctly aligned with the DisplayPort real-time video input. The simplest way is to use the AXIS subset converter in the AXI Stream to switch color channels as needed.
To assist in debugging and understanding the design, I have also included several integrated logic analyzers in the design.
With the design complete, we can now build the hardware design and implement the software application using SDK.
Software Development
After completing the hardware design, the next step is to write the software that will configure the IP in programmable logic. This configuration will allow them to display video on the DisplayPort monitor.
Thus, our application software will perform the following steps:
-
Configure GPIO and enable power for PCAM5
-
Set Video Mode 1280 x 720 at 60 FPS and configure VTC
3) Configure VDMA frame size and memory storage
-
Configure Sensor Demosaic
-
Configure Gamma Correction
-
Use I2C link to configure PCAM5
-
Enable reading and writing frame buffers from VDMA
-
Set up DisplayPort
As we do with other projects, we need to import hardware specifications, create applications, and BSP.
To use a real-time video source on the DisplayPort, we first need to update the BSP settings and regenerate it.
In the BSP settings, change avbuf to dppsu and wait for the BSP to regenerate.
Once the BSP is correctly configured, we can create the application code. The BSP will provide all the API functions required to use the IP blocks in PL.
The main application is as follows:
#include <stdio.h>
#include "platform.h"
#include "xil_printf.h"
#include "xvtc.h"
#include "vga_modes.h"
#include "xv_tpg.h"
#include "xvidc.h"
#include "xavbuf.h"
#include "xavbuf_clk.h"
#include "xvidc.h"
#include "xdpdma_video_example.h"
#include "xiicps.h"
#include "i2c.h"
#include "xaxivdma.h"
#include "xaxivdma_i.h"
#include "xgpiops.h"
#include "xv_demosaic.h"
#include "xv_gamma_lut.h"
#include "math.h"
XVtc VtcInst;
XVtc_Config *vtc_config ;
XV_tpg tpg;
XV_tpg_Config *tpg_config;
XIicPs iic_cam;
XAxiVdma vdma;
XAxiVdma_DmaSetup vdmaDMA;
XAxiVdma_Config *vdmaConfig;
XGpioPs gp_cam;
VideoMode video;
XV_demosaic cfa;
XV_gamma_lut gamma_inst;
u8 SendBuffer [10];
u8 RecvBuffer [10];
u16 gamma_reg[1024];
#define DEMO_MAX_FRAME (720*1280)
#define DEMO_STRIDE (1280*2)
#define DISPLAY_NUM_FRAMES 3
#define IIC_cam XPAR_XIICPS_0_DEVICE_ID
#define cam_gpio XPAR_XGPIOPS_0_DEVICE_ID
#define CAM_ID 0x78
#define IIC_CAM_ADDR 0x3c
#define IIC_SCLK_RATE 100000
#define MUX_ADDR 0x70
void detect_camera();
int Initial_setting_1 ( u32 *cfg_init , int cfg_init_QTY );
void read_camera();
void gamma_calc(float gamma_val);
u32 frameBuf[DISPLAY_NUM_FRAMES][DEMO_MAX_FRAME];
u32 *pFrames[DISPLAY_NUM_FRAMES];
int main()
{
XVtc_Timing vtcTiming;
XVtc_SourceSelect SourceSelect;
XGpioPs_Config *GPIO_Config;
int Status;
init_platform();
disable_caches();
print("Hello World\n\r");
v...
}
Files with I2C configurations
//config_word_t const cfg_init_[] =
static u32 cfg_init [][] =
{
//[7]=0 Software reset; [6]=1 Software power down; Default=0x02
{0x3008, 0x42},
//[1]=1 System input clock from PLL; Default read = 0x11
{0x3103, 0x03},
//[3:0]=0000 MD2P,MD2N,MCP,MCN input; Default=0x00
{0x3017, 0x00},
//[7:2]=000000 MD1P,MD1N, D3:0 input; Default=0x00
{0x3018, 0x00},
//[6:4]=001 PLL charge pump, [3:0]=1000 MIPI 8-bit mode
{0x3034, 0x18},
//PLL1 configuration
//[7:4]=0001 System clock divider /1, [3:0]=0001 Scale divider for MIPI /1
{0x3035, 0x11},
//[7:0]=56 PLL multiplier
{0x3036, 0x38},
//[4]=1 PLL root divider /2, [3:0]=1 PLL pre-divider /1
{0x3037, 0x11},
//[5:4]=00 PCLK root divider /1, [3:2]=00 SCLK2x root divider /1, [1:0]=01 SCLK root divider /2
{0x3108, 0x01},
//PLL2 configuration
//[5:4]=01 PRE_DIV_SP /1.5, [2]=1 R_DIV_SP /1, [1:0]=00 DIV12_SP /1
{0x303D, 0x10},
//[4:0]=11001 PLL2 multiplier DIV_CNT5B = 25
{0x303B, 0x19},
{0x3630, 0x2e},
{0x3631, 0x0e},
{0x3632, 0xe2},
{0x3633, 0x23},
{0x3621, 0xe0},
{0x3704, 0xa0},
{0x3703, 0x5a},
{0x3715, 0x78},
{0x3717, 0x01},
{0x370b, 0x60},
{0x3705, 0x1a},
{0x3905, 0x02},
{0x3906, 0x10},
{0x3901, 0x0a},
{0x3731, 0x02},
//VCM debug mode
{0x3600, 0x37},
{0x3601, 0x33},
//System control register changing not recommended
{0x302d, 0x60},
//??
{0x3620, 0x52},
{0x371b, 0x20},
//?? DVP
{0x471c, 0x50},
{0x3a13, 0x43},
{0x3a18, 0x00},
{0x3a19, 0xf8},
{0x3635, 0x13},
{0x3636, 0x06},
{0x3634, 0x44},
{0x3622, 0x01},
{0x3c01, 0x34},
{0x3c04, 0x28},
{0x3c05, 0x98},
{0x3c06, 0x00},
{0x3c07, 0x08},
{0x3c08, 0x00},
{0x3c09, 0x1c},
{0x3c0a, 0x9c},
{0x3c0b, 0x40},
//[7]=1 color bar enable, [3:2]=00 eight color bar
{0x503d, 0x00},
//[2]=1 ISP vflip, [1]=1 sensor vflip
{0x3820, 0x46},
//[7:5]=010 Two lane mode, [4]=0 MIPI HS TX no power down, [3]=0 MIPI LP RX no power down, [2]=1 MIPI enable, [1:0]=10 Debug mode; Default=0x58
{0x300e, 0x45},
//[5]=0 Clock free running, [4]=1 Send line short packet, [3]=0 Use lane1 as default, [2]=1 MIPI bus LP11 when no packet; Default=0x04
{0x4800, 0x14},
{0x302e, 0x08},
//[7:4]=0x3 YUV422, [3:0]=0x0 YUYV
//{0x4300, 0x30},
//[7:4]=0x6 RGB565, [3:0]=0x0 {b[4:0],g[5:3],g[2:0],r[4:0]}
{0x4300, 0x6f},
{0x501f, 0x01},
{0x4713, 0x03},
{0x4407, 0x04},
{0x440e, 0x00},
{0x460b, 0x35},
//[1]=0 DVP PCLK divider manual control by 0x3824[4:0]
{0x460c, 0x20},
//[4:0]=1 SCALE_DIV=INT(3824[4:0]/2)
{0x3824, 0x01},
//MIPI timing
// {0x4805, 0x10}, //LPX global timing select=auto
// {0x4818, 0x00}, //hs_prepare + hs_zero_min ns
// {0x4819, 0x96},
// {0x482A, 0x00}, //hs_prepare + hs_zero_min UI
//
// {0x4824, 0x00}, //lpx_p_min ns
// {0x4825, 0x32},
// {0x4830, 0x00}, //lpx_p_min UI
//
// {0x4826, 0x00}, //hs_prepare_min ns
// {0x4827, 0x32},
// {0x4831, 0x00}, //hs_prepare_min UI
//[7]=1 LENC correction enabled, [5]=1 RAW gamma enabled, [2]=1 Black pixel cancellation enabled, [1]=1 White pixel cancellation enabled, [0]=1 Color interpolation enabled
{0x5000, 0x07},
//[7]=0 Special digital effects, [5]=0 scaling, [2]=0 UV average disabled, [1]=1 Color matrix enabled, [0]=1 Auto white balance enabled
{0x5001, 0x03}
};
static u32 cfg_simple_awb[][2] =
{
// Disable Advanced AWB
{0x518d ,0x00},
{0x518f ,0x20},
{0x518e ,0x00},
{0x5190 ,0x20},
{0x518b ,0x00},
{0x518c ,0x00},
{0x5187 ,0x10},
{0x5188 ,0x10},
{0x5189 ,0x40},
{0x518a ,0x40},
{0x5186 ,0x10},
{0x5181 ,0x58},
{0x5184 ,0x25},
{0x5182 ,0x11},
// Enable simple AWB
{0x3406 ,0x00},
{0x5183 ,0x80},
{0x5191 ,0xff},
{0x5192 ,0x00},
{0x5001 ,0x03}
};
static u32 cfg_720p_60fps[][2] =
{//1280 x 720 binned, RAW10, MIPISCLK=280M, SCLK=56Mz, PCLK=56M
//PLL1 configuration
{0x3008, 0x42},
//[7:4]=0010 System clock divider /2, [3:0]=0001 Scale divider for MIPI /1
{0x3035, 0x21},
//[7:0]=70 PLL multiplier
{0x3036, 0x46},
//[4]=0 PLL root divider /1, [3:0]=5 PLL pre-divider /1.5
{0x3037, 0x05},
//[5:4]=01 PCLK root divider /2, [3:2]=00 SCLK2x root divider /1, [1:0]=01 SCLK root divider /2
{0x3108, 0x11},
//[6:4]=001 PLL charge pump, [3:0]=1010 MIPI 10-bit mode
{0x3034, 0x1A},
//[3:0]=0 X address start high byte
{0x3800, (0 >> 8) & 0x0F},
//[7:0]=0 X address start low byte
{0x3801, 0 & 0xFF},
//[2:0]=0 Y address start high byte
{0x3802, (8 >> 8) & 0x07},
//[7:0]=0 Y address start low byte
{0x3803, 8 & 0xFF},
//[3:0] X address end high byte
{0x3804, (2619 >> 8) & 0x0F},
//[7:0] X address end low byte
{0x3805, 2619 & 0xFF},
//[2:0] Y address end high byte
{0x3806, (1947 >> 8) & 0x07},
//[7:0] Y address end low byte
{0x3807, 1947 & 0xFF},
//[3:0]=0 timing hoffset high byte
{0x3810, (0 >> 8) & 0x0F},
//[7:0]=0 timing hoffset low byte
{0x3811, 0 & 0xFF},
//[2:0]=0 timing voffset high byte
{0x3812, (0 >> 8) & 0x07},
//[7:0]=0 timing voffset low byte
{0x3813, 0 & 0xFF},
//[3:0] Output horizontal width high byte
{0x3808, (1280 >> 8) & 0x0F},
//[7:0] Output horizontal width low byte
{0x3809, 1280 & 0xFF},
//[2:0] Output vertical height high byte
{0x380a, (720 >> 8) & 0x7F},
//[7:0] Output vertical height low byte
{0x380b, 720 & 0xFF},
//HTS line exposure time in # of pixels
{0x380c, (1896 >> 8) & 0x1F},
{0x380d, 1896 & 0xFF},
//VTS frame exposure time in # lines
{0x380e, (984 >> 8) & 0xFF},
{0x380f, 984 & 0xFF},
//[7:4]=0x3 horizontal odd subsample increment, [3:0]=0x1 horizontal even subsample increment
{0x3814, 0x31},
//[7:4]=0x3 vertical odd subsample increment, [3:0]=0x1 vertical even subsample increment
{0x3815, 0x31},
//[2]=0 ISP mirror, [1]=0 sensor mirror, [0]=1 horizontal binning
{0x3821, 0x01},
//little MIPI shit: global timing unit, period of PCLK in ns * 2(depends on # of lanes)
{0x4837, 36}, // 1/56M*2
//Undocumented anti-green settings
{0x3618, 0x00}, // Removes vertical lines appearing under bright light
{0x3612, 0x59},
{0x3708, 0x64},
{0x3709, 0x52},
{0x370c, 0x03},
//[7:4]=0x0 Formatter RAW, [3:0]=0x0 BGBG/GRGR
{0x4300, 0x00},
//[2:0]=0x3 Format select ISP RAW (DPC)
{0x501f, 0x03},
{0x3008, 0x02},
};
DisplayPort configuration files
/***************************** Include Files *********************************/
#include "xil_exception.h"
#include "xil_printf.h"
#include "xil_cache.h"
#include "xdpdma_video_example.h"
/************************** Constant Definitions *****************************/
#define DPPSU_DEVICE_ID XPAR_PSU_DP_DEVICE_ID
#define AVBUF_DEVICE_ID XPAR_PSU_DP_DEVICE_ID
#define DPDMA_DEVICE_ID XPAR_XDPDMA_0_DEVICE_ID
#define DPPSU_INTR_ID 151
#define DPDMA_INTR_ID 154
#define INTC_DEVICE_ID XPAR_SCUGIC_0_DEVICE_ID
#define DPPSU_BASEADDR XPAR_PSU_DP_BASEADDR
#define AVBUF_BASEADDR XPAR_PSU_DP_BASEADDR
#define DPDMA_BASEADDR XPAR_PSU_DPDMA_BASEADDR
#define BUFFERSIZE 1280 * 720 * 4 /* HTotal * VTotal * BPP */
#define LINESIZE 1280 * 4 /* HTotal * BPP */
#define STRIDE LINESIZE /* The stride value should be aligned to 256*/
/************************** Variable Declarations ***************************/
u8 Frame[BUFFERSIZE] __attribute__ ((__aligned__(256)));
XDpDma_FrameBuffer FrameBuffer;
/**************************** Type Definitions *******************************/
/*****************************************************************************/
/***
*
* Main function to call the DPDMA Video example.
*
* @param None
*
* @return XST_SUCCESS if successful, otherwise XST_FAILURE.
*
* @note None
*
******************************************************************************/
int run_dppsu()
{
int Status;
Xil_DCacheDisable();
Xil_ICacheDisable();
xil_printf("DPDMA Generic Video Example Test \r\n");
Status = DpdmaVideoExample(&RunCfg);
if (Status != XST_SUCCESS) {
xil_printf("DPDMA Video Example Test Failed\r\n");
return XST_FAILURE;
}
xil_printf("Successfully ran DPDMA Video Example Test\r\n");
return XST_SUCCESS;
}
/****************************************************************************/
/***
* The purpose of this function is to illustrate how to use the XDpDma device
* driver in Graphics overlay mode.
*
* @param RunCfgPtr is a pointer to the application configuration structure.
*
* @return XST_SUCCESS if successful, else XST_FAILURE.
*
* @note None.
*
****************************************************************************/
int DpdmaVideoExample(Run_Config *RunCfgPtr)
{
u32 Status;
/* Initialize the application configuration */
InitRunConfig(RunCfgPtr);
Status = InitDpDmaSubsystem(RunCfgPtr);
if (Status != XST_SUCCESS) {
return XST_FAILURE;
}
SetupInterrupts(RunCfgPtr);
//xil_printf("Generating Overlay.....\n\r");
GraphicsOverlay(Frame, RunCfgPtr);
/* Populate the FrameBuffer structure with the frame attributes */
FrameBuffer.Address = (INTPTR)Frame;
FrameBuffer.Stride = STRIDE;
FrameBuffer.LineSize = LINESIZE;
FrameBuffer.Size = BUFFERSIZE;
XDpDma_DisplayGfxFrameBuffer(RunCfgPtr->DpDmaPtr, &FrameBuffer);
return XST_SUCCESS;
}
/****************************************************************************/
/***
* The purpose of this function is to initialize the application configuration.
*
* @param RunCfgPtr is a pointer to the application configuration structure.
*
* @return None.
*
* @note None.
*
****************************************************************************/
void InitRunConfig(Run_Config *RunCfgPtr)
{
/* Initial configuration parameters. */
RunCfgPtr->DpPsuPtr = &DpPsu;
RunCfgPtr->IntrPtr = &Intr;
RunCfgPtr->AVBufPtr = &AVBuf;
RunCfgPtr->DpDmaPtr = &DpDma;
RunCfgPtr->VideoMode = XVIDC_VM_1280x720_60_P;
RunCfgPtr->Bpc = XVIDC_BPC_12;//XVIDC_BPC_8;
RunCfgPtr->ColorEncode = XDPPSU_CENC_RGB;
RunCfgPtr->UseMaxCfgCaps = 1;
RunCfgPtr->LaneCount = LANE_COUNT_2;
RunCfgPtr->LinkRate = LINK_RATE_540GBPS;
RunCfgPtr->EnSynchClkMode = 0;
RunCfgPtr->UseMaxLaneCount = 1;
RunCfgPtr->UseMaxLinkRate = 1;
}
/****************************************************************************/
/***
* The purpose of this function is to initialize the DP Subsystem (XDpDma,
* XAVBuf, XDpPsu)
*
* @param RunCfgPtr is a pointer to the application configuration structure.
*
* @return None.
*
* @note None.
*
****************************************************************************/
int InitDpDmaSubsystem(Run_Config *RunCfgPtr)
{
u32 Status;
XDpPsu *DpPsuPtr = RunCfgPtr->DpPsuPtr;
XDpPsu_Config *DpPsuCfgPtr;
XAVBuf *AVBufPtr = RunCfgPtr->AVBufPtr;
XDpDma_Config *DpDmaCfgPtr;
XDpDma *DpDmaPtr = RunCfgPtr->DpDmaPtr;
/* Initialize DisplayPort driver. */
DpPsuCfgPtr = XDpPsu_LookupConfig(DPPSU_DEVICE_ID);
XDpPsu_CfgInitialize(DpPsuPtr, DpPsuCfgPtr, DpPsuCfgPtr->BaseAddr);
/* Initialize Video Pipeline driver */
XAVBuf_CfgInitialize(AVBufPtr, DpPsuPtr->Config.BaseAddr, AVBUF_DEVICE_ID);
/* Initialize the DPDMA driver */
DpDmaCfgPtr = XDpDma_LookupConfig(DPDMA_DEVICE_ID);
XDpDma_CfgInitialize(DpDmaPtr,DpDmaCfgPtr);
/* Initialize the DisplayPort TX core. */
Status = XDpPsu_InitializeTx(DpPsuPtr);
if (Status != XST_SUCCESS) {
return XST_FAILURE;
}
/* Set the format graphics frame for DPDMA*/
Status = XDpDma_SetGraphicsFormat(DpDmaPtr, RGBA8888);
if (Status != XST_SUCCESS) {
return XST_FAILURE;
}
/* Set the format graphics frame for Video Pipeline*/
Status = XAVBuf_SetInputNonLiveGraphicsFormat(AVBufPtr, RGBA8888);
if (Status != XST_SUCCESS) {
return XST_FAILURE;
}
/* Set the QOS for Video */
XDpDma_SetQOS(RunCfgPtr->DpDmaPtr, 11);
/* Enable the Buffers required by Graphics Channel */
//XAVBuf_EnableGraphicsBuffers(RunCfgPtr->AVBufPtr, 1);
XAVBuf_SetInputLiveVideoFormat(&AVBufPtr, RGB_10BPC);
XAVBuf_EnableVideoBuffers(&AVBufPtr, 1);
/* Set the output Video Format */
XAVBuf_SetOutputVideoFormat(AVBufPtr, RGB_10BPC);
/* Select the Input Video Sources.
* Here in this example we are going to demonstrate
* graphics overlay over the TPG video.
*/
XAVBuf_InputVideoSelect(AVBufPtr, XAVBUF_VIDSTREAM1_LIVE, XAVBUF_VIDSTREAM2_NONE);
/* Configure Video pipeline for graphics channel */
//XAVBuf_ConfigureGraphicsPipeline(AVBufPtr);
/* Configure the output video pipeline */
XAVBuf_ConfigureOutputVideo(AVBufPtr);
/* Disable the global alpha, since we are using the pixel based alpha */
XAVBuf_SetBlenderAlpha(AVBufPtr, 0, 0);
/* Set the clock mode */
XDpPsu_CfgMsaEnSynchClkMode(DpPsuPtr, RunCfgPtr->EnSynchClkMode);
/* Set the clock source depending on the use case.
* Here for simplicity we are using PS clock as the source*/
XAVBuf_SetAudioVideoClkSrc(AVBufPtr, XAVBUF_PL_CLK, XAVBUF_PS_CLK);
/* Issue a soft reset after selecting the input clock sources */
XAVBuf_SoftReset(AVBufPtr);
return XST_SUCCESS;
}
/****************************************************************************/
/***
* The purpose of this function is to setup call back functions for the DP
* controller interrupts.
*
* @param RunCfgPtr is a pointer to the application configuration structure.
*
* @return None.
*
* @note None.
*
****************************************************************************/
void SetupInterrupts(Run_Config *RunCfgPtr)
{
XDpPsu *DpPsuPtr = RunCfgPtr->DpPsuPtr;
XScuGic *IntrPtr = RunCfgPtr->IntrPtr;
XScuGic_Config *IntrCfgPtr;
u32 IntrMask = XDPPSU_INTR_HPD_IRQ_MASK | XDPPSU_INTR_HPD_EVENT_MASK;
XDpPsu_WriteReg(DpPsuPtr->Config.BaseAddr, XDPPSU_INTR_DIS, 0xFFFFFFFF);
XDpPsu_WriteReg(DpPsuPtr->Config.BaseAddr, XDPPSU_INTR_MASK, 0xFFFFFFFF);
XDpPsu_SetHpdEventHandler(DpPsuPtr, DpPsu_IsrHpdEvent, RunCfgPtr);
XDpPsu_SetHpdPulseHandler(DpPsuPtr, DpPsu_IsrHpdPulse, RunCfgPtr);
/* Initialize interrupt controller driver. */
IntrCfgPtr = XScuGic_LookupConfig(INTC_DEVICE_ID);
XScuGic_CfgInitialize(IntrPtr, IntrCfgPtr, IntrCfgPtr->CpuBaseAddress);
/* Register ISRs. */
XScuGic_Connect(IntrPtr, DPPSU_INTR_ID,
(Xil_InterruptHandler)XDpPsu_HpdInterruptHandler, RunCfgPtr->DpPsuPtr);
/* Trigger DP interrupts on rising edge. */
XScuGic_SetPriorityTriggerType(IntrPtr, DPPSU_INTR_ID, 0x0, 0x03);
/* Connect DPDMA Interrupt */
XScuGic_Connect(IntrPtr, DPDMA_INTR_ID,
(Xil_ExceptionHandler)XDpDma_InterruptHandler, RunCfgPtr->DpDmaPtr);
/* Initialize exceptions. */
Xil_ExceptionInit();
Xil_ExceptionRegisterHandler(XIL_EXCEPTION_ID_IRQ_INT,
(Xil_ExceptionHandler)XScuGic_DeviceInterruptHandler,
INTC_DEVICE_ID);
/* Enable exceptions for interrupts. */
Xil_ExceptionEnableMask(XIL_EXCEPTION_IRQ);
Xil_ExceptionEnable();
/* Enable DP interrupts. */
XScuGic_Enable(IntrPtr, DPPSU_INTR_ID);
XDpPsu_WriteReg(DpPsuPtr->Config.BaseAddr, XDPPSU_INTR_EN, IntrMask);
/* Enable DPDMA Interrupts */
XScuGic_Enable(IntrPtr, DPDMA_INTR_ID);
XDpDma_InterruptEnable(RunCfgPtr->DpDmaPtr, XDPDMA_IEN_VSYNC_INT_MASK);
}
/****************************************************************************/
/***
* The purpose of this function is to generate a Graphics frame of the format
* RGBA8888 which generates an overlay on 1/2 of the bottom of the screen.
* This is just to illustrate the functionality of the graphics overlay.
*
* @param RunCfgPtr is a pointer to the application configuration structure.
* @param Frame is a pointer to a buffer which is going to be populated with
* rendered frame
*
* @return Returns a pointer to the frame.
*
* @note None.
*
****************************************************************************/
u8 *GraphicsOverlay(u8* Frame, Run_Config *RunCfgPtr)
{
u64 Index;
u32 *RGBA;
RGBA = (u32 *) Frame;
/*
* Red at the top half
* Alpha = 0x0F
* */
for(Index = 0; Index < (BUFFERSIZE/4) /2; Index ++) {
//RGBA[Index] = 0x0F0000FF;
}
for(; Index < BUFFERSIZE/4; Index ++) {
/*
* Green at the bottom half
* Alpha = 0xF0
* */
//RGBA[Index] = 0xF000FF00;
}
return Frame;
}
Testing
Once the application is complete, we can create a debug configuration, download the bit file, and run the application software.
An image should be visible on the monitor, but we can also check the ILA. These displays will show the input video stream received from the MIPI CSI-2 IP core and VDMA.
AXIS transmits video frames using sideband signals Tuser and Tlast. Tuser indicates the start of a new frame, while Tlast indicates the end of a line.
The third ILA monitors AXI Stream to Video Output, allowing us to monitor the lock signal on the output.
When all of this is running, the image output should be visible:
The final resource utilization of the solution is shown in the figure below, and we still have enough space to implement interesting algorithms.
Source Code
https://github.com/ATaylorCEngFIET/Genesys_ZU_MIPI_PCAM