Abstract:
Video signal processing through SOPC is currently a research hotspot. This article proposes a design scheme for an HDMI multi-mode display module based on FPGA for such systems. First, the driving timing of HDMI is analyzed, and the driving signal generation circuit is designed. Then, according to the configuration parameters, multiple video signals are subjected to multi-level ALPHA blending, achieving HDMI output and display of multiple video signals, with adjustable position and transparency for each video. To ensure the real-time performance of the video, a pipelined design method is used to enhance the speed for more complex calculations. The module is written in Verilog HDL, making it highly versatile.
Chinese Citation Format: Xiang Zihao, Lu Anjiang. Design of HDMI multi-mode display module based on FPGA[J]. Application of Electronic Technique, 2017, 43(12): 48-51.English Citation Format: Xiang Zihao, Lu Anjiang. Design of HDMI multi-mode display module based on FPGA[J]. Application of Electronic Technique, 2017, 43(12): 48-51.
1 Introduction
With the continuous increase in the level of informationization in society, people’s requirements for video processing are becoming increasingly high, and the amount of data processed by video processing systems is also increasing. In embedded video processing systems, there are currently three mainstream solutions: ARM-based, DSP-based, and FPGA-based. FPGA differs from the other two chips as it is a semi-custom circuit with a large number of logic units. By configuring these logic units, corresponding circuits can be constructed to achieve the desired functions. Due to its hardware acceleration characteristics, FPGA is widely used in high-speed video processing systems. For such video processing systems, constructing a System-on-a-Programmable-Chip (SOPC) is currently the mainstream solution. SOPC is an on-chip system that implements the entire system’s logic functions on a single chip, with features such as easy design, flexible configuration, online debugging, and system reuse. Among the interfaces of video processing systems, HDMI is the latest high-definition multimedia interface, which has advantages such as high bandwidth, small size, high intelligence, and content protection, making it widely used in high-definition displays and televisions.
This paper proposes a design method for an HDMI multi-mode display module based on FPGA for SOPC-based video processing systems, which can provide a universal HDMI multi-mode display component for SOPC and can be applied to various HDMI transmitters and video with different parameters. Through this module, multiple video displays can be achieved, and the position and transparency of each video can be configured, providing a solution for multi-mode display in video processing systems.
2 Overall Scheme Design
To drive the HDMI transmission chip and achieve multi-mode display through ALPHA blending, it is necessary to design corresponding driving circuits according to the working timing of HDMI, and also to perform multi-level ALPHA blending calculations on the video data according to specified coordinate information and transparency parameters, finally outputting the driving signals and video data. Based on these functional requirements, the entire module is divided into two sub-modules: the HDMI driving module and the ALPHA blending module. The system block diagram is shown in Figure 1.
In the HDMI driving module, two counters are designed to count the clock signals and rows according to the working timing of the HDMI interface, generating line synchronization signals, field synchronization signals, and data bus enable signals during the respective clock cycles. A coordinate indicating circuit is designed to count the rows and columns of the currently output effective video data using the two counters, outputting the count value which is used to read video data and perform ALPHA blending calculations at the corresponding coordinates.
Four ALPHA blending modules are instantiated (the number can be set according to requirements), allowing for up to 4 video signals to be displayed in multi-mode. In the ALPHA blending module, data read signals are issued to read cached video data at specified coordinates based on the count values generated by the coordinate indicating circuit, and the corresponding alpha values (transparency) are generated in the specified area. An ALPHA blending calculation circuit is designed to perform ALPHA blending between foreground video data and background video data. The ALPHA blending calculation circuit adopts a pipelined design method, dividing the entire calculation process into multiple stages for ALPHA blending calculations, with each stage’s calculation results stored in registers during one clock cycle, provided to the next stage for the next clock cycle calculation. ALPHA blending calculations may lead to data output delays; therefore, another coordinate indicating circuit is instantiated, delaying the synchronization signals, field synchronization signals, and data bus enable signals by the corresponding cycles before inputting them into the circuit to generate new synchronized row and column count values for the next ALPHA blending module. This example provides two display modes, 4-channel split display and PIP (Picture in Picture) display, for verification. When the configured display mode is 4-channel split display, the first channel video acts as the foreground and first blends with a preset background color, where the overlapping part of the background has a transparency of 0 (completely not displayed), and the foreground has a transparency of 1 (completely displayed), then the blended video data is used as the background to blend with the second channel video, and this process continues for all 4 channels, with each channel’s position not overlapping. In PIP display mode, the first channel video serves as the background, while the second channel video is displayed in the center of the display, with the overlapping part of the background having a transparency of 0, and the foreground having a transparency of 1.
3 HDMI Driving Module Design
The HDMI driving module is primarily responsible for outputting corresponding driving signals based on different configuration information, enabling the video data to be output normally through the HDMI transmitter.
3.1 HDMI Transmitter Operating Mode
When video data is transmitted via HDMI, the HDMI receiving/transmitting chip encodes it into data packets using the minimal transmission differential signal (TMDS) encoding technology. Although FPGA supports various standard LVDS (Low-Voltage Differential Signaling), it cannot fully accommodate TMDS (Transition Minimized Differential Signaling), hence HDMI receiving/transmitting chips are required to implement HDMI interface functionality. The transmission between FPGA and such HDMI chips is typically achieved through a set of parallel buses, including data bus, IIC bus, and driving signals. The data bus is used for transmitting video data, while the IIC bus is used for configuring the HDMI transmitting chip. The HDMI_CLK in the driving signals is the synchronization clock signal, DE_HDMI is the data bus enable signal, HSY_HDMI is the line synchronization signal, and VSY_HDMI is the field synchronization signal. For the module to work normally, the HDMI chip must be configured for RGB output. The working timing of the HDMI interface for transmitting RGB signals is similar to that of the VGA interface, but it does not require converting digital signals into analog signals. DE_HDMI, HSY_HDMI, and VSY_HDMI are output with a fixed time relationship, while HDMI_CLK serves as a reference clock synchronized with these signals.
During a field’s time segment, depending on the video refresh frequency and resolution, during the synchronization segment, the line synchronization signal first maintains a high level for a certain period, then goes low and outputs video data after the corresponding period, with this gap being the display back porch. After the video data output is completed, a corresponding period is waited before starting the transmission for the next line, with this gap being the display front porch. The timing of the field synchronization signal still maintains a high level for a certain period, then goes low, and after a corresponding line time (the number of clock cycles required to transmit one line of data), outputs all lines in a field, and after all lines’ video data transmission is completed, waits the corresponding line time before starting the transmission for the next line. The timing segments are shown in Figure 2.
Where the timing segment of VSY_HDMI is the same as that of HSY_HDMI.
3.2 Verilog HDL Code Design
Between the FPGA and the HDMI transmitter, the method for transmitting video data is to send pixel data one by one in sync with the clock signal. For each field of data, the sequence starts from the first row, transmitting from left to right, and after completing, it starts transmitting the second row, continuing until the last row’s data transmission is completed. Therefore, in the code, two count values are defined: the x-axis count value x_cnt and the y-axis count value y_cnt. x_cnt increments by 1 every clock cycle, resetting after counting one row; y_cnt increments by 1 after x_cnt completes counting one row, resetting after counting all rows in a field. The counter Verilog HDL code is as follows:
always @(posedge clk or negedge rst_n)
if(!rst_n) x_cnt <= 12’d0;
else if(x_cnt >= HDMI_HTT) x_cnt <= 12’d0;
else x_cnt <= x_cnt + 1’b1;
always @(posedge clk or negedge rst_n)
if(!rst_n) y_cnt <= 12’d0;
else if(x_cnt == HDMI_HTT) begin
if(y_cnt >= HDMI_VTT) y_cnt <= 12’d0;
else y_cnt <= y_cnt + 1’b1;
Where HDMI_HTT is the number of clock cycles required to transmit one row, and HDMI_VTT is the number of rows in a field. Based on the count values, the line synchronization signals and field synchronization signals are raised at the corresponding times, and the Verilog HDL code is as follows:
always @(posedge clk or negedge rst_n)
if(!rst_n) HSY_HDMI <= 1’b0;
else if(x_cnt < HDMI_HST) HSY_HDMI <= 1’b1;
else HSY_HDMI <= 1’b0;
always @(posedge clk or negedge rst_n)
if(!rst_n) VSY_HDMI <= 1’b0;
else if(y_cnt < HDMI_VST) VSY_HDMI <= 1’b1;
else VSY_HDMI <= 1’b0;
Where HDMI_HST is the number of clock cycles occupied by the line synchronization segment, and HDMI_VST is the number of rows in the field synchronization segment. In the counting area belonging to the data segment, the data bus enable signal is set to high; during this time, valid video data is transmitted, and its Verilog HDL code is as follows:
always @(posedge clk or negedge rst_n)
if(!rst_n) DE_HDMI <= 1’b0;
else if((x_cnt >= (HDMI_HST + HDMI_HBP)) && (x_cnt < (HDMI_HST + HDMI_HBP + HDMI_HVT)) && (y_cnt >= (HDMI_VST + HDMI_VBP)) && (y_cnt < (HDMI_VST + HDMI_VBP + HDMI_VVT)))
DE_HDMI <= 1’b1;
else DE_HDMI <= 1’b0;
Where HDMI_HBP is the number of clock cycles occupied by each row’s data segment, and HDMI_VBP is the number of rows in the data segment. HDMI_HTT, HDMI_VTT, HDMI_HST, HDMI_VST, HDMI_HBP, HDMI_VBP, and other parameter signals are input to the module and can be configured with corresponding configuration information based on video parameters and HDMI transmitter models, providing broad applicability.
4 ALPHA Blending Module Design
This module displays up to 4 video data in specified coordinate ranges based on configuration information (input video coordinate information, alpha value) and performs ALPHA blending according to the set transparency. First, the synchronization signals DE_HDMI, HSY_HDMI, VSY_HDMI generated by the HDMI driving module are connected to the input of the coordinate indicating circuit, which defines two sets of counters x_cnt and y_cnt to count the x and y coordinates of the effective video output within the display area, incrementing x_cnt by 1 every clock cycle from the starting position of each row of effective video data, resetting after counting one row of effective video data, and y_cnt increments by 1 after completing each row of effective video transmission, resetting after field synchronization. The coordinate indicating circuit generates effective video coordinate signals, allowing for the issuance of read request signals at the appropriate times based on the set video position information to read effective video data and simultaneously read the alpha values at the appropriate times.
The ALPHA blending calculation circuit is responsible for performing ALPHA blending between background video data and foreground video data according to the current alpha value. This module achieves hardware acceleration of ALPHA blending calculations through a multi-stage pipelined design, greatly enhancing calculation speed.
For RGB encoded video data, the formula for performing ALPHA blending is shown in Equation 1 (this formula also applies to YUV encoded video data).
Where R1, G1, B1 correspond to the R, G, B components of the blended video data, Rf, Gf, Bf correspond to the R, G, B components of the foreground data, Rb, Gb, Bb correspond to the R, G, B components of the background data. In the pipelined design, the combinational logic is divided into three stages: the first stage performs multiplication on the expanded data via multipliers, the second stage conducts addition, and the third stage performs reduction. The block diagram of this module is shown in Figure 3.
Since FPGA designs belong to digital circuit designs, calculations cannot directly compute decimals at the circuit level. Therefore, the method of first expanding the calculation data, completing calculations, and then reducing is adopted. Specifically, the method is to first expand both sides of the formula by 256 times, complete the calculations, and then reduce by 256 times, dividing the expanded formula into three calculation steps.
First Step: Perform multiplication operations as shown in Equations (2) and (3).
Second Step: Perform addition operations, adding the intermediate values R11, G11, B11 of the foreground data calculated in the first stage with the intermediate values R12, G12, B12 of the background data, as shown in Equation 4.
Third Step: Perform reduction operations, right-shifting the intermediate values obtained in the second step by 8 bits. To ensure data accuracy, overflow processing is performed on the obtained data. The code is as follows:
assign o_data_R = data_R_tmp[8] ? 8’hff : data_R_tmp[7:0];
assign o_data_G = data_G_tmp[8] ? 8’hff : data_G_tmp[7:0];
assign o_data_B = data_B_tmp[8] ? 8’hff : data_B_tmp[7:0];
Where o_data_R, o_data_G, o_data_B are the R, G, B components of the video after completing ALPHA blending and overflow processing, and data_R_tmp, data_G_tmp, data_B_tmp are the R, G, B components of the video after ALPHA blending but without overflow processing. Overflow processing checks whether the highest bit of the 9-bit video data output from the pipeline is 0; if not, output 8’hff; otherwise, output the first 8 bits of data.
5 Verification Results
This module is used as a display module for a four-channel video processing system based on SOPC, compiled and configured onto an FPGA, and run on the development board. The HDMI transmitter used is ADV7513, and the loaded configuration information is for 4-channel split display, directly displaying the output video data on the monitor, with the actual effect shown in Figure 4.
Due to experimental limitations, there is only one signal source; therefore, this signal source is connected to all four channels, and it can be observed that each channel’s video can be fully displayed. When the configuration information is switched to PIP display, the overlapping part of the background has a transparency of 0, and the window has a transparency of 1, with the actual effect shown in Figure 5.
It can be seen that the overlapping part of the window can be fully displayed, and the background is completely invisible. The entire system works normally, and the module has achieved the corresponding functionality.
6 Conclusion
This paper designs an HDMI multi-mode display module based on FPGA, which can drive HDMI output to display multiple video signals and can configure the display position of each video as well as the transparency of the overlapping parts. By setting the driving information, the module can be adapted to various models of HDMI transmitters and videos with different parameters. The design enhances processing speed through pipelining, strengthening the immediacy of the display. Therefore, in SOPC-based video processing systems, it can serve as a universal, high-speed, multifunctional HDMI output display component with good application prospects.
References
[1] Liu Ziyan, Feng Liang, Qi Jia. Design of a hardware platform for a real-time video tracking system based on FPGA[J]. Sensors and Microsystems, 2014, 33(7): 98-102.
[2] Li Huang. Design and implementation of an HDMI display system based on FPGA[D]. Shanghai: East China Normal University, 2008.
[3] Liang Yi, Tao Tangyao, Shi Weiya, et al. Design of a Camera Link-HDMI high-definition video converter based on Cyclone IV[J]. Application of Electronic Technique, 2013, 39(5): 12-14.
[4] Chen Zhijie. Research and FPGA implementation of high-definition mixed video matrix[D]. Quanzhou: Huaqiao University, 2014.
Author Information:
Xiang Zihao, Lu Anjiang
(School of Big Data and Information Engineering, Guizhou University, Guiyang, Guizhou 550025)
