Understanding FPGA and Heterogeneous Computing

For a software developer, you may have heard of FPGA, and even used FPGA for computer architecture verification in college course design, but your first impression may be that this is something for hardware engineers.

Currently, with the rise of artificial intelligence, GPUs have taken the historical stage thanks to deep learning, and they are actively running various businesses, from training to inference. FPGA is also gradually entering data centers, leveraging its advantages. So next, let’s talk about how FPGA can enable programmers to develop more friendly without having to write tedious RTL code and without using simulation software like VCS or Modelsim, making unit tests easy to achieve.

The shift in this programming philosophy is due to FPGA’s implementation of programming through OpenCL, allowing programmers to program FPGA simply by adding appropriate pragmas in C/C++. To achieve higher performance for your FPGA applications implemented with OpenCL, you need to be familiar with the hardware introduced below. Additionally, we will introduce compilation optimization options that help better implement RTL conversion and mapping for your OpenCL applications and deploy them to FPGA for execution.

FPGA is a high-specification integrated circuit that can achieve functions of infinite precision through continuous configuration and combination, as it does not have fixed bit widths for basic data types like CPUs or GPUs. Instead, FPGA can perform very flexibly. In the use of FPGA, it is particularly suitable for some low-level operations, such as bit masking, shifting, and addition, which can be easily implemented.

To achieve parallel computing, FPGA contains look-up tables (LUTs), registers, on-chip memory, and arithmetic operation cores (such as digital signal processing (DSP) blocks). These internal modules in FPGA are connected through a network, and through programming, the connections can be configured to achieve specific logical functions. This reconfigurable characteristic of network connections provides FPGA with high-level programmability. (The programmability of FPGA is reflected in changing the connections between various modules and logical resources.)

For example, the programmability of LUTs in FPGA can be equivalently understood by programmers as a memory (RAM). A 3-bit input LUT can be equivalently understood as a memory with 3 address lines and 8 one-bit storage units (an array of length 8, where each element is 1 bit). When needing to implement a 3-bit bitwise AND operation, the array of length 8 stores the bitwise AND results of the 3-bit input numbers, totaling 8 possibilities. When needing to implement a 3-bit bitwise XOR, the array of length 8 stores the bitwise XOR results of the 3-bit input numbers, also totaling 8 possibilities. Thus, within one clock cycle, the bitwise operations of 3 bits can be obtained, and different functional bitwise operations can be achieved, which is completely programmable (equivalent to modifying the values in RAM).

Example of a 3-bit input LUT implementing bitwise AND:

Note: 3-bit input LUT look-up table

We see that the three-input bitwise AND operation is implemented through LUT in FPGA.

As shown above, a 3-input, 1-output LUT implementation. When LUTs are combined in parallel, series, and other ways, more complex logical operations can be achieved.

For traditional FPGA development and software development, the toolchain can be simply compared in the following table:

Note: Comparison table of traditional FPGA and software development

It is important to highlight that the synthesis phase of compilation is significantly different from software development compilation. General processors like CPUs and GPUs are already produced ASICs with their own instruction sets. However, for FPGA, everything is blank; there are only components, and nothing exists, but one can create any structural form of circuits, providing very high freedom. This freedom is both an advantage of FPGA and a disadvantage in the development process.

Writing this reminds me of a recent meme from “The Mysterious Programmers”:

Note: Comic source “The Mysterious Programmers 56” by Xi Qiao

Traditional FPGA development is like being 10 years old with Linux; to eat a cake, you need to process the raw materials from scratch. FPGA is in this state; to implement an algorithm, you need to write RTL, design state machines, and verify correctness through simulation.

Complex systems require the use of finite state machines (FSMs), generally requiring the design of three parts of logic as shown in the diagram: combinational circuits, sequential circuits, and output logic. Combinational logic determines what the next state is, sequential logic stores the current state, and output logic combines combinational and sequential circuits to get the final output result.

Then, for a specific algorithm, design the flow of logic in the state machine:

The implemented RTL is as follows:

Traditional RTL design is simply a nightmare for programmers, and the toolchain is completely different, development thinking is entirely different, and timing analysis is required. If a clock cycle is off, everything must be redone and re-verified; it all seems too low-level and inconvenient. So, let’s leave this to professional FPGAers. The following introduces OpenCL development for FPGA, which is somewhat like a 25-year-old Linux. With a higher level of abstraction, it is naturally more convenient to use.

OpenCL has injected fresh blood into FPGA development, a programming language oriented towards heterogeneous systems, with FPGA as a selectable device for heterogeneous implementation. The CPU Host controls the entire execution flow of the program, while the FPGA Device serves as a means of heterogeneous acceleration. The heterogeneous architecture helps to free the CPU from processing methods that it is not good at, delegating those to the Device side. Currently, typical heterogeneous devices include: GPU, Intel Phi, and FPGA.

Note: Quoted from wiki

Open Computing Language (OpenCL) is a framework for writing programs that execute across heterogeneous platforms consisting of central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), field-programmable gate arrays (FPGAs) and other processors or hardware accelerators. OpenCL specifies a programming language (based on C99) for programming these devices and application programming interfaces (APIs) to control the platform and execute programs on the compute devices. OpenCL provides a standard interface for parallel computing using task-based and data-based parallelism.

In summary: OpenCL is a framework for programming heterogeneous platforms, with the main heterogeneous devices being CPUs, GPUs, DSPs, FPGAs, and other hardware accelerators. OpenCL develops device-side code based on C99 and provides corresponding APIs for invocation. OpenCL provides standard interfaces for parallel computing to support task parallelism and data parallelism.

Here we analyze the matrix multiplication case from the Altera official website. You can download the case via the following link: Altera OpenCL Matrix Multiplication.

The code structure is as follows:

Among them, the code related to FPGA is matrix_mult.cl, which describes the kernel function. This part of the function will generate RTL code through the compiler and then map it to FPGA circuits.

The definition of the kernel function is as follows:

The pattern is quite fixed; it should be noted that __global indicates data coming from the CPU, stored in global memory, which can be FPGA on-chip storage resources, DDR, QDR, etc., depending on the FPGA’s OpenCL BSP driver. num_simd_work_items indicates the width of SIMD. reqd_work_group_size indicates the size of the workgroup. These concepts can be referenced in the OpenCL usage manual.

The function implementation is as follows:

For simulation, the programmer does not need to worry about how the timing flows; they only need to verify the logical function. The Altera OpenCL SDK provides the functionality of a CPU emulation device, which can be done as follows:

In the above script, the -march=emulator sets up a device executable file for CPU debugging. -g adds the debugging flag. —board is used to create a debugging file that matches the device. CL_CONTEXT_EMULATOR_DEVICE_ALTERA indicates the number of devices for CPU emulation.

After executing the above script, the output is as follows:

When setting Device = 8 during simulation, it simulates 8 devices running (512, 512) * (512, 512) matrix scale, and finally verifies correctness. Next, we can compile it to run on the FPGA device.

At this point, we are ready to download the code to execute on the FPGA. The only thing needed to do is use the compiler provided by the OpenCL SDK to adapt the *.cl code to the FPGA, executing the compilation command as follows:

This process is relatively slow and generally takes several hours to over ten hours, depending on the size of resources on the FPGA. (Currently, this part of the time is temporarily unsolvable, because the compilation here is actually creating a circuit that can work properly, and the software will perform layout and routing work.)

After waiting for the compilation to complete, the generated matrix_mult.aocx file can be burned to the FPGA.

The burning command is as follows:

At this point, the task is completed, and the host-side program can be run:

It can be seen that matrix multiplication can run normally on FPGA, with a throughput of about 119 GFlops.

From the development process described above, OpenCL greatly liberates FPGA developers from development cycles, and for software developers, it is also relatively easy to get started. This is its advantage, but currently, there are still some issues during development, such as insufficient compiler optimization, leading to performance gaps compared to RTL writing; the compilation time to the Device side is too long. However, these will gradually improve with the development of the industry.

Additionally, for those interested in FPGA or those who have used FPGA for solutions, feel free to discuss together.