Click the blue text above to follow for more exciting content.
This article contains a total of 825 words, and reading it will take about 2 minutes.
The power consumption of GPUs is very high because they cannot effectively utilize on-chip memory and need to frequently read from off-chip DRAM. Although their advantage in throughput allows GPUs to almost monopolize the deep learning field, the reliance on off-chip storage makes them significantly weaker than FPGAs in terms of power consumption and latency.
Taking NVIDIA’s GPU as an example, training with CUDA involves four main steps: 1) Copying data from the CPU’s external storage (DRAM) to the GPU’s memory; 2) The CPU loads (Launch) the computations to be performed, i.e., the Kernel to the GPU; 3) The GPU executes the instructions sent by the CPU; 4) The GPU finally writes the results back to the CPU’s DRAM, and then proceeds to the next Kernel computation.
Therefore, CUDA involves two read/write operations to storage. In contrast, FPGAs can cache the results of the first Kernel in the on-chip distributed BRAM (Block Random Access Memory, a dedicated RAM resource within the FPGA), allowing the entire algorithm to be completed without needing to read/write external storage.
The energy consumed in reading from DRAM is over 100 times that of SRAM, and 6400 times for addition operations. The GPU’s need to frequently read from DRAM results in its power consumption being much higher than that of FPGAs, and the bandwidth of DRAM often becomes a performance bottleneck. A typical power consumption for an FPGA is usually between 30W and 50W, while a single GPU can consume as much as 250W to 400W, leading to a power density of up to 28kW in a single cabinet. This creates significant pressure on the existing cooling systems in data centers, often requiring specialized modifications to the cooling and power supply systems to accommodate power densities exceeding 15kW per cabinet, whereas the tens of watts consumed by FPGAs can be compatible with existing data center cooling without requiring additional modifications.
FPGAs can flexibly utilize on-chip storage, resulting in power consumption that is far lower than that of GPUs. FPGAs can complete the entire algorithm on-chip without needing to read from DRAM. For example, Deep Insight Technology has developed an ESE model using FPGAs and tested it on different processors (CPU/GPU/FPGA), finding that the training time on FPGAs was the shortest and energy consumption was the least. In terms of energy consumption, CPU Dense consumes 11W, CPU Sparse consumes 38W, GPU Dense consumes 202W, which is the highest consumption scenario, while GPU Sparse consumes 136W. In contrast, FPGAs only require 41W; in terms of training latency, FPGAs take 82.7μs, which is far less than the CPU’s 6017.3μs, and only one-third of the GPU’s training time.
Risk Warning: This content only represents the analysis, speculation, and judgment of the Breakthrough Research, and is published here solely for the purpose of conveying information, not as a basis for specific investment targets. Investment carries risks; proceed with caution!Copyright Statement: The copyright of this content belongs to the original party or author. If reproduced, please indicate the source and author, retain the original title and ensure the integrity of the article content, and bear legal responsibilities for copyright and other issues.
END
