Author | Ben Dickson
Translator | Daxiaofei
Editor | Chen Si
The rise of artificial intelligence has triggered a massive demand for GPUs in the market, but the application of GPUs in AI scenarios faces issues such as short lifespan and high usage costs. Field Programmable Gate Arrays (FPGAs), which are customizable hardware processors, may be a better solution. With the resolution of programmability issues on FPGAs, they are set to become the choice for AI applications in the market.
Field Programmable Gate Arrays (FPGAs) address many of the issues faced by GPUs when running deep learning models.
In the past decade, the resurgence of artificial intelligence has greatly benefited the graphics card industry. Companies like Nvidia and AMD have seen their stock prices soar as it has become evident that their GPUs are effective in training and running deep learning models. In fact, Nvidia has transformed its business; it was previously a company focused solely on GPUs and gaming, but now, in addition to being a cloud GPU service provider, Nvidia has also established a dedicated AI research lab.
However, Ludovic Larzul, CEO and co-founder of machine learning software company Mipsology, states that GPUs still have some flaws that pose challenges in AI applications.
Larzul indicates that the solution to these problems is to implement Field Programmable Gate Arrays (FPGAs), which is also the research focus of their company. FPGAs are processors that can be customized after manufacturing, making them more efficient than standard processors. However, programming FPGAs is challenging, and Larzul hopes to address this issue with a new platform developed by his company.
Specialized AI hardware has become a standalone industry, but there is still no consensus on what constitutes the best infrastructure for deep learning algorithms. If Mipsology successfully completes its research experiments, many AI developers currently struggling with GPUs will benefit.
Challenges Faced by GPU Deep Learning
Three-dimensional graphics are the fundamental reason GPUs possess such large memory and computational power, sharing a commonality with deep neural networks: both require extensive matrix operations.
Graphics cards can perform matrix operations in parallel, significantly accelerating computation speed. Graphics processors can reduce the time required to train neural networks from days or weeks to just hours or minutes.
As the supply of graphics hardware continues to increase, the market demand for GPUs in deep learning has also spawned a plethora of public cloud services that provide powerful GPU virtual machines for deep learning projects.
However, graphics cards are also limited by hardware and environmental factors. Larzul explains, “Neural network training typically occurs in a controlled environment, and the systems running neural networks encounter various constraints during deployment—this can put pressure on the actual use of GPUs.”
GPUs require substantial power, generate significant heat, and necessitate cooling with fans. This is not a major issue when training neural networks on desktop workstations, laptops, or rack servers. However, many environments deploying deep learning models are not friendly to GPUs, such as autonomous vehicles, factories, robots, and many smart city environments, where hardware must endure environmental factors like heat, dust, humidity, motion, and power limitations.
Larzul states, “In some critical application scenarios, such as video surveillance in smart cities, hardware is required to be exposed to environmental factors (like sunlight) that adversely affect GPUs. GPUs are limited by transistor technology, which means they need timely cooling when operating at high temperatures, which is not always feasible. Achieving this requires more power, maintenance costs, etc.”
Lifespan is also a concern. Generally, the lifespan of a GPU is about 2-5 years, which is not a significant issue for gamers who upgrade their computers every few years. However, in other fields, such as the automotive industry, hardware durability is paramount, which presents challenges. Particularly, excessive exposure to harsh environments, combined with high-intensity usage, will shorten the lifespan of GPUs.
Larzul notes, “From a commercial viability perspective, applications like autonomous vehicles may require up to 7-10 GPUs (most of which will fail in less than four years), making the cost of smart or autonomous vehicles impractical for most buyers.”
Other industries, such as robotics, healthcare, and security systems, face similar challenges.
FPGA and Deep Learning
FPGAs are customizable hardware devices that can adjust their components, allowing them to be optimized for specific types of architectures (such as Convolutional Neural Networks). Their customizable features reduce power requirements and provide higher performance in terms of computation speed and throughput. They also have a longer lifespan, approximately 2-5 times that of GPUs, and exhibit greater adaptability to harsh environments and other special environmental factors.
Some companies have already integrated FPGAs into their AI products. Microsoft is one such company, offering FPGA-based machine learning technology as part of its Azure cloud service products.
However, the drawback of FPGAs is their programming difficulty. Configuring an FPGA requires knowledge and expertise in hardware description languages (such as Verilog or VHDL). Machine learning programs are typically written in high-level languages like Python or C, making it very challenging to convert their logic into FPGA instructions. Running neural networks modeled in TensorFlow, PyTorch, Caffe, and other frameworks on FPGAs often requires a significant amount of manpower and time.
“To program an FPGA, you need to assemble a team of hardware engineers who know how to develop FPGAs and hire an excellent architect who understands neural networks, spending years developing a hardware model that ultimately compiles and runs on the FPGA, while also addressing issues of FPGA efficiency and usage frequency,” Larzul states. Additionally, you need extensive mathematical skills to accurately compute models with lower precision and a software team to map AI framework models to hardware architectures.
Larzul’s company, Mipsology, aims to bridge this gap with Zebra. Zebra is a software platform that allows developers to easily port deep learning code to FPGA hardware.
Larzul states, “We provide a software abstraction layer that hides the complexities that typically require advanced FPGA expertise. Just load Zebra, input a Linux command, and Zebra can get to work—it doesn’t require compilation, no changes to the neural network, and no need to learn any new tools. However, you can keep your GPU for training.”
Zebra provides an abstraction layer for converting deep learning code into FPGA hardware instructions.
Prospects for AI Hardware
Mipsology’s Zebra platform is one of many solutions for developers exploring the use of FPGAs in AI projects. Xilinx, a leader in the FPGA field, has developed Zebra and integrated it into their circuit boards. Other companies, such as Google and Tesla, are also actively developing dedicated AI hardware for their cloud products and edge computing environments.
There have also been developments in neuromorphic chips, which are computer architectures specifically designed for neural networks. Intel is at the forefront of neuromorphic computing, having developed several model architectures, although this field is still in its early stages of development.
There are also application-specific integrated circuits (ASICs), which are chips manufactured for specific AI needs. However, ASICs lack the flexibility of FPGAs and cannot be reprogrammed.
Larzul concludes, “We have decided to focus on the software business, exploring research to enhance neural network performance and reduce latency. Zebra runs on FPGAs, so it can support AI inference without changing hardware. Each refresh of the FPGA firmware can bring us higher performance improvements, thanks to its efficiency and shorter development cycles. Additionally, FPGAs offer many selectable options, providing good market adaptability.”
Original English Text:
https://bdtechtalks.com/2020/11/09/fpga-vs-gpu-deep-learning/
Are you also “watching” this? 👇