Highlights: In the fiercely competitive AI chip market, FPGA has quietly carved out a niche.

Introduction: The concept of “AI chips” has gradually gained popularity over the past year, becoming more familiar to the public. As the industry transitions from a phase of rapid growth to one of accelerated implementation and integration, more AI chip companies are beginning to carve out their own differentiated paths.

After the first season of its AI chip series report, Zhiyuan is once again embarking on a deep dive into nearly a hundred core companies across the entire AI chip industry chain for differentiated coverage. This is one of the second season reports on the AI chip industry by Zhiyuan.

With the development of big data and the increase in computing power, AI chips are ushering in a new wave of explosive growth.

In the past two years, an increasing number of players have emerged in the AI chip space using FPGA, including domestic AI chip startups like Deep Insight Technology, Baidu XPU, Microsoft Project Brainwave, and specialized AI chips for medical imaging from Deep Thinking.

Although ASICs often perform better in terms of performance and power consumption, we find that in the fiercely competitive AI chip market, FPGA has secured a place due to its unique advantages. Some media outlets even report that FPGA is the ultimate future of AI chips.

Whether this viewpoint is accurate is beside the point, but it has indeed stirred up quite a bit of discussion in the AI chip sector, prompting us to ask, “Is FPGA truly the new savior of AI chips?”

FPGA’s KFC and McDonald’s: Xilinx vs Intel Altera

FPGA (Field-Programmable Gate Array) is a flexible computing architecture that lies between general processors like CPUs and GPUs and dedicated integrated circuits (ASICs). It allows users to program hardware flexibly while maintaining fixed hardware. Its development cycle is shorter than that of ASICs, but the cost of individual FPGAs is higher compared to mass-produced ASICs.

Due to its larger fault tolerance, FPGA has traditionally been used as a hardware verification method before the tape-out of ASIC chips.

Currently, the four main FPGA manufacturers are based in the United States: Xilinx, Altera, Lattice, and Microsemi. Together, they hold over 98% of the market share, with Xilinx and Altera dominating the market, holding approximately 49% and 39% of the market, respectively.

In the past three years, the FPGA industry has seen multiple mergers and acquisitions. In June 2015, Intel announced its acquisition of Altera for $16.7 billion. In the first half of 2016, Tsinghua Unigroup acquired 6.07% of Lattice’s shares on the open market, and in November of the same year, Lattice was acquired by Canyon Bridge for $1.3 billion, although this deal has yet to receive approval from U.S. regulators.

These mergers have not affected the overall landscape of FPGA in the AI chip market. The two giants, Xilinx and Intel, have established similar FPGA strategic layouts, focusing on the data center market and striving to simplify FPGA programming.

1. The Creator of FPGA – Xilinx

When talking about FPGA, one cannot avoid mentioning Xilinx, as this chip technology was pioneered by them. Since 2011, Xilinx has proposed the concept of All Programmable, expanding FPGA technology from traditional fields such as communications, aerospace, and defense into applications in AI and cloud computing.

Last December, Zhiyuan conducted an in-depth interview with Zhou Haitai, Senior Director of Global Sales and Marketing for Xilinx in the Asia Pacific and Japan region. Zhou revealed that their 16nm products are the most popular among AI application manufacturers, as their cutting-edge technology can integrate more programmable logic elements into chips to meet the strong computing power demands of AI.

As a long-standing leader in the FPGA industry, Xilinx has over 20,000 downstream customers, including Amazon AWS and China’s BAT cloud service giants.

In March of this year, shortly after Victor Peng took over as the new CEO of Xilinx, he announced the company’s three major strategic layouts for the AI era: priority on data centers, accelerating the development of eight mainstream markets, and launching a new generation of AI chip architecture called ACAP.

The first ACAP AI chip, codenamed “Everest,” will be built using TSMC’s 7nm process and is expected to tape out this year, with deliveries to customers in 2019.

In the area of edge AI, Xilinx acquired Deep Insight Technology, one of China’s three AI chip unicorns, this year, focusing on terminal AI. Deep Insight is one of the few players in China that has achieved good results in the AI chip space using FPGA.

Additionally, Xilinx’s FPGA has long been deeply involved in the automotive sector, suitable for handling increasingly complex advanced driver-assistance systems (ADAS) and autonomous driving.

A CEO of an AI chip startup revealed that compared to other AI chip companies entering the autonomous driving market, only Xilinx has automotive-grade FPGAs, and there is no other platform with a better cost-performance ratio for L1-L3 levels, indicating that Xilinx has a good market opportunity for at least the next five years.

2. Intel’s Acquisition of Altera

Another major event in the FPGA industry is Intel’s acquisition of Altera. To enhance its competitiveness in the AI chip space, Intel acquired Altera for a staggering $16.7 billion in 2015, marking Intel’s largest acquisition to date. Intel subsequently established a department for the R&D of FPGA chips.

FPGA is one of Intel’s AI chip strategies, but its most important AI chip project has been the long-delayed Nervana neural network processor project.

Intel has adopted a dual approach to its FPGA strategy: on one hand, it is creating CPU-FPGA hybrid devices that allow FPGA and processors to work together; on the other hand, it is building programmable acceleration cards (PAC) based on Arria FPGA or Stratix FPGA.

Based on FPGA technology, Intel has built a robust NFV ecosystem that covers software and hardware vendors, system integrators, telecommunications operators, OTT vendors, and other related enterprises.

Furthermore, Intel has provided a complete FPGA solution stack for its customers, including Intel Xeon Scalable processors that contain FPGA, to OME vendors.

After acquiring the chip company eASIC in July, Intel integrated it into the PSG department and revealed that this acquisition primarily aimed to address customer pain points, meeting the cost and energy consumption reduction needs of FPGA clients, and providing scalable technology to reduce the costs of FPGA products with 16nm, 10nm, and 7nm processes.

Currently, acceleration cards remain one of the main forms of FPGA entering the hardware field, but Intel is exploring other directions and forms, actively promoting collaborations with other data center OEMs in the FPGA space.

Major Players in the FPGA AI Chip Market

In the past two years, the application of FPGA in data centers has become increasingly widespread. Currently, FPGA servers have been deployed in seven major super cloud computing data centers globally: IBM, Facebook, Microsoft Azure, AWS, Baidu Cloud, Alibaba Cloud, and Tencent Cloud. Due to the high risks associated with ASIC routes, only Google has deployed TPUs in bulk.

1. The Most Proficient in FPGA – Microsoft AI: Project BrainWave

Microsoft Azure’s relationship with FPGA dates back eight years, when the Bing search engine fell short of Google in both search results and response speed. To meet Bing’s needs, a Microsoft researcher proposed a hardware design that could run Bing’s machine learning algorithms on FPGA, which received Microsoft’s approval at the end of 2010.

Today, FPGA’s outstanding performance in accelerating data processing has led to its deployment in Microsoft’s Bing, Azure cloud services, and Office 365.

On March 26 of this year, Microsoft’s official Weibo account announced the launch of the Project Brainwave deep learning acceleration platform, built using Intel Arria FPGA and Stratix FPGA chips. This platform integrates DNN processing units into FPGA and supports popular deep learning frameworks like Microsoft’s Cognitive Toolkit and Google’s TensorFlow, capable of executing large-scale machine reading comprehension tasks required for Bing’s intelligent search functions.

At the Microsoft Build 2018 conference in May, the Brainwave AI chip was opened for cloud trial use, allowing developers to access AI services provided by the Project Brainwave chip computing platform via Microsoft Azure cloud. Microsoft claims its latency is five times lower than Google TPU, directly competing with Google. Project Brainwave can be seen as a low-latency AI chip built on Intel FPGA technology.

Overall, Microsoft’s FPGA-based AI chip is one of the most recognized FPGA solutions in the industry.

2. Baidu XPU

In contrast to Microsoft’s focus on Intel FPGA, Baidu has a more open attitude towards FPGA manufacturers. Since 2011, Baidu has been applying FPGA across multiple core businesses such as search, image, and voice, deploying a large number of FPGAs in its data centers, cloud computing platforms, and autonomous driving projects.

At last year’s Hot Chips conference in California, Baidu unveiled the XPU, a 256-core cloud computing acceleration chip developed in collaboration with Xilinx. Leveraging the strengths of FPGA accelerators in handling computational tasks, XPU combines the versatility of GPUs with the high efficiency and low power consumption of FPGAs, aiming to process various computational tasks, optimize and accelerate Baidu’s deep learning platform PaddlePaddle, and achieve a balance of performance and efficiency.

However, at this year’s Baidu AI Developer Conference, Baidu launched its first cloud-based ASIC AI chip, Kunlun, which is said to be derived from Baidu’s XPU technology.

Additionally, Baidu and Intel have a collaboration history spanning over ten years, with their cooperation in AI strengthening in recent years.

At this year’s Baidu AI Developer Conference, Gadi Singer, Vice President of Intel’s AI Division and General Manager of AI Architecture, revealed that Baidu is developing a heterogeneous computing platform based on Intel’s latest FPGA technology, which will flexibly accelerate workloads on Baidu Cloud.

3. Deep Insight Technology

Deep Insight Technology is a young but impressive AI chip company in China, established two years ago, with a valuation exceeding $1 billion last year. On July 18 of this year, it was acquired by Xilinx, the world’s largest FPGA manufacturer, and its research achievements have won awards at top international AI conferences.

The high difficulty and long cycle of FPGA development (generally taking 3-6 months) have deterred many companies, but Deep Insight has identified an opportunity from this pain point. Since its establishment in 2016, Deep Insight Technology has been developing machine learning solutions based on the Xilinx FPGA technology platform. The neural network pruning technology optimized by Deep Insight can run on Xilinx FPGA devices, achieving better performance and energy efficiency.

The company has proposed a series of Deep Learning Processing Units (DPU) that provide a black box with interfaces for manufacturers that do not want to use FPGAs directly. Manufacturers only need to train their models, import the model and data from the interface, and they can directly obtain the required output. Deep Insight handles the compression and compilation, generating instructions that can run on FPGA.

▲DPU Series: Aristotle board for edge applications (top) and Descartes board for big data applications (bottom)

However, in the later stages, Deep Insight Technology has also begun to venture into ASIC AI chip design and tape-out, having previously released two AI chips, “Listening to the Waves” and “Observing the Sea,” announcing tape-out in 2018, although no related product information has been seen yet.

4. Tencent Cloud, Huawei Cloud, Alibaba Cloud

In recent years, Xilinx and Intel have been very active in the Chinese cloud service market, collaborating with top domestic cloud computing service providers such as Huawei Cloud, Alibaba Cloud, and Tencent Cloud.

At the beginning of last year, Tencent Cloud released China’s first high-performance heterogeneous computing infrastructure – FPGA cloud servers, followed by the launch of FX3 instances based on Xilinx VU9P FPGA cards and FI3 instances based on Intel Stratix10 FPGA cards.

Huawei Cloud also announced the launch of the Huawei Cloud FPGA Accelerated Cloud Server (FACS) platform in Europe in early this year, in collaboration with Xilinx FPGA, equipped with Xilinx’s high-performance Virtex UltraScale+ VU9P FPGA, providing users with high-performance FPGA accelerated cloud services. Huawei Cloud also launched FPGA accelerated cloud server FP1 DPDK instances and FP1 OpenCL instances.

In May of this year, Alibaba Cloud released the next-generation FPGA computing instance F3, using its self-developed high-performance FPGA acceleration card, equipped with Xilinx’s 16nm Virtex UltraScale+ device VU9P, offering 16 specifications of VU9P strength, pioneering unified FPGA SHELL architecture and FPGA virtualization support solutions, enabling Alibaba Cloud customers to accelerate various workloads such as machine learning, data analysis, genomics, and video processing.

5. Other Players

Currently, domestic companies capable of producing FPGA include Unigroup Guoxin, Zhi Duo Crystal, and AgateLogic, among others. Domestic companies such as Deep Thinking AI, Rui Wei Intelligent, Deep Dimension Technology, Gao Yun Semiconductor, Anlu Technology, and Jingwei Yage have all announced their capabilities in developing FPGA-based AI chips.

Unigroup Guoxin is the only publicly listed company in China capable of mass-producing FPGAs, providing stable supplies of FPGA, ASIC, and special microprocessors to the military. The company officially launched China’s first high-performance FPGA chip with embedded high-speed interfaces (serdes) – Pango PGT180H, in September 2016.

Deep Thinking AI released the world’s first FPGA-based AI chip for medical imaging at the GTIC 2018 Global AI Chip Innovation Summit in March this year – M-DPU, which can classify 90,000 cells in 100 seconds, while also equipped with advanced medical imaging algorithm acceleration cores and deep learning acceleration cores.

CEO Yang Zhiming of Deep Thinking told Zhiyuan that the development cycle of FPGA is much faster than that of ASIC, making it suitable for the rapidly evolving algorithms in intelligent medical imaging. Deep Thinking’s DPU is not a general-purpose DPU, but a DPU tailored for specific application fields, with its greatest advantage in the deep integration of DPU with specific field algorithms, forming a one-stop overall solution for specific scenarios.

Rui Wei Intelligent is one of the earliest AI chip companies in China to utilize FPGA technology, having developed FPGA-based AI chips back in 2015 to meet the high computing power and low power consumption needs of front-end cameras, achieving large-scale commercial use in 2016. Its AI chips enable cameras to process images at 25 frames per second, extracting clear facial images for cloud-based recognition and analysis.

In addition to startups, domestic research institutions such as Peking University, Tsinghua University, and the Chinese Academy of Sciences have conducted in-depth research in the field of FPGA-based AI chips. For example, Peking University, in collaboration with SenseTime, proposed a fast Winograd algorithm based on FPGA that significantly reduces algorithm complexity, improving CNN performance on FPGA.

Development Trends of FPGA in the AI Chip Industry

Currently, FPGA in the AI chip industry is showing several different development trends: one is to launch optimized architectures based on FPGA, another is to maximize the usage boundaries of FPGA, and a third is to use FPGA as a stepping stone to gradually transition to dedicated custom chips (ASIC).

1. Optimized Architectures Based on FPGA

The development route of optimized architectures based on FPGA is represented by Xilinx. In response to the issues of high costs and development difficulties, Xilinx launched a new generation of AI chip architecture ACAP in March this year, which took four years and over $1 billion to develop, directly challenging NVIDIA and Intel processors.

2. Long-Term Maximization of FPGA Usage

Microsoft is a typical player that maximizes the usage boundaries of FPGA over the long term. For the past eight years, Microsoft has been dedicated to using FPGA as its core AI computing platform. From using dedicated servers filled with FPGAs to FPGA accelerator card clusters connected via dedicated networks, and now to large-scale FPGA cloud services in shared data center networks, Microsoft has been exploring various possibilities for FPGA deployment, seeking the optimal path to leverage FPGA advantages.

3. Is FPGA’s Endpoint ASIC?

Currently, the mainstream view in the industry is that as AI algorithms mature and solidify, AI chips will ultimately transition to ASICs, which have more fixed chip architectures.

However, Xilinx CEO Victor Peng holds a different view, telling Zhiyuan that the speed of algorithm iteration and application innovation will continue for a long time, possibly maintaining for another 10-20 years, making a flexible and adaptable computing platform the key to innovation.

Moreover, it is unlikely that a single AI chip architecture will dominate the market in the future; we will increasingly need new chip architectures that can adapt to numerous AI applications and the constantly changing AI algorithms.

Meanwhile, Liu Bin, the market development manager for Intel’s programmable solutions division in the Asia-Pacific region, also believes that FPGA will hold a certain market share across all market sectors. From a native market perspective, FPGA and ASIC will coexist in a competitive and cooperative symbiotic environment.

“Dedicated ASICs are still very competitive,” Liu said, “but FPGA will hold a share in all vertical markets, and this flexibility and universality is the main value of FPGA.”

A CEO of an AI chip startup told Zhiyuan that at least until deep learning algorithms stabilize, FPGA will have its survival space. Many companies have found that after solidifying traditional algorithms, their internal algorithm teams often change the algorithms within six months.

Yang Zhiming, CEO of Deep Thinking, believes that the integration of FPGA and ASIC is the future of AI computing. Pure ASIC chips cannot completely replace FPGA, nor do they possess FPGA’s flexibility. Just as ASICs have not been able to fully replace FPGA in traditional fields, in the AI field, FPGA and ASIC will also coexist and complement each other. Fields such as intelligent industrial robots, smart communications, intelligent medical care, and intelligent equipment are applications where other AI chips find it difficult to intervene, except for FPGA.

Conclusion: With Deep Learning Support, FPGA’s Future is Promising

In the past year, AI chips have entered a red ocean. In addition to traditional chip giants like Xilinx and Intel, numerous domestic and international AI chip startups have also joined the battlefield. The programmability of FPGA allows software and terminal application companies to flexibly modify solutions based on algorithms, making it increasingly prominent in the AI chip market.

Currently, AI is still in its early stages, transitioning from training to inference. This process requires compressing trained models to a fraction of their original size without significantly losing model accuracy. During this phase, AI is optimizing and upgrading in a way that favors FPGA development.

FPGA may be a niche chip, but its universality and flexibility give it suitable application scenarios to showcase its advantages in the AI market. Before a significant gap opens up between various AI chips, every technical route has space to create more value.

This account is a signed account of NetEase News – NetEase Number “Each has its own attitude”.