Editor’s Note: Since Xilinx created the FPGA in 1984, this programmable logic device has secured a place in fields such as communications, medical, industrial control, and security due to its advantages in performance, time-to-market, cost, stability, and long-term maintenance. It has also seen significant growth in recent years. In the last two years, with the boom in cloud computing, high-performance computing, and artificial intelligence, the attention on FPGAs has reached unprecedented heights. This article discusses the past, present, and future of FPGAs from a foundational perspective.
We know that compared to specialized ASICs, FPGAs have advantages in time-to-market and cost. Additionally, in most cases, FPGAs execute certain functions more efficiently than software operations on CPUs. This is why we believe they will not only be used in various corners of data center servers, switches, and storage layers but also have the capability to accelerate entire workflows.
However, we cannot be overly optimistic, especially after Intel acquired FPGA manufacturer Altera for $16.7 billion in December 2015.
At the end of 2014, the then-independent Altera company was eyeing the development prospects of CPU+FPGA parallel computing in data centers—a market valued at about $1 billion at the time. This was not the $250 million CPU-GPU market in data centers or the $9 billion market for directly applying CPU processors.
Altera made this decision because they believed this combination had programming simplicity and energy efficiency advantages over the other two options. People are very familiar with CPUs and find it not too difficult to find C programmers. Therefore, for most developers, continuously using this solution for executing computational tasks does not require taking too much risk, but energy efficiency is relatively low, especially under dense computation and inherent parallel workloads.
As for the CPU+GPU solution, programmers are not very familiar with it, but it has high efficiency.
According to Altera’s estimates, programming a hybrid CPU-FPGA system using OpenCL is easier for programmers than using Nvidia’s CUDA environment (some aspects are certainly debatable), but hard programming FPGAs with HDL is quite difficult, thus requiring OpenCL or other abstraction layers to transfer loads from the CPU to the FPGA.
Intel’s Acquisition of Altera Changes the FPGA Landscape
This billion-dollar data center market was divided among Altera, Xilinx, and other FPGA suppliers. After Intel acquired Altera in June 2015, this market became more complex.
In 2014, before the acquisition, 16% of Altera’s $1.9 billion revenue came from computing, networking, and storage businesses related to data centers, totaling $304 million. Those manufacturers of communication and wireless device systems, who have been deeply engaged in this field for over a decade, want higher energy efficiency, lower costs, and greater scalability—areas where FPGAs excel. Additionally, it is worth mentioning that using FPGAs to perform these functions does not require an operating system and corresponding software like CPUs do. This part of the revenue accounted for 44% of Altera’s revenue, totaling $835 million.
Another 22% of Altera’s revenue, or $418 million, came from industrial control, military equipment, and automotive manufacturing. They face the same dilemma and thus choose FPGAs to handle some of their workloads.
In fact, as early as 2014, Intel had its sights set on the potential $115 billion market for various types of chips. Among them, programmable logic devices (mainly FPGAs) accounted for about 4%, ASICs for 18%, and the rest was a mix of ASSPs.
In the field of programmable logic devices, Intel estimated that Altera held 39% of the $4.8 billion market, Xilinx held 49%, and the remaining suppliers accounted for the remaining 12%.
At that time, Intel did not acquire Altera because the growth rate of the FPGA business was almost as fast as that of its data center group (which provides chips, chipsets, and motherboards for server, storage, and switch manufacturers).
Furthermore, Intel did not do so because the slowing pace of Moore’s Law posed an increasing competitive threat to FPGAs.
In fact, if applied, data centers do not need to install multiple Xeon CPUs but can install one FPGA, GPU, or DSP accelerator. Since Intel cannot continue to provide more cores and accelerators for Xeons, they concluded that FPGAs should be treated as accelerators.
Unless FPGAs can generate $500 million in revenue in data centers or $1 billion or more in a few years, Intel would rather sacrifice two to three times the Xeon revenue than relinquish Xeon revenue.
With Deep Learning, FPGA’s Prospects Are Bright
According to Intel’s forecast, they plan to increase FPGA business at a nearly linear growth rate from now until 2023. We are always skeptical about this. However, FPGA business has grown somewhat over time (about 2.5 times compared to 15 years ago).
Intel also expects FPGA revenue to double between 2014 and 2023. According to Intel’s forecast, the compound annual growth rate from 2014 to 2023 is 7%, and its revenue should be slightly below the projected $8.9 billion. Interestingly, since Intel’s forecast does not include FPGA revenue share from data computing centers (servers, switches, and networks), this will change significantly. Let’s analyze:
If Altera and Xilinx’s market shares remain unchanged, and assuming Altera’s revenue in networking, computing, and storage remains constant, then Altera’s business revenue in this part will reach about $560 million by 2023. We believe Intel’s data underestimates the pressure data centers face in providing more efficient and flexible computing. We believe the prospects for FPGAs are far better than this forecast. In other words, many supporters of FPGA technology have long awaited the day when FPGAs will gain legitimate computing status in data centers.
Ironically, Intel itself, as an FPGA programming expert, a user of hardware description languages, and a well-known ASIC manufacturer, has become a major player in promoting FPGAs as the preferred choice for accelerators. Such accelerators can serve as independent discrete computing elements or as hybrid CPU-FPGA devices.
This is why since 2016, we have seen all news about Altera indicating that FPGAs will experience massive growth. So at least in the short term, they have little choice but to dress up other FPGA manufacturers.
This acquisition is not only a milestone in FPGA development but also Intel’s acknowledgment of the enormous potential of FPGAs. As powerful computing accelerators of the future, FPGAs will not only influence major corporate decisions and market trends but also accelerate workloads within enterprises, facilitate internal searches in large-scale data centers, and enhance the status of high-performance computing simulations.
As we crossed into 2016, FPGAs added machine learning and deep learning to their application ranks, which struck another heavy blow for the FPGA industry.
Why Everyone Favors FPGAs
First, the software stack for programming FPGAs has evolved, especially with Altera’s help, FPGAs have increased support for the OpenCL development environment. But not everyone is a fan of OpenCL.
Initially, Nvidia created its own CUDA parallel programming environment for its Tesla GPU accelerators. Then SRC Computers not only provided hybrid CPU-FPGA systems for defense and intelligent fields as early as 2002 but also further commercialized its Carte programming environment in mid-2016, which can automatically convert C and Fortran programs into FPGA’s hardware description language (HDL).
Another factor driving FPGA adoption is that as chip manufacturing technology becomes increasingly difficult to scale down, improving multi-core CPU performance is becoming more challenging. Although CPU performance has made significant leaps, it is mainly used to expand CPU performance throughput rather than the individual performance of single CPU cores (we know architectural enhancements are difficult). However, both FPGA and GPU accelerators have seen convincing improvements in performance per watt.
According to Microsoft’s running tests, the performance per watt of CPU-FPGA and CPU-GPU hybrid computing is comparable. GPUs run hotter and have similar performance per watt, but they also bring stronger working capabilities.
Improved performance per watt explains why the world’s most powerful supercomputers transitioned to parallel clusters in the late 1990s and why they are now turning to hybrid machines instead of Intel’s next Xeon Phi processor, which is primarily CPU-GPU hybrid, known as Knights Landing (KNL).
With the help of Altera FPGA coprocessors and Xeon Phi processors Knights Landing, Intel can not only maintain its competitive edge in the high-end market but also continue to lead in competition with the Open Power Alliance formed by Nvidia, IBM, and Mellanox.
Intel firmly believes that workloads in large-scale computing, cloud, and HPC markets will grow rapidly. To promote its computing business to continue thriving, it can only become a seller of FPGAs; otherwise, others will seize this only avenue.
But Intel does not say this to everyone. They state: “We do not see this as a defensive battle or anything else,” said Intel CEO Brian Krzanich at a press conference following the Altera acquisition announcement.
“We believe that both IoT and data centers are massive. These are also the products our customers want to build. 30% of our cloud workloads will be on these products, based on our predictions of how we see trend changes and market developments.
This is to prove that these workloads can be transferred to silicon in one way or another. We believe the best approach is to use a combination of Xeon processors and FPGAs with the best performance and cost advantages. This will bring better products and performance to the industrial sector. In IoT, this will expand the potential market against ASICs and ASSPs; in data centers, it will shift workloads to silicon, driving rapid growth in the cloud.
Krzanich explained: “You can think of FPGAs as a bunch of gates that can be programmed at any time. According to their ideas, their algorithms will become smarter over time and with learning. FPGAs can serve as accelerators in multiple fields, performing facial searches while encrypting, and can essentially reprogram FPGAs in microseconds. This is much cheaper and more flexible than the cost of large-scale single custom components.”
Intel Sees a Bigger Opportunity
Intel sees a bigger opportunity.
Intel CEO Brian Krzanich announced after the acquisition was completed that by 2020, up to one-third of cloud service providers will use hybrid CPU-FPGA server nodes, which is shocking news. This also gives Altera, which has been targeting data centers since the end of 2014, about $1 billion in FPGA opportunities. This amount is about three times Nvidia’s current popular Tesla computing engine revenue.
At the beginning of 2014, Intel showcased a prototype of a Xeon-FPGA chip in the same package and planned to launch this chip in 2017. This was shortly after the proposal of a Xeon concept with FPGA circuits put forward by Diane Bryant, GM of the data center group at the time.
During the conference announcing the Altera deal, Krzanich did not specify the timeline for launching this Xeon-FPGA device, but he stated that Intel would create a single-die hybrid Atom-FPGA device aimed at the IoT market. Intel is exploring whether a single package hybrid for Atom and Altera FPGA is needed during the transitional phase.
In early 2016, during a conference call with Pacific Crest Securities, Intel’s cloud infrastructure group general manager Jason Waxman discussed Intel’s data center business with research analysts, stating that FPGAs had become a hot topic.
First, although he did not name any specific manufacturers or specifications of any devices, Waxman confirmed that Intel had already provided hybrid computing engine samples of Xeon plus FPGA to certain customers.
During the meeting, Waxman elaborated on the reasons driving Intel’s acquisition of Altera and its entry into programmable computing devices. Intel clearly wants to make FPGAs mainstream, even if this may cannibalize some of Xeon’s business in data centers. (We believe that Intel thinks this infighting is inevitable, and the best way to control it is to make FPGAs part of the Xeon lineup.)
Waxman said: “I think this acquisition may involve many things, and some of them have already gone beyond the scope of the data center group.”
First, a potential core business is often driven by manufacturing leadership advantages. In this regard, we can control it well, and doing so has good synergy.
Furthermore, the IoT “group” is also very interested in this.
As far as we know, the expansion of certain large-scale workloads (such as machine learning and certain network functions) is attracting more attention. We realize that we may achieve some breakthroughs in performance, which will be a good opportunity to transplant FPGAs from data center applications to more suitable and broadly developed fields.
However, within the data center group, FPGAs are merely companions to CPUs, helping to solve problems for cloud service providers and other types of large-scale applications.
Intel believes that key applications with priority and high demand for FPGA acceleration include machine learning, search engine indexing, encryption, and data compression. As Waxman pointed out, these are often very targeted and do not have a unified use case. This is the basis for Krzanich’s assertion that one-third of cloud service providers will use FPGA acceleration within five years.
Overcoming FPGA Barriers
While everyone complains about how difficult it is to program FPGAs, Intel does not back down. Although not revealing too many related plans, Waxman proposed some methods to make FPGAs easier to use and understand.
Waxman said: “What we have is unique, something others cannot provide. That is our ability to understand these workloads and drive acceleration.
“We see a shortcut to promote machine learning, accelerate storage encryption, and accelerate network functions,” Waxman emphasized. This is based on our deep understanding of these workloads, which is why we see such opportunities.
But now FPGAs still face some difficulties because people are currently writing RTL. We are a company that writes RTL, so we can solve this problem. First, we make it work, then we can lower the entry barrier. The third step is real economies of scale, all relying on integration and manufacturing strength.
To address these barriers, we offer a range of methods.
X86+FPGA?
Regarding speculation that Intel intends to replace Xeons with FPGAs, Waxman stated that this is nonsense.
Waxman stated that for algorithms with strong demands for high speed and repeatability, FPGAs with inherent advantages are the best choice. For data operations and transformations with extremely high latency demands, FPGAs are also candidates.
Considering that Altera has integrated ARM processors and FPGAs on a SoC, it is natural to think that Intel would try to fully replace the ARM core with an X86 core to create similar devices. But it seems that this will not happen.
First, at the Intel financial statement meeting in the second quarter of 2016, Krzanich promised that Intel would strengthen support for customers currently using Altera’s ARM-FPGA chips.
Waxman further clarified: “Our view is that we will integrate FPGAs into Xeons in some form. We have publicly announced that we will create the first generation of devices using this single package, but we will adjust direction based on progress, and it may even be realized on the same die. We will understand from customer feedback what the right combination is.
By the way, I still expect to see non-integrated systems that maintain their system-level synergy. We will not integrate Xeon with FPGAs in various ways; instead, we will find the right targets and balance in the market.”
Programming Issues Take Center Stage
Although Altera’s toolset utilizes the OpenCL programming model to obtain application code and convert it into RTL (the native language of FPGAs), interestingly, Intel does not believe that the future success of FPGAs in data centers is based on improvements in the integration of OpenCL with RTL tools or the broader adoption of OpenCL.
Waxman also emphasized: “This is not based on OpenCL.” While we do see OpenCL as a pathway to further expand FPGA applications, the initial cloud deployment of FPGAs may be completed by more capable companies, but they do not require us to provide OpenCL, Waxman added.
Waxman hinted that Intel has plans to make FPGAs easier to program, although he could not speak “freely” about it. He stated that Intel will provide programmers with RTL libraries to facilitate their invocation of routines deployed on FPGAs and promote the formation of gates for application routines on FPGAs, rather than letting them create routines themselves. This makes sense, similar to the approach Convey (now a division of Micron Technology) used years ago with FPGA-accelerated systems.
Waxman said: “I think there is a continuous acceleration. At first, you may not know what you are trying to accelerate; you are just doing some trials, so at this stage of acceleration, what you want is a more general purpose. When you really want to accelerate, you will want more efficient, lower power, and less space, and that is when you will focus on FPGAs.”
Waxman also cited Microsoft’s use of FPGAs in its “Catapult” system as an example.
The system uses its Open Cloud Server and adds FPGA mezzanine cards as accelerators. We studied this project in March, applying these accelerators to execute the same image recognition training algorithm on Google, and the results showed that the 25-watt FPGA device outperformed the server using Nvidia Tesla K20 GPU accelerators (235 watts) in terms of performance per watt.
As we said, we have no doubt about the performance data released by Microsoft and Google. However, measuring the application performance of discrete GPUs or FPGAs and their thermal profiles is unfair. You have to see this at the server node level.
If you realize this, the Microsoft server with FPGA assistance is only slightly ahead of the Google server using Tesla K20s at the system level. (These are just our estimates based on image processing performance per watt.) In this comparison, Microsoft should not consider costs. And frankly, unlike Tesla GPUs equipped with everything, Microsoft’s open cloud server did not use Juice or Cooling. Real evaluations would use GPU mezzanine cards while also considering heat, performance, and price factors.
But the focus of Waxman’s discussion remains that. “At some point, you really want that solution that can surprise you and achieve lower power consumption. And that is what our FPGA solution excels at.”
Cloud Business
Finally, we must consider Intel’s cloud business. These customers currently account for 25% of their data center group’s revenue.
Overall, their purchasing volume grows by about 25% each year. It is expected that from 2016 onwards, the overall data center group business will grow by 15% in the coming years. Let’s do some calculations.
If Intel’s plans are implemented as scheduled, its data center group’s revenue will reach $16.6 billion in 2016. Cloud service providers (including cloud builders and large-scale computing users using our language on The Next Platform) account for about $4.1 billion, while the rest belongs to Intel’s data center, with sales data of about $12.5 billion. Therefore, Intel’s data center business growth is around 12% (excluding cloud), which is half of the cloud rate. Intel needs to meet the growth of the cloud and the apparent demand for FPGAs in any way, even if it only occupies a little of Xeon capacity. For Intel, this choice is better than allowing GPU acceleration to continue to grow.
Programming may be a major reason hindering the widespread adoption of FPGAs (unlike other accelerators, which have rich development ecosystems, such as Nvidia GPU’s CUDA). This drives programmers to extend designs based on C language or use OpenCL instead of the low-level models that have plagued FPGA development in the past. But even with so many milestones in the application process, FPGAs are still not favored by the mainstream. We will explore methods and opportunities to solve programming issues.
While we have communicated with many suppliers in this relatively small ecosystem (including Altera and Xilinx, the two major suppliers), according to long-term FPGA researcher Russell Tessier, the day when FPGAs can spread their wings in a broader market is still ahead, and new developments mean broader adoption.
He has studied FPGAs for over twenty years at the University of Massachusetts (he also worked at Altera and is the founder of the virtual machine engineering company acquired by Mentor Graphics) and believes that the situation of FPGAs transitioning from scientific projects to enterprise applications is officially changing. He believes the key lies in improvements in design tools, as designers continuously enhance their high-level designs. In addition, tool vendors can better guide chip development. He added that the large amount of logic within devices means users can achieve more functionality, making FPGAs more attractive across more fields.
Tessier said: “An obvious trend for FPGAs in recent years is that these devices are becoming easier to program.”
Xilinx currently encourages the use of its Vivado product for design in C language. Altera also has an already developed OpenCL environment. The key is that both companies are trying to create an environment that allows users to use more familiar programming (such as C and OpenCL) without having to rely on RTL design experts skilled in Verilog or VHDL. Although good progress has been made in recent years, this is still in the advancing stage, but it will help move more things into the mainstream.
One factor that truly benefits FPGAs is that if used in conjunction with chips, establishing a fast internal interconnect can solve limitations in memory and data movement. This advantage is the main incentive for Intel to acquire Altera. Additionally, if large companies like Intel and IBM can actively promote the construction of the FPGA software ecosystem, its application market will expand rapidly. The mainstreaming of FPGAs (at least not as important as GPUs now) may occur more quickly.
Tessier explained: “The increase in standard core processor integration is certainly key. The past barriers were languages and tools, and as these barriers diminish, new collaboration opportunities for chip suppliers open up. With these and other “mainstreaming” trends emerging, the application areas of FPGAs will continue to grow. For example, financial service firms were the first users to employ FPGAs for financial trend and stock selection analysis, but use cases are expanding. Now there are stronger devices that can solve larger problems.
Broader Application Areas
In addition, FPGAs are discovering new uses through other new fields, including DNA sequencing, security, encryption, and some critical machine learning tasks.
Of course, we hope FPGAs become powerful and “enter” the world’s largest cloud and large-scale data centers, which Xilinx data center division vice president Hamant Dhulla strongly agrees with. At the beginning of 2016, he told The Next Platform, “Heterogeneous computing is no longer a trend but a reality,” which was around the time Microsoft launched the Catapult case using FPGAs (now there are many or will be many in the future), and Intel acquired Altera and saw more widespread applications of FPGAs in data centers.
From machine learning, high-performance computing, data analysis, and other fields, FPGAs are emerging in more diverse application areas. These are all related to the increasing availability of on-chip memory embedded in FPGAs, which FPGA manufacturers and potential end-users are looking forward to.
Dhulla stated that the market potential is large enough for Xilinx to adjust its business model. In recent years, storage and networking have dominated the FPGA user base. However, in the next five years, demand on the computing side will far exceed that of storage and networking, and all will continue to develop along a stable growth line.
In other hot areas for FPGAs (including machine learning), they are more like a “collaborative” accelerator with GPUs. Undoubtedly, for many machine learning workloads, the training part is primarily handled by GPUs. Therefore, a lot of computing power is needed here, just like in HPC, where the power envelope tradeoff is worth it. However, these customers purchase dozens or hundreds of GPUs rather than tens of thousands, and a large number of accelerators are used in the inference part of the machine learning pipeline, which is where the market lies.
As we pointed out, Nvidia is using two separate GPUs (using M4 for training and a lower-power M4 inserted to cut server costs) to offset this, but Dhulla believes FPGAs can still lower power consumption by adopting PCIe methods and can also be embedded in large-scale data centers.
Their SDAccel programming environment makes it more practical by providing high-level interfaces for C, C++, and OpenCL, but the real way to promote large-scale and HPC adoption is through end-user examples.
When it comes to these early users, it sets the stage for the next generation of FPGA applications, Dhulla points to companies like Edico Genome. Xilinx is currently also collaborating with customers in other fields, including oil and gas and historical computing in finance. Early customers are applying Xilinx’s FPGAs in machine learning, image recognition and analysis, and security, which can be seen as the first step in their computing acceleration business development.
Despite poor double-precision performance and overall pricing, the real large-scale application opportunity for FPGAs lies in the cloud. Because FPGAs can provide advantages that GPUs cannot. If FPGA vendors can convince their end-users that their accelerators can provide significant performance improvements (in some cases they do), for critical workloads. Providing a programming environment that advances OpenCL development through complexity-wise with other accelerators (such as CUDA) by offering FPGAs in the cloud to solve pricing issues. This could be a new hope.
Of course, this hope comes from deploying FPGAs within ultra-dense server cloud architectures rather than on single-machine sales. This model has already occurred in the financial services of FPGAs.
Just as their GPU accelerator “partners” are pulling around deep learning to quickly gain more users, FPGA devices are exploring a real opportunity to invade the market by solving neural network and deep learning problems.
New application hosts mean new markets, and as cloud applications promote the elimination of some management overhead, it may mean broader adoption. FPGA vendors are striving to push their applications in some key areas of machine learning, neural networks, and search. FPGAs are becoming increasingly common in ultra-large-scale contexts in fields such as natural language processing, medical imaging, and deep data detection.
In the past year, various applications of FPGAs have been exposed, especially in deep learning and neural networks, as well as in image recognition and natural language processing. For example, Microsoft uses FPGAs to provide twice the search service on 1,632 nodes and employs innovative high-throughput networks to support Altera FPGA-driven workloads. China’s search engine giant Baidu (also a user of many deep learning and neural network tasks with GPUs) is executing storage control with FPGAs, with daily data throughput ranging from 100TB to 1PB.
Large-scale data centers and other applications using FPGAs are attracting more attention to the single-precision floating-point performance of FPGAs.
While some cases use (including the Baidu example), GPUs as computing accelerators and FPGAs on the storage side, researchers from Altera, Xilinx, Nallatech, and IBM in the OpenPower alliance have demonstrated the bright prospects of FPGAs in cloud deep learning.
It can be said that we are now in a golden age for FPGAs.
Reposted from Semiconductor Industry Observation
1.35 years old, the eternal topic of engineers!
2.Never offend programmers; their revenge methods are very ruthless, 11 lines of code will make you doubt your life!
3.Software engineers vs. hardware engineers, which one will you serve in the future?
4.Overview of Embedded IoT Protocols
5.What are the career development directions in the embedded field?
6.Resistance: A timeless grievance, I am not called resistance, I am called conductance!