In April of this year, the consulting firm VDC Research wrote in “Embedded Processor Architectures” that the achievements of RISC-V have expanded beyond embedded computing into storage technology and HPC, demonstrating a trend of broader application for this architecture. This statement was also quoted by Andrea Gallo, CEO of RISC-V International, at this year’s RISC-V European Summit and China Summit.
Recently, Electronic Design has discussed the current development of RISC-V in the fields of HPC AI, automotive, and embedded applications. The reason RISC-V has achieved its current results is closely related to the trend of application-oriented chip design—its open, customizable, and scalable characteristics allow it to better adapt to applications; at the same time, openness also means community collaboration, leveraging the strengths of many to improve the ecosystem, enabling it to challenge existing market leaders.
Andrea also mentioned in his speech that the era has shifted from traditional SoC design to “workload-designed” systems, focusing on workloads to design the best architecture, which RISC-V can achieve. In this interview with Electronic Engineering Magazine, we further explored the topic of AI HPC—let’s see how RISC-V is preparing in the high-value, high-difficulty market.
Andrea Gallo, CEO of RISC-V International
What is the progress of AI HPC applications?
Krste Asanović, the father of RISC-V (Chief Architect of RISC-V International, Chief Architect of SiFive, and Emeritus Professor at the University of California, Berkeley), stated at the summit: “RISC-V is an ISA that can do everything; it will be a more widely applicable instruction set than any other ISA.”
Although many companies and experts at the RISC-V China Summit believe that RISC-V is particularly suitable for AI applications, it must be acknowledged that RISC-V’s current applications in AI data centers and other HPC-related areas are relatively limited. While we do not have market share data for different instruction set processors in the HPC market, Omdia presented a set of data at the summit showing the market shares of different instruction sets in various fields within the MCU microcontroller market in China for 2024, as shown in the following image.
Xu Song, Chief Analyst at Informa Data Services, commented that RISC-V has shown considerable development in the consumer electronics field, especially in audio and wearables, “and the industrial control market is also continuously making positive layouts,” but in some high-threshold, high-value fields such as automotive and high-performance computing, the market share remains low. This set of data may not be very valuable for reference, but it still provides a rough picture of RISC-V’s market position in the current microcontroller arena.
However, Xu also clearly mentioned that IP suppliers are making relevant layouts in areas such as automotive-grade, high-performance computing, high-speed connectivity, and AI accelerators. “RISC-V can adapt to diverse lightweight edge applications; it can also develop upwards to accommodate high-computation-intensive scenarios, providing excellent energy efficiency.”
Therefore, Andrea cited data from SHD Group in his keynote speech: by 2031, the shipment of RISC-V SoC chips is expected to reach 20 billion units, achieving over 25% penetration in the SoC market. Meanwhile, RISC-V’s market share in consumer, computing, automotive, data center, industrial, and networking sectors is expected to reach between 26% and 39%.
Among these, we are particularly interested in AI HPC. First, it is important to clarify that the term “HPC (High-Performance Computing)” here is broadly defined; just as TSMC categorizes PC processors under HPC in its financial reports: under this definition, different types of data centers (including supercomputing centers, enterprise computing data centers, superclusters, cloud computing data centers, AI factories, and edge data centers) can all be considered HPC.
Typically, narrow definitions of HPC may only include 1-2 categories (especially supercomputers, which solve complex problems such as climate science, fluid dynamics, nuclear fusion, etc.). When focusing on AI, AI HPC as a subclass encompasses various intelligent computing data centers, as well as the “AI factories” that NVIDIA and many GPU companies are currently keen to discuss.
Andrea introduced us to the current status of RISC-V in data centers: “There has been quite a bit of news in this area, including Rivos and Canonical collaborating to provide scalable RISC-V solutions in the data center field; XuanTie has also released its latest server-grade chip (C930); the RISC-V Koji instance in the Fedora data center has also gone live; Jintide has developed a server CPU chip V100 for the next generation of AI applications; and there is also the Ventana Veyron V2 data center processor…”
“Recently, within the RISC-V ecosystem, we have ratified the server SoC specifications and are collaborating with server platforms. The value lies in that we have defined common hardware peripherals around the RISC-V core; we are also defining and standardizing the software-hardware interfaces; this way, operating system vendors will find it easier to port to the RISC-V platform.”
Regarding the narrow definition of HPC, Andrea mentioned the EU DARE SGA1 project—led by the Barcelona Supercomputing Center, which received €240 million in funding for a RISC-V-based supercomputing stack targeting HPC and AI, and also mentioned that “the Xiangshan project at Beijing Open Chip Institute is also developing an HPC processor based on RISC-V.” “RISC-V’s development in this field is faster than in other vertical markets, with many research laboratories dedicated to HPC work.”
Specifically in AI application scenarios, “NVIDIA is expected to have shipped over 1 billion RISC-V processors last year; Ahead Computing raised $215 million in its first round of financing; Axelera AI received over $61 million in grants to develop AI chiplets and platforms for high-performance computing; and Andes Technology is collaborating with Meta to develop AI accelerators…” “Many companies are developing innovative architectures based on RISC-V for AI.”
At the main forum of the RISC-V China Summit alone, we saw companies and institutions such as Tenstorrent, Alibaba DAMO Academy, Beijing Open Source Chip Research Institute, Zhihe Computing, and ZTE Microelectronics discussing their technological achievements and product plans based on the RISC-V instruction set in the fields of HPC and AI applications. The head of Alibaba DAMO Academy’s XuanTie confidently stated in his speech, “We are going to develop in the direction of HPC,” and “I believe that based on this trend, RISC-V will dominate in various fields,” including “high performance and high computing power areas.”
What enables RISC-V to develop in the HPC AI field?
To understand the “why”: if the “what” refers to “RISC-V is rapidly developing in the AI HPC field, potentially sharing the market with x86 and Arm in the future”; then the key to the “why” is “what exactly does RISC-V rely on?”
Borrowing a question posed by Bao Yungang (Deputy Director of the Institute of Computing Technology, Chinese Academy of Sciences, Secretary-General of the RISC-V Alliance, and Chief Scientist at the Beijing Open Source Chip Research Institute) at the summit’s main forum: “If Arm is doing well, why switch to RISC-V?” The answer to this question is certainly not solely due to RISC-V’s open-source nature achieving technological independence—although Andrea also specifically mentioned that RISC-V allows companies to avoid being tied to a single supplier, “mastering one’s own destiny is important.” If RISC-V does not have real technological or commercial advantages, it would only be praised in the electronics industry without substantial market presence.
The first factor is undoubtedly the flexibility of the RISC-V instruction set.“Flexibility allows companies to develop innovative solutions tailored to specific workloads.” “For AI, the appeal of RISC-V from a custom development perspective lies in the fact that chip designers can adjust specific extensions as needed: keeping what is necessary and discarding what is not.” Andrea mentioned, “This way, we can minimize die size and power consumption while optimizing performance for specific workloads and operations that need to run on RISC-V processors.”
Recent articles have detailed the cost and energy efficiency benefits arising from the flexibility of the RISC-V instruction set. This characteristic perfectly aligns with the current trend of “application-oriented chip design.”
Thus, we see that NVIDIA’s current GPU chips also choose to use RISC-V cores as controllers—at last year’s RISC-V North America Summit, NVIDIA introduced the RISC-V cores used for resource computation and power management within GPUs: depending on the complexity of different chips, NVIDIA custom-developed between 10 to 40 RISC-V cores for GPUs; moreover, NVIDIA even designed at least three types of RISC-V microcontroller cores specifically for its GPUs, and developed over 20 custom extensions to achieve additional performance, functionality, and security…
Jim Keller (CEO of Tenstorrent) and Lian Weihang (Chief Architect of Tenstorrent) have both mentioned on multiple occasions that the choice of RISC-V is due to its openness and flexibility. Lian Weihang stated at the 2022 RISC-V Summit that when he first joined the company, he evaluated the companion CPU for the ML processor. He approached Arm to inquire whether they could support a specific data type, and Arm’s response was negative. It is said that Arm required two years of internal discussions and negotiations with partners for such support. Thus, RISC-V quickly became the new choice.
The second factor lies in the business model.“Regarding the licensing model of the RISC-V instruction set: all our specifications are publicly available. Anyone, whether in Silicon Valley, Europe, India, or China, can use the RISC-V specifications and standards for free,” Andrea said in the interview, “and then they can develop locally based on RISC-V standards.” “Startups can also more easily create value and quickly build innovative products based on RISC-V.”
On one hand, this brings about a reduction in chip design costs: Bao Yungang mentioned in his keynote speech that RISC-V’s openness, using open-source implementations and open-source toolchains, can “structurally reduce costs.” He cited an example of developing a 64-core server chip with a production target of 100,000 units, estimating the development cost to be around 750 million RMB, of which “IP licensing fees and royalties account for about 250 million RMB, or 33%.”
If the “joint development model based on open source can reduce costs by about 30%”, “companies can fully allocate these costs elsewhere. RISC-V can bring us new opportunities.”
On the other hand, such a business model also implies “collaboration.” “Because it is a global standard, we can collaborate with all companies within the RISC-V ecosystem,” Andrea mentioned, “investments made in RISC-V ecosystems in other vertical fields can also be utilized for AI; everyone can benefit from the resources of numerous IP suppliers”… coupled with the collaborative construction of resources such as operating systems, foundational software, toolchains, and compilers, there are greater opportunities to challenge existing market leaders.
Tenstorrent’s mention of “democratizing AI” during their presentation this year, along with their comprehensive open-source approach to hardware and software components, plus the OCA (Open Chiplet Architecture) standard, emphasizes the high costs faced by diverse AI application demands in chip and system development, necessitating the need for “collaboration” and “co-construction” within the entire RISC-V ecosystem to reduce AI usage costs and promote “AI democratization.” The foundation of this logic is the value generated by RISC-V ecosystem collaboration, which is difficult for ecosystems like x86 and Arm to achieve.
At the same time, we believe it is equally important to have unified RISC-V standards. Under the premise of having both openness and unified standards, “only then will companies investing in RISC-V have the confidence to build the future and roadmap.” This is also why companies in the HPC market believe that “collaboration” and “consensus” will give participants in the RISC-V ecosystem the opportunity to challenge the CUDA ecosystem.
What is the progress of standards and ecosystem construction?
Unification and standardization are among the important tasks of RISC-V International, as this involves the direction of the entire ecosystem and is the foundation for discussing “collaboration” and “innovation.” Krste specifically reported on the progress of RISC-V Profile in his keynote speech, in addition to the recently approved “RVA23 as the most important profile of RISC-V,” which is a very complete and competitive feature set, “different RVA23 hardware solutions will emerge by the end of this year and next year.”
Andrea also mentioned in the interview that RVA23 specifies a series of mandatory and optional new features and extensions—this type of profile advancement can greatly ensure the software compatibility of different RISC-V implementations, serving as a “baseline feature set” for processors, which is crucial for reducing the cost of porting software. He stated that RVA23 has undergone a long development, review, and certification process, as it needs to be applicable to a wide range of applications and system software ecosystems.
RVA23 includes many approved ISA extensions, such as vector extensions (RVV) for parallel data processing (especially AI/ML), hypervisor extensions for virtualization, certain floating-point instructions for numerical computation, compression instructions for code density, and cache and memory management extensions for system-level performance…
The next small update planned is RVA23p1, which will only add optional items—and in the future, there may be 1-2 small version updates released each year. Krste added that the optional items for small version updates are all to prepare for the next major update, RVA30, which is expected to be released in 2-3 years (not expected for another 2-3 years).
In addition to Krste mentioning DSP and >32b long instruction extension support, it is particularly worth mentioning the matrix extension that many people referred to at the summit, which Andrea also mentioned several times in the interview: RISC-V is working on matrix extensions, “vector extensions, support for BFloat 16 data format, and matrix extensions can allow AI inference to run directly on RISC-V processors without the need for external NPUs, improving efficiency.” This itself is an extension that reflects the advantages of RISC-V, “executing inference within the CPU avoids issues like memory copying and accelerator latency; it also reduces power consumption and chip size.”
When discussing HPC, he added: “The recently released RISC-V Vector C Intrinsics can also help developers optimize HPC libraries.” The RISC-V Vector C Intrinsics API can be used for low-level C language access to RVV instructions, allowing for finer control of vector operations, thus benefiting the performance and efficiency of HPC libraries. It also includes support for hypervisors for HPC, and even considerations for supercomputing algorithms aimed at critical simulations, such as weather and earthquake predictions.”
“Additionally, performance and power consumption are not the only considerations; we also need to ensure the security required by national laboratories and supercomputing applications. Therefore, we also collaborate with members to listen to the needs of different companies. Last year, we approved extensions aimed at security and resilience: pointer masking for overall control flow integrity, landing pads, shadow stacks, supervisor domain isolation, physical memory protection (PMP), and IO protection.” “When it comes to HPC and data centers, there are also extensions like RERI (RAS Error-record Register Interface), which is an important industry standard.”
“Due to the characteristics of RISC-V, we can quickly add value and bring innovation. Our role is to ensure collaboration among all members, to evolve standards, develop RISC-V extensions, and align globally.”
In fact, RISC-V International has specifically established an AI/ML SIG (Special Interest Group), “We take a bottom-up approach to analyze AI/ML processing needs, understand the acceleration of new op codes; and through a top-down approach, collaborate with platforms to discover the best ways to integrate RISC-V into all ML frameworks.”
Not only in AI and HPC directions, but also when facing different vertical markets, RISC-V International will establish corresponding SIGs, “Further investment in SIGs will allow us to collaborate with software developers, device manufacturers, and product manufacturers in each vertical field.” After all, different vertical fields have their own platform ecosystems and standards, “We need to understand the correct standards between software and hardware in these fields,” and based on this, build common standards to make RISC-V the instruction set that Krste describes as “more widely applicable than any ISA.”
In addition to the instruction set itself, when chips are truly aimed at application developers, Andrea also mentioned that in 2024, RISC-V will send out nearly 300 boards to key developers in major open-source projects; “We are also working with academic institutions and laboratories to build development boards that can be accessed remotely for developers.”
Moreover, “We have a large number of online training courses related to RISC-V, which are continuously increasing and are all free, ranging from designing FPGAs based on RISC-V to supporting real-time operating systems and Linux, as well as developing drivers,” and “My personal favorite is ‘How to Build a RISC-V CPU Core’… If you are new to RISC-V, following these tutorials, you will feel the beauty of RISC-V and the simplicity of this architecture.”
In terms of other collaborations, “We are collaborating with many operating system vendors, including Yocto, Red Hat, Canonical, etc.” “Operating system vendors have achieved 95%-100% compatibility.” In terms of compilers, “Our collaboration with the GCC and LLVM communities is also quite close.” For instance, when approving RVA23, patches for GCC/LLVM support were already in place. “Adding the necessary support for reference models and toolchains is an important condition for extension approval.”
Some of the development toolchain work will be done directly by RISC-V members, while others will be handled by the RISE (RISC-V Software Ecosystem) project. For example, at the RISC-V European Summit, RISE Vice President Barna Ibrahim discussed the collaboration with RISC-V International, stating that in addition to compilers and runtime, they have recently formed an AI/ML working group to port PyTorch and Llama.cpp to the RISC-V architecture.
In this way, we have a more comprehensive understanding of the construction of the RISC-V ecosystem, the definition and establishment of standards, and the work of RISC-V—of course, our interview focused only on the AI HPC direction. In fact, Andrea’s keynote speech also specifically mentioned RISC-V’s development in various vertical fields such as automotive and aerospace, but the overall model and thinking should be roughly similar.
More importantly, in the AI era, we can also clarify why many speakers and experts at the RISC-V China Summit’s forums described “AI and RISC-V as mutually complementary and promoting each other,” even asserting that RISC-V is the most suitable CPU instruction set for AI chips and applications—possessing flexibility, unified standards, and collaborative participation from ecosystem players is the most direct reason.
“Finally, I want to add that RISC-V can achieve rapid innovation,” Andrea concluded at the end of the interview, “Companies can develop their custom extensions and innovate quickly in the market; and they can also establish common standards with the RISC-V ecosystem. Therefore, such speed of innovation is something only RISC-V can achieve.”
