(This article is translated from Semiconductor Engineering)
There is an increasing discussion in the industry about the many benefits of the RISC-V architecture, but is it really the right starting point? While it is not without flaws, it may offer the flexibility needed for gradual progress.
The development of computer architecture and software has followed the trajectory of processors developed 80 years ago. These processors were designed to leverage foundational technologies to solve sequential scalar arithmetic problems, and as long as there is enough memory, this technology can solve any finite problem.
The chip industry has shown a reluctance to abandon that approach, especially in sectors still running software developed 50 years ago. Software paradigms and backward compatibility have a significant influence. The migration of programming from single-processor architectures to multi-processor architectures took decades, until NVIDIA developed CUDA, which allowed applications for massively parallel processors to begin development outside of highly specialized applications.
So, when the typical workloads of many applications are data flow problems with a small amount of control, why does the industry still need another CPU architecture? The answer lies in the fact that the RISC-V architecture can evolve in various ways, some of which may support a slow migration rather than attempting to leap across a vast chasm.
At the recent RISC-V community annual summit, the increasingly widespread applications of RISC-V and its growing influence were showcased, especially in areas where there are not many legacy software barriers. Andrea Gallo, CEO of RISC-V International, stated, “RISC-V is no longer just a microcontroller that you didn’t even know about before; it has integrated into your products. RISC-V has entered a whole new phase.”
Gallo also cited several significant advancements made by RISC-V: “Infineon stated that they are applying RISC-V in automotive applications. The EU is funding HPC projects and automotive projects. Meta is using it in their accelerator cards for AI. NVIDIA estimates that by 2024, the shipment of RISC-V cores in their GPUs will reach one billion.”
RISC-V has also made significant strides in the field of cutting-edge processors. “It is not only applied to low-end processors; in some areas, it can also serve as a coprocessor or accelerator for GPUs,” said Mohit Wani, Chief Product Manager at Synopsys. “NVIDIA demonstrated that nearly 30 different functional modules in their product line are implemented with RISC-V-based cores.”
However, RISC-V still needs to overcome some commercial and technical barriers. Roland Jancke, Head of Design Methods at Fraunhofer IIS Adaptive Systems Engineering, stated, “The automotive industry is increasingly focusing on RISC-V, but previously it was cautious because RISC-V is an open architecture. In the automotive industry, when a problem arises, it is necessary to clarify responsibility, and the open community model makes it difficult to hold a single supplier accountable. Nowadays, due to potential cost reductions, automotive companies are paying more attention to RISC-V, but this requires a complete ecosystem to support it. Not only do we need tools to develop processors, but we also need upper-layer software support. Although RISC-V is gaining market share, it still has a long way to go to become a mainstream processor in the automotive sector.”
However, not everyone believes this is worth paying attention to. Steve Roddy, Chief Marketing Officer at Quadric, stated, “RISC-V is not a solution for AI—neither for training nor for inference. RISC-V is just another control CPU, similar to Arm, x86, MIPS, Xtensa, and ARC processors. The latter two also provide designers with instruction set customization capabilities similar to or even superior to RISC-V. Therefore, the technical characteristics of RISC-V do not provide a significant improvement over previous generations.”
To reconcile these two extreme viewpoints, a long-term understanding of how the industry operates is necessary. Venki Narayanan, Senior Director of System Architecture and Embedded Solutions at Microchip Technology, stated, “RISC-V has the potential to drive the evolution of AI, which is precisely what AI development requires. Whether at the learning level or the inference level, models are constantly evolving, requiring support for various data types, various storage units, local memory, and the ability to perform more customized computations faster. From this perspective, there are many paths to realization, and RISC-V is achieving this through Domain-Specific Architectures.”
Unique Opportunities
In the chip industry, new application scenarios without legacy software adaptation are rare, but artificial intelligence (AI) is one such case. Moreover, given the rapid pace of AI technology development, any software may be disrupted by new breakthroughs before it matures. This creates a perfect environment for the continuous evolution and adaptation of technology.
The openness of RISC-V provides freedom in architecture design. Gallo of RISC-V International stated, “AI accelerator cards can integrate a large number of RISC-V cores, even having different types of cores within the same cluster. For example, in hundreds of small modules, some cores are dedicated to data transmission, while others focus on the inference computation itself. By adding custom instructions, efficient tensor computation units can also be integrated. This is how RISC-V influences chip architecture.”
While the licensing model of RISC-V brings cost advantages, it is not the only reason. “Its value goes far beyond that,” Wani of Synopsys stated, “In terms of how accelerators and processors connect, they typically interact through high-speed interfaces in a memory-mapped mode. For developers, if they need to perform tasks on the accelerator, they only need to send data and task information through the interface and wait for the results to return.”
However, this communication is costly and leads to cores being idle. “30% of the time is wasted on data transmission and result return,” Wani stated, “If specific operations can be executed natively through proprietary vector pipelines and directly connected to the accelerator, these time losses can be avoided. This flexibility exists only within the RISC-V architecture.”
Roddy of Quadric disagrees: “All control CPUs face the same severe limitations in AI applications (especially in inference scenarios). The design intent of CPUs is to handle pointer jumps in random code, not matrix or tensor operations. They provide at most the computational throughput for vector multiplication but are limited by the bandwidth bottlenecks of traditional load/store. Therefore, any claim to use RISC-V for AI must bundle independent matrix engines, which introduces graph partitioning issues, a fatal weakness of CPU-based solutions. The correct innovative direction to solve AI problems is to develop architectures optimized for native matrix/tensor operations, breaking away from CPU-centric memory caching and speculative out-of-order pipeline dependencies.”
RISC-V achieves these functions by integrating custom features into traditional control processors. Gallo stated, “Some members have developed custom tensor instructions, which is the flexibility of RISC-V, allowing for the development of custom instructions for specific workloads and fully bearing the total cost of custom applications. At the same time, standardization and expansion specifications are also valuable, as they can share the maintenance costs of compilers, toolchains, and libraries. We already have vector instructions and are developing matrix instructions, and we will adopt different solutions based on application scenarios: whether it is accelerator cards, AI IoT, or edge AI, the implementation of matrix acceleration will differ.”
With no other candidates available, the industry can only utilize existing resources. Narayanan of Microchip Technology stated, “Data flow is very important in AI and many computing elements, and the growth in computational demand requires more energy-efficient implementations. This involves not only the microarchitecture of instructions, fetching, execution, and writing back but also how to organize microarchitecture and data flow, especially how to efficiently transmit large amounts of data between layers, without always relying on DDR read/write.”
As artificial intelligence rapidly develops, a high degree of flexibility is required. “The types of AI models that need to be efficiently implemented are constantly increasing,” said Nilam Ruparelia, Head of AI and Communications at Microchip Technology. “ChatGPT requires Transformers, which is also why AI is popular, but there are many models on the market with complexities far below that of Transformers, yet they also require higher performance. For example, classic CNNs, RNNs, and LSTMs require mathematical operation unit architectures and DSP module architectures to better support these models and improve computational efficiency.”
AI is not a single problem, and flexibility remains important. “AI has multiple levels,” Narayana stated, “There are segmentation, object detection, classification, and Transformers, each level using different data types. How are these levels implemented? How are they executed efficiently? How is data transmitted between layers? The solutions to these questions depend on architectural design.”
This demand involves not only computational logic but also data processing. “RISC-V has an inherent advantage, allowing for hardware customization without violating the Instruction Set Architecture (ISA) specifications,” Ruparelia of Microchip Technology stated. “Whether at the instruction level or microarchitecture level, it can optimize processing capabilities for specific data types, building custom computing solutions for specific workloads. This flexibility is crucial for quickly responding to new data types, and in the near future, new data types will continue to emerge in different application fields.”
However, the complete requirements for processors optimized for future tasks remain unclear. “If we develop an edge inference processor, it may be used for classification, detection, segmentation, or even some Transformer operations,” Narayanan stated. “These are fixed layers, and we know how to design more optimized computing units to handle such workflows. The processors we build are optimized for current needs, but this means that once new layers, new operation types, or similar changes emerge, the existing architecture can support them, but the efficiency cannot compare with natively adapted designs.”
Ecology
The ecosystem has always been a significant barrier to the adoption of RISC-V in certain application areas, but it is currently making steady progress. “We have upgraded our membership in the Yocto project to platinum level,” Gallo stated, “This sends a very strong signal to the ecosystem. Yocto is the most popular embedded Linux distribution, adding new ISA and new architectures for the first time in many years, and becoming a platinum member means RISC-V will stand on equal footing with other architectures. Yocto is used not only for embedded Linux and edge AI IoT but also for consumer-grade set-top box TVs and automotive infotainment systems.”
The mobile sector is catching up. “Google has listed RISC-V as a top technology in Android development,” Wani stated. “As the software stack matures, we will also see entry-level application cores in popular areas such as mobile devices and laptops.”
When enough leaders join, other companies will follow suit. “Red Hat announced that they have launched a RHEL developer preview for RISC-V,” Gallo stated. “Fedora has also supported RISC-V.”
Significant investments are pouring in. “Another interesting project in Europe is the DARE project,” Lapides stated. “This project will invest 260 to 280 million euros over the next five years. Three major suppliers are building chiplets, not just processor IP. One is a general-purpose CPU, one is a vector accelerator, and the other is an AI accelerator. These chiplets will be integrated. The combination of RISC-V and new chiplet architectures is quite promising.”
Conclusion
RISC-V may not be the perfect solution for many applications, especially AI-related applications, but it may provide a path for evolution. Furthermore, there are currently no better alternatives in the industry.
In the semiconductor industry, few technological innovations achieve success. By establishing an open and continuously evolving community, people can quickly develop definitions of current needs, future demands, and future prospects, potentially achieving the ultimate goal without taking on excessive risks. Through gradual progress, the ecosystem can keep pace and move forward steadily.
END