On July16, the fifthRISC-V China Summit was successfully held at the Zhangjiang Science Hall in Shanghai. Dr. Yang Yi, co-founder and COO of Yixing Intelligent, delivered a keynote speech titled “Creating an Innovative Computing Architecture by Combining RISC-V and Virtual Instruction Technology” at the summit.
Quoting the concept of “Software 3.0” from OpenAI founding member Andrej Karpathy, Dr. Yang Yi mentioned that with the rise of large language models, the paradigm of software development is undergoing a fundamental transformation —natural language prompts are replacing traditional programming code, and LLM is becoming the new programming interface. This marks a fundamental shift in how software is built, interacted with, and conceived. The fundamental transformation of the software paradigm also relies on the support of matching “Hardware 3.0″. In Software 3.0, the majority of computation is multi-dimensional matrix operations, and the network model is composed of multiple layers of “graphs. The computation of graphs has the characteristics of data flow, requiring hardware design that matches the characteristics of data flow. Thus, we have seen the explosion of domain-specific architectures (DSA), represented by Google’s TPU, which is born for the efficient training and inference of LLM. At the same time, we can also find shadows of DSA in GPGPU, such as the inclusion of designs like Tensor cores and TMA.
Regarding Hardware 3.0, Dr. Yang Yi pointed out that AI computation is a domain-specific paradigm, and we will emphasize the utilization of computing power to achieve maximum hardware efficiency. However, we must also face the diversity of models and the massive programming user demands, so balancing generality and specificity within the domain is a core requirement. In simpler terms, the utilization of computing power not only refers to the proportion of computing units that can be loaded during program execution but also indicates how many users can conveniently use it. From this perspective, computational efficiency and generality are not contradictory but are integrated into a “user demand”.
Dr. Yang Yi explained Yixing Intelligent’s consideration of using RISC-V + RVV to build the AI computing architecture. The openness, modular design, and customizable dedicated AI acceleration instruction set of RISC-V can greatly assist in building DSA; its simplified architecture lowers the threshold for chip design, helping companies iterate quickly.
“Yixing Intelligent uses RISC-V as the foundation for architectural innovation. Its open instruction set allows for the customization of AI dedicated extensions, and the RVV variable-length vector instructions can directly adapt to tensor computations, with vector masking mechanisms directly supporting sparse matrix operations. More importantly, the increasingly mature ecological advantages: LLVM, GCC and other mainstream compilers have supported RISC-V, and mainstream AI frameworks are actively adapting to the RISC-V platform. This provides strong support for the rapid implementation of our products.”
Regarding the instruction path selection of DSA, Dr. Yang Yi combined Yixing Intelligent’s EVAMIND™ AI kernel to deeply explore the innovative combination of RISC-V instruction extensions and virtual instruction technology (VISA). VISA is an original intermediate layer created by Yixing Intelligent, situated between the AI compiler and the backend compiler. It serves as both a software intermediate layer encapsulation and a set of macro instructions that are truly issued in out-of-order fashion in the hardware of the EVAMIND™ kernel, abstracting a unified virtual ISA above the hardware ISA. The role of VISA is to solve three key problems encountered in the evolution of AI computing architecture:1) Deep coupling between hardware, operators, and software stacks such as compilers, restricting each other’s evolution.2)) Difficulty in balancing generality and specificity in the design process of AI processors.3)) The steep decline of intermediate representations (IR) in AI compilation, from Tensor to fine-grained SIMD, leading to the loss of optimization opportunities. In the architecture of EVAMIND™, fine-grained hardware instructions are used to implement coarse-grained VISA operators; performance optimization is performed within each VISA operator using software pipelining, loop unrolling, and other methods. For each VISA operator, excellent performance is achieved, while the upper part of the AI compilation only needs to focus on the level of VISA operators, reducing the difficulty of instruction issuance.
By using the fusion of VISA and RISC-V microinstructions, the architecture design of EVAS has the following characteristics: within the DSA, different heterogeneous computing and data operation units are scheduled out of order with virtual instructions (VISA) that have tensor semantics, maintaining dedicated properties in the field of AI; at the same time, within the computing units, the combination of RVV decomposes and executes VISA macro instructions into fine-grained and Turing-complete microinstructions. Here, the virtual instructions are macro instructions that are truly issued by the scalar core hardware, but internally they are efficiently optimized soft cores. This soft-hardware combination enhances the efficiency of dedicated AI computation while improving the programming experience for users.
Dr. Yang Yi introduced that in the EVAMIND™ kernel, the scalar engine is responsible for the coordination and control of the entire system; the VISA scheduler ensures the arrangement and out-of-order issuance of coarse-grained macro instructions; the tensor engine is specifically responsible for matrix operations and tensor computations; the 4D acceleration engine is responsible for data movement inside and outside the core, as well as activation, reduce and various transformations and transpositions of 4D matrices; in addition, the RISC-V RVV vector engine provides dedicated hardware extensions for AI, ensuring the efficient execution of fine-grained microinstructions.
“Based on this kernel, the company is about to launch a new generation of intelligent computing products. We use RISC-V equipped with virtual instruction set technology, balancing generality and specificity, integrating multiple sets of EVAMIND™ kernels, making it equipped with a Turing-complete vector engine + large tensor units. This design balances dedicated and general computing tasks, enhancing efficiency through hardware-level instruction.”
“This product supports various floating-point and fixed-point data types such as INT4, INT8, FP8, FP16, BF16, and also supports mixed precision computing required by large models. Especially in terms of native support for FP8/INT4, this solution can achieve a 2 to 4 times improvement in computational throughput. In addition, it supports various parallel and pipelined computing methods, achieving extreme utilization of computing resources.”
“As a company dedicated to providing cutting-edge AI computing architectures and high-performance parallel computing solutions, Yixing Intelligent hopes to provide a new generation of general and dedicated computing acceleration solutions through the RISC-V open instruction set ecosystem, driving the progress and sustainable development of human society.” Dr. Yang Yi envisioned at the end of his speech.
In the wave of rapid development of AI technology, computing architecture is entering a critical stage of comprehensive upgrade. Yixing Intelligent provides a new approach to solving the core contradictions in the field of AI computation through innovative soft-hardware collaborative design concepts. In the future, Yixing Intelligent will continue to deepen the construction of the RISC-V ecosystem, promote the standardization of virtual instruction technology, and work with more industry partners to build an open and efficient AI computing ecosystem.
In the era of Software 3.0, only through continuous technological innovation and architectural breakthroughs can we truly inject strong momentum into the development of the digital economy. As Dr. Yang Yi envisioned at the summit:“We are standing at a historical juncture of the transformation of AI computing architecture, and the innovative combination of RISC-V and virtual instruction technology will redefine the development direction of the next generation of intelligent computing.“