Accelerating DeepSeek on RISC-V Platform with XTheadVector: Unlimited Potential of RISC-V + AI

This Friday, during an offline exchange event organized by the Software Research Institute of the Chinese Academy of Sciences [1], partners from the PLCT laboratory testing team demonstrated how to run the DeepSeek R1 Distill model on the Milk-V Pioneer Box equipped with the SG2042 processor using the toolchain integrated with RuyiSDK, successfully accelerating it with the XTheadVector extension. This indicates that all chip modules/development boards that support the XTheadVector (including RVV0.7.1) extension, such as the TH1520/Milk-V Meles, can better utilize hardware computing power when running large models like DeepSeek.We would like to express our gratitude to the OpenBLAS open-source community. The BLAS software library is essential for AI, scientific computing, and more, and currently, only the OpenBLAS open-source project provides support for RVV0.7.1.The corresponding code for the PPT can be found at:https://github.com/wychlw/plct/blob/main/memo/deepseek_on_llama.cpp.mdThe demonstration address is:https://asciinema.org/a/hIX2a1zshOUyYMymv1Fadv7xvThe demonstration address mentioned above:https://asciinema.org/a/eRkibHqHPMO5v09G8pUKpNkDBWelcome to join the PLCT laboratory testing team internship![1] The 7th RISC-V Hardware and Software Frontier and Open Source Infrastructure Seminar (Workshop for RISC-V Research and Infra) is an offline RISC-V seminar organized by the Software Institute, held bi-weekly in the 4th-floor conference room of the Software Institute. Currently, only offline participation is available.