Running DeepSeek on RISC-V with XTheadVector Extension

This Friday, during an offline communication event organized by the Software Research Institute of the Chinese Academy of Sciences [1], partners from the PLCT laboratory testing team demonstrated how to run the DeepSeek R1 Distill model on the Milk-V Pioneer Box equipped with the SG2042 processor using the toolchain integrated in RuyiSDK, successfully accelerating it with the XTheadVector extension. This means that all chip modules/development boards supporting the XTheadVector (including RVV0.7.1) extension (such as TH1520/Milk-V Meles) can make better use of hardware computing power when running large models like DeepSeek.Special thanks again to the OpenBLAS open-source community. The BLAS software library is an essential foundation for AI, scientific computing, etc., and currently, only the OpenBLAS open-source project provides support for RVV0.7.1.The PPT corresponding code can be referenced at:https://github.com/wychlw/plct/blob/main/memo/deepseek_on_llama.cpp.mdRunning DeepSeek on RISC-V with XTheadVector ExtensionRunning DeepSeek on RISC-V with XTheadVector ExtensionRunning DeepSeek on RISC-V with XTheadVector ExtensionRunning DeepSeek on RISC-V with XTheadVector ExtensionRunning DeepSeek on RISC-V with XTheadVector ExtensionRunning DeepSeek on RISC-V with XTheadVector ExtensionRunning DeepSeek on RISC-V with XTheadVector ExtensionThe demonstration address here:https://asciinema.org/a/hIX2a1zshOUyYMymv1Fadv7xvRunning DeepSeek on RISC-V with XTheadVector ExtensionThe demonstration address mentioned above:https://asciinema.org/a/eRkibHqHPMO5v09G8pUKpNkDBRunning DeepSeek on RISC-V with XTheadVector ExtensionWelcome to join the PLCT laboratory testing team internship![1] The 7th RISC-V Hardware and Software Frontier and Open Source Infrastructure Workshop, organized by the Software Institute, is held offline every two weeks in the 4th-floor conference room of the Software Institute. Currently, only offline attendance is available.

Related posts

  1. Exploring Future Chip Trends: Mastering RISC-V Architecture Quickly!
  2. Running DeepSeek on RISC-V with XTheadVector Extension Enabled
  3. A Decade of RISC-V in China: From Introduction to Independent Innovation
  4. C++ Learning Manual – Object-Oriented Programming (OOP) 30 – Constructors and Destructors
  5. Chapter 25 of ‘Getting Started with FreeRTOS’: Practical Memory Leak Detection
  6. Embedded C Programming: How to Properly Handle Fatal and Non-Fatal Errors During Program Execution?
  7. Stocks Related to Computing Power Chips!

Leave a Comment