Running DeepSeek on RISC-V with XTheadVector Extension Enabled

This Friday, during an offline communication event organized by the Software Research Institute of the Chinese Academy of Sciences [1], partners from the PLCT laboratory testing team demonstrated how to use the toolchain integrated in RuyiSDK to run the DeepSeek R1 Distill model on the Milk-V Pioneer Box equipped with the SG2042 processor, and successfully accelerated it using the XTheadVector extension. This means that all chip modules/development boards supporting the XTheadVector (including RVV0.7.1) extension (for example, TH1520/Milk-V Meles) can make better use of hardware computing power when running large models like DeepSeek.We would like to emphasize our gratitude to the OpenBLAS open-source community. The BLAS software library is a fundamental library essential for AI, scientific computing, etc., and currently, only the OpenBLAS open-source project provides support for RVV0.7.1.The corresponding code for the PPT can be found at:https://github.com/wychlw/plct/blob/main/memo/deepseek_on_llama.cpp.mdRunning DeepSeek on RISC-V with XTheadVector Extension EnabledRunning DeepSeek on RISC-V with XTheadVector Extension EnabledRunning DeepSeek on RISC-V with XTheadVector Extension EnabledRunning DeepSeek on RISC-V with XTheadVector Extension EnabledRunning DeepSeek on RISC-V with XTheadVector Extension EnabledRunning DeepSeek on RISC-V with XTheadVector Extension EnabledRunning DeepSeek on RISC-V with XTheadVector Extension EnabledHere is the demonstration link:https://asciinema.org/a/hIX2a1zshOUyYMymv1Fadv7xvRunning DeepSeek on RISC-V with XTheadVector Extension EnabledThe demonstration link mentioned above:https://asciinema.org/a/eRkibHqHPMO5v09G8pUKpNkDBRunning DeepSeek on RISC-V with XTheadVector Extension EnabledWelcome to join the PLCT laboratory testing team for an internship![1] The 7th RISC-V Software and Hardware Frontier and Open Source Infrastructure Workshop, organized by the Software Institute, is an offline RISC-V seminar held bi-weekly in the 4th floor conference room of the Software Institute. Currently, participation is only available offline.

Related posts

  1. Exploring Future Chip Trends: Mastering RISC-V Architecture Quickly!
  2. Running DeepSeek on RISC-V with XTheadVector Extension
  3. A Decade of RISC-V in China: From Introduction to Independent Innovation
  4. C++ Learning Manual – Object-Oriented Programming (OOP) 30 – Constructors and Destructors
  5. Chapter 25 of ‘Getting Started with FreeRTOS’: Practical Memory Leak Detection
  6. Embedded C Programming: How to Properly Handle Fatal and Non-Fatal Errors During Program Execution?
  7. AI and Medical Imaging Fusion Modeling with 3D Printing for Precise Bone Tumor Resection
  8. Zhuanzhuan Frontend Weekly Issue 166: How to Build an AI Frontend Engineer Based on Multi-Agent Architecture
  9. Research on Pedestrian Recognition Based on Automotive Radar Sensors

Leave a Comment