It is time to establish a high-performance “progress bar” for RISC-V. The Jiachen Project invites RISC-V vendors to donate machines to participate in the co-construction of PTS.
Background, Mission, and Historical Significance of PTS
The idea of the RISC-V Open Source Software Performance Tracking System (Performance Tracking System, PTS) was first conceived in 2020. At that time, Mr. Wu Wei, the principal of the Jiachen Project and co-chair of the RISC-V International Foundation’s Code Speed SIG, hoped to build a visual system similar to Mozilla’s <span><span>AreWeFastYet</span></span><span><span>.</span></span><span><span>com</span></span> platform to track the performance of various important open-source software, including JavaScript Engines, on RISC-V chips and systems. At that time, the development of RISC-V IP and chips was still focused on the embedded field, and there were only a few models of development boards capable of running a Linux desktop environment, such as the SiFive Unmatched.
Screenshot of the Mozilla AreWeFastYet website, which is also the model for PTS.
At that time, the PTS project was quite ahead of its time, and the biggest concern was that if the performance gap between RISC-V chips and Arm/X86 was made public too early, it could have an unintended negative impact on the global ecological development momentum of RISC-V.
After a year of establishment, the Code Speed SIG was disbanded and reorganized for various reasons, and the PTS project was shelved until now. During this period, the PLCT laboratory attempted to initiate the construction of PTS several times in 2021 and 2023, but due to various resource constraints and layoffs, it was unable to reach the finish line.
On New Year’s Eve 2024, the Jiachen Project was launched, bringing a turning point. After more than a year of development, over 50 companies have joined the Jiachen Project, gradually forming mutually supportive projects such as the “RISC-V Development Board Drift Program” and the “Open Source Intern Joint Recruitment and Training Program,” making the construction of PTS possible again. The Jiachen Project has now accumulated sufficient venue sponsorship and operational support (from companies like Kuya Technology, Yingqi Intelligent, PLCT Laboratory, Hefei University of Technology, Dalian University of Technology, etc.) and has gathered enough RISC-V development board equipment (including 200 sets of TH1520 development boards donated by Alibaba’s Damo Academy team for LicheePi 4A). The hardware and funding support needed for the initial phase of PTS construction is now ready.
The construction and operation of PTS are crucial for the development of the global RISC-V ecosystem. Currently, RISC-V has completed its coverage in embedded and other fields and is sprinting towards high-performance computing and data center scenarios. The global open-source software ecosystem has provided considerable functional support for RISC-V, and the focus going forward will be to find various optimization opportunities to enhance speed. An automated performance evaluation system and the performance database it generates, which is easy to query and compare, will greatly assist developers in identifying performance regressions and finding optimization opportunities.
PTS will be hosted as an open-source project under the Jiachen Project’s GitHub organization account. If all goes well, starting from October 1, 2025, it will be accessible via <span><span><span>pts</span></span></span><span><span><span>rv2036</span></span></span><span><span><span>.</span></span></span><span><span><span>org</span></span></span>. All performance data collected by PTS will also be open-sourced and hosted under the <span><span>https</span></span><span><span>:</span></span><span><span>//github.com/rv2036</span></span> organization account.
Scope of Open Source Software and Evaluation Collection
The following software is planned to be included in the observation scope during the initial phase. More software of interest to the RISC-V community will be gradually added. Common performance evaluation suites include the SPEC CPU 20xx series, CoreMark series, etc. Different languages and execution environments have different test collections, and PTS will default to collecting as many open-source free test suites as possible for tracking.
Compiler Toolchains
- GNU Toolchain: Performance of compilers for languages such as C/C++, Fortran, etc.
- Clang/LLVM: Performance of compilers for languages such as Clang, Flang, etc.
- Rust Toolchain
Virtual Machines
- V8
- Spidermonkey
- OpenJDK
- Jeandle
- Go Runtime
- LuaJIT
- .NET
Emulators
- QEMU
- Box64
- Wine-CE
System Libraries and Computational Libraries, Computational Stacks
- glibc
- musl-c
- llvm libc
- OpenBLAS
- llama.cpp
- Eigen
- Highway
- etc.
Scope of RISC-V Chips/Systems to be Tracked
Main Testing Models (sufficient devices)
- TH1520: Sipeed LicheePi 4A
- SG2042: Milk-V PioneerBox
Standard Testing Models (at least 1 device can be used stably for a long time)
- EIC7700: Milk-V Megrez
- K1: Sipeed LicheePi 3A
Expected Testing Models (no stable machines yet, but call for donation)
- Sophgo SG2044
- A210
- SpacemiT K3
- DP1000: Milk-V Titan
Benchmark Operating Systems
- RevyOS: The Debian distribution used by default for the XuanTie IP series
- KarsierOS: A commercially maintained version developed and maintained by Kuya Technology based on the openEuler community version
- Ubuntu: Canonical provides native support for some RISC-V development boards
- deepin: A community of Jiachen Project members, with strong localization advantages in graphical interfaces
- openKylin: A community of Jiachen Project members, with strong localization advantages in graphical interfaces
Selection of Comparison Baselines
PTS will gradually introduce the following hardware systems as reference comparisons for non-RISC-V architectures:
- RaspberryPi 4B – Q3 2025
- RaspberryPi 5 – Q4 2025 – call for donation
- Mac mini w/ M1 – Q4 2025 – call for donation
- AMD9950x – Q1 2026 – call for donation
- Mac mini w/ M4 – Q2 2026 – call for donation
Roadmap and Results Release
Starting from Q4 2025, PTS will release the “RISC-V Performance Improvement Report” within 15 days after the end of each quarter.
Q3 2025
- PTS begins construction.
- The first batch of RISC-V devices goes online (TH1520, SG2042, Xiangshan Nanhu single-core).
- V8 enters the PTS observation scope.
- Baseline RaspberryPi 4B goes online.
Q4 2025
- The inaugural issue of the “RISC-V Performance Improvement Report” is published.
- Spidermonkey enters the observation scope.
- GNU Toolchain enters the observation scope.
- Clang/LLVM enters the observation scope.
- SPEC CPU 2006/2017 enters the observation scope.
- Xiangshan Nanhu quad-core goes online.
- Milk-V Titan goes online.
- ESWIN EIC7700 SBC goes online.
- Mac mini M1 goes online.
- RaspberryPi 5 goes online.
Q1 2026
- A210 device goes online.
- Xiangshan Kunming Lake series chips go online.
- More chips go online.
Q2 2026
- Testing equipment increases to 100 units, over 20 types.
- Common open-source databases enter the evaluation scope.
- Stability testing is increased.
Q3 2026
- Testing equipment increases to 200 units, tracking software projects exceed 50, and performance evaluation metrics exceed 500.
- All common open-source test suites related to HPC are included in the evaluation scope.
Q4 2026
- All evaluation metrics of interest to data centers enter the observation scope.
Q1 2027
- Enter stable maintenance mode.
- Testing hardware exceeds 500 units (sets), tracking software projects exceed 100, and performance metrics exceed 1000.
- Add at least 2 models of 128-core RISC-V processors.
Collaboration with the Open Source Community and Teams
Even after 5 years of preparation, this is still a very ambitious project, with too much work to be done and explored. The principal of the Jiachen Project welcomes all developers and RISC-V-related teams interested in this project to join.
RISC-V Development Board Solicitation
RISC-V chip and development board manufacturers are welcome to donate or lend devices. Vendors participating in the Jiachen Project PTS will be able to observe the support situation of the open-source software community for their chips and hardware systems more quickly, promptly fix performance regressions, and maintain a competitive advantage.
Summary and Outlook
RISC-V is a vibrant world, and you are welcome to join! Let us take a decade to complete the adaptation and optimization for RISC-V in all foundational key industry fields and form a top talent network of over ten thousand people.
References:
- [1] https://lists.riscv.org/g/sig-code-speed/topic/slidesproposalof_code/79032157
- [2] https://arewefastyet.com/win11/benchmarks/overview?numDays=90
- [3] https://github.com/rv2036