RTOS Benchmark is a test suite specifically designed to measure the core operation performance of Real-Time Operating Systems (RTOS). It quantifies the time and overhead of basic operations such as thread creation/switching, mutex/semaphore, timers, and message queues through a series of benchmark tests. Currently, it mainly supports Zephyr (qemu_x86, frdm_k64f) and FreeRTOS (frdm_k64f), with plans to expand to more boards/RTOS.
Why Use It? What Pain Points Does It Address?
- • From Observations to Data: Previously, claims about the speed of a certain RTOS were often based on feelings or single-point demos, lacking unified, reproducible quantitative data. RTOS Benchmark provides a standardized set of test cases and scoring methods.
- • Easier Cross-Comparison: Want to compare the thread switching cost of Zephyr vs FreeRTOS on a specific MCU? Run the same set of benchmarks, and the data can be directly compared.
- • Regression Testing: After kernel changes or scheduler optimizations, you can use this benchmark suite for performance regression testing, preventing a change from drastically increasing latency without anyone noticing.
- • Blueprint Role: The support for QEMU mainly provides developers with examples, making it easier to integrate other boards/architectures into the test suite. Although the results from QEMU should be taken with caution, it can facilitate CI setup.
Supported Platforms and Scope of Application
- • Zephyr: Supports frdm_k64f and qemu_x86 (Note: QEMU results are not as reliable as real boards)
- • FreeRTOS: Supports frdm_k64f
- • VxWorks: Support requires version 24.03 or higher, providing both POSIX and non-POSIX testing methods. More boards and RTOS may be added in the future.
Installation Requirements (General Environment) First, prepare the common dependencies:
- • ARM GNU Toolchain (for frdm_k64f): After installation, point ARMGCC_DIR to the installation directory.
- • Zephyr: Set up the environment according to the official Zephyr Getting Started guide, configuring ZEPHYR_BASE, west, etc.
- • FreeRTOS (if using FRDM): Download and extract the corresponding SDK from the NXP MCUXpresso SDK Builder, ensuring FreeRTOS support is selected.
- • pyOCD: Used for flashing the FRDM-K64F.
- • If running VxWorks, confirm that the VxWorks version is >= 24.03, and follow the official process to create VSB/VIP and include the benchmark component.
Advantages (Why It’s Worth Using)
- • Unified and Reproducible: Provides a standardized set of tests, facilitating comparisons across RTOS/boards.
- • Reasonable Coverage: Tests common core operations such as threads, synchronization primitives, messages, and timers.
- • Good Extensibility: Supports integrating new boards or RTOS as “blueprints,” with an active community for further expansion.
- • Supports Multiple RTOS: Already includes integration examples for Zephyr, FreeRTOS, and VxWorks.
Disadvantages & Limitations
- • Limited Reliability of QEMU Results: The latency model in a virtualized environment differs from real hardware.
- • Hardware Binding: Currently, there are not many officially supported real boards; if you want to run it on your specific hardware, you need to integrate and validate it yourself.
- • Single Metric: Focuses on the time/overhead of basic operations, while complex system-level performance (e.g., memory fragmentation, long-term stability) is not the main focus.
- • Requires Some Environment Configuration Cost: Especially the dependencies of different RTOS/SDKs, there is a learning curve for first-time users.
Quick Overview Table (A Starting Point)
| Item | Description |
| Test Content | Thread/task creation, context switching, mutex/semaphore, queues, timers, etc. |
| Currently Supported RTOS | Zephyr, FreeRTOS, VxWorks |
| Recommended Hardware | FRDM-K64F (officially supported), other boards can be extended |
| CI Friendliness | Can run on QEMU as a CI blueprint (results should be taken with caution) |
| Best Suited Audience | RTOS developers, kernel optimization engineers, teams with comparative testing needs |
ConclusionRTOS Benchmark is a very practical “performance checkup” tool, suitable for comparing the overhead of basic RTOS operations, performing performance regression, or integrating new boards into a unified testing system. It will not replace long-term system-level stress testing or functional testing, but it is quite effective in “quantifying low-level overhead.” For reliable data, it is best to run on real hardware and keep the testing environment (toolchain, clock configuration) consistent.
Project Address: https://github.com/zephyrproject-rtos/rtos-benchmark
If you want, I can:
- • Help you write an automation script to parse serial output into CSV;
- • Or provide specific steps to integrate a certain board you have into RTOS Benchmark. Just let me know what you need.