Team and Paper Information
Today, we introduce a work completed in collaboration with Southern University of Science and Technology, Ant Group, Peking University, and Zhongguancun Laboratory, titled “ASTERINAS: A Linux ABI-Compatible, Rust-Based Framekernel OS with a Small and Sound TCB” which has been published in the CCF-A conference ATC 2025 (2025 USENIX Annual Technical Conference).
-
Paper link: https://www.usenix.org/system/files/atc25-peng-yuke.pdf
Abstract
How can we build a feature-rich, general-purpose operating system (OS) based on Rust while achieving memory safety with a minimal and reliable Trusted Computing Base (TCB)? Existing Rust-based operating systems fail to meet this requirement due to the improper use of unsafe Rust in kernel development. To address this challenge, we propose a novel operating system architecture called framekernel, which realizes the full potential of Rust in internal privilege separation within the kernel, ensuring the minimality and soundness of the TCB. We introduce OSTD, a simplified framework for secure Rust OS development, and ASTERINAS, a framekernel OS fully implemented in secure Rust using OSTD, which is compatible with the Linux ABI. ASTERINAS supports over 210 Linux system calls and provides performance comparable to Linux while maintaining a minimal memory-safe TCB that occupies only about 14.0% of the codebase. These results highlight the practicality and advantages of the framekernel architecture in building secure and efficient operating systems.
Background and Motivation
Despite 30 years of research on memory safety errors in operating systems written in C, achieving true memory safety remains challenging. The recent CrowdStrike vulnerability incident illustrates this, where millions of Windows computers crashed due to out-of-bounds memory access in faulty drivers. It is estimated that 60-70% of security vulnerabilities in system software written in C stem from memory safety issues.
Rust is an efficient systems programming language that provides strong guarantees for memory, type, and thread safety without compromising runtime performance, thanks to its powerful type system and unique ownership model.
1. Ownership model, borrowing, and lifetimes. In Rust’s ownership model, each value has a unique variable, and the value’s lifetime is tied to the owner’s scope; it is dropped when it goes out of scope. Ownership can be borrowed through references, but must adhere to lifetime constraints enforced at compile time;
2. Type system. The Rust compiler implements a custom type system that combines comprehensive compile-time checks. Rust compilation ensures that all accesses occur within valid lifetimes, generates correct memory offsets, and performs boundary checks, thus guaranteeing memory safety in both time and space;
3. Undefined Behavior (UB). Undefined behavior in Rust refers to operations that compromise the language’s correctness and safety guarantees, including memory, thread, and type safety. The Rust reference book lists common UBs such as data races, memory accesses based on dangling or misaligned pointers, out-of-bounds memory access, violating pointer aliasing rules, and modifying immutable bytes. The Rust language team maintains an official UB detection tool called Miri, which serves as a de facto executable standard for UB. While Miri effectively detects UBs in standard Rust applications, it is not suitable for Rust-based operating systems due to its design not considering the needs of low-level systems programming, especially in memory management and hardware interaction.
In recent years, with the maturation and popularity of the Rust programming language, there has been a strong momentum in developing memory-safe operating systems based on Rust. Rust provides guarantees for memory safety through innovative language features such as ownership, borrowing, and lifetimes, achieving safe memory management without relying on garbage collection. With the support of Linus Torvalds, the Linux kernel has officially adopted Rust as its second programming language and integrated the Rust for Linux (RFL) project to facilitate writing leaf-type kernel modules in Rust. Additionally, new operating system kernels like Tock, RedLeaf, and Theseus have been built from scratch using Rust, further demonstrating Rust’s potential in this field. Although adopting Rust is an important step towards achieving kernel memory safety, it is not sufficient on its own, as Rust-based operating systems must include unsafe Rust code. The safety of various low-level controls required for kernel programming cannot be statically verified by the Rust compiler, thus only allowed in special code blocks marked with the unsafe keyword.
To mitigate the risks associated with unsafe in Rust, two widely accepted best practices have emerged: (1) use unsafe cautiously and (2) encapsulate unsafe code within safe abstractions. However, existing Rust-based operating system kernels often fail to meet these standards. Unsafe Rust code permeates a significant portion of Rust-based operating systems. The proportion of unsafe crates in Linux, Tock, RedLeaf, and Theseus is 55%, 93%, 62%, and 32%, respectively.
Given the limitations of previous work, the authors pose the question: Is it possible to build a feature-rich, general-purpose Rust-based operating system kernel almost entirely in safe Rust?
This paper introduces the framekernel, a novel operating system architecture designed to achieve a minimal and reliable TCB for Rust-based operating systems. In the framekernel architecture, the entire operating system resides in a single address space (similar to a monolithic kernel) and is implemented in Rust. The kernel is logically divided into two parts: a privileged OS framework (similar to a microkernel) and unprivileged OS services. Only the privileged framework is allowed to use the unsafe keyword, while the unprivileged services must be entirely written in safe Rust. As the TCB, the privileged framework encapsulates all low-level, hardware-facing unsafe operations behind safe APIs. Using these safe APIs, the unprivileged OS services can implement various OS functionalities, including device drivers. The framekernel minimizes the TCB size without incurring additional overhead from hardware isolation. Thus, the framekernel effectively combines the advantages of both monolithic and microkernel architectures.
In summary, this paper makes the following contributions:
-
Proposes the framekernel, a novel operating system architecture that implements internal privilege separation in the kernel using Rust and combines the advantages of monolithic and microkernel architectures;
-
Develops OSTD, a small yet robust operating system framework that facilitates the development of operating systems in safe Rust;
-
Develops ASTERINAS, a highly optimized operating system based on OSTD that is compatible with the Linux ABI;
-
Conducts extensive evaluations of ASTERINAS and OSTD regarding performance, TCB size, and security.
Framekernel Architecture
ASTERINAS introduces a novel operating system architecture called framekernel, which unleashes the full capabilities of Rust, bringing together the best features of both monolithic and microkernel designs.
In the framekernel architecture, the entire operating system resides in the same address space (like a monolithic kernel) and must be written in Rust. However, there is a twist—the kernel is divided into two halves: the OS framework (similar to a microkernel) and OS services. Only the OS framework is allowed to use unsafe Rust, while OS services must be specifically written in safe Rust.
Thus, the memory safety of the kernel can be reduced to the memory safety of the OS framework, minimizing the TCB associated with kernel memory safety. On the other hand, the single address space allows different parts of the kernel to communicate in the most efficient manner, such as through function calls and shared memory. Thanks to the framekernel architecture, ASTERINAS can provide better performance and enhanced security.
Figure 1 Differences between Traditional Kernel and Framekernel
OSTD Operating System Framework
ASTERINAS OSTD is a powerful and solid foundation for the secure kernel development of ASTERINAS, providing relevant high-level safe Rust APIs:
1. Memory management: e.g., allocating and accessing physical pages;
2. Task management: e.g., context switching between kernel tasks;
3. User space: e.g., manipulating and entering user space;
4. Interrupt handling: e.g., registering interrupt handlers;
5. Timer management: e.g., registering timer handlers;
6. Driver support: e.g., performing DMA and MMIO;
7. Boot support: e.g., retrieving information from the bootloader;
8. Synchronization: e.g., locking and sleeping.
These APIs are implemented for expressive abstractions within the OS, which can be used by OSTD clients when interacting with the secure kernel through OSTD. As shown in the figure below, to achieve secure user-kernel interaction, the API allows OSTD clients to jump to user space and execute until a trap occurs (UserMode), manipulate CPU registers in user mode (UserContext), and manage user address space (VmSpace). For secure kernel logic, it includes synchronization primitives (e.g., SpinLock, Rcu, Mutex, WaitQueue, and CpuLocal) and efficient data collection types (e.g., LinkedList and RbTree). To achieve secure kernel peripheral interaction, it provides mechanisms for registering interrupt handlers (IrqLine), performing MMIO and PIO (IoMem and IoPort), and creating coherent or streaming DMA mappings (DmaCoherent and DmaStream). All three aspects of operating system development can allocate and access physical pages through Frame (a frame) or Segment (multiple contiguous frames), referred to as frames.
Figure 2 Example of Interaction between OSTD and OSTD Clients
OSTD uses a frame metadata system to track the status of each page frame, including reference counts and customizable metadata fields. All per-frame metadata is stored in a large static array, which is allocated and initialized during the early boot phase. The reference count equals the total number of Frame and Segment instances referencing that frame, allowing OSTD to manage the frame lifecycle correctly.
ASTERINAS Operating System
The main architecture of ASTERINAS is shown in the figure below. It is an operating system built on top of the framekernel based on OSTD. ASTERINAS implements a large subset of Linux functionalities, including virtual memory, user processes, preemptive scheduling, IPC, page caching, virtual file systems, and sockets, providing over 210 system calls. It supports various file systems (e.g., Ext2, exFAT32, OverlayFS, RamFS, ProcFS, and SysFS), socket types (e.g., TCP, UDP, Unix, and Netlink), and devices (e.g., Virtio Block, Virtio Network, Virtio Vsock, USB controllers, and USB HID). It supports two CPU architectures: x86-64 and RISC-V.
Figure 3 ASTERINAS Operating System Architecture
Performance Testing
The authors conducted evaluations in three aspects: performance, Trusted Computing Base, and security.
1. In terms of performance, tests were conducted in a single-core environment (Intel i7-10700, 32GB RAM, SSD), with Linux 5.15 (CPU mitigations and large pages disabled for fair comparison) as the benchmark. The tests were divided into micro-benchmarking, macro-benchmarking, security overhead, and IOMMU optimization. In micro-benchmarking (LMbench), ASTERINAS achieved an average normalized performance of 1.08 (relative to Linux) across 54 tests, indicating performance comparable to or slightly better than Linux, although some operations like file system operations performed slightly lower due to unimplemented optimizations similar to Linux; in macro-benchmarking (Nginx, Redis, SQLite), ASTERINAS performed better on Nginx and Redis, but slightly worse on SQLite compared to Linux; in security mechanism overhead testing, the security checks of OSTD (e.g., boundary checks, ownership verification) introduced additional overhead, but most were below 3%, with the highest overhead being frame allocation (6.7%), having a minor impact on overall performance; in IOMMU optimization, using pooling mechanisms (Persistent Mapping) reduced the frequency of DMA mapping setups, significantly improving performance. Dynamic mechanisms led to bandwidth reductions (30% for network devices, 15% for block devices).
2. For the Trusted Computing Base assessment, the Linked Code Size (LCS) metric was used to measure the runtime TCB code size. The TCB of ASTERINAS occupies only 14.0% of the codebase, significantly lower than other Rust OS (RedLeaf 66.1%, Theseus 62.4%, Tock 43.8%).
3. In terms of security assessment, the authors developed a security tool called KERNMIRI, based on Rust’s official testing tool MIRI, supporting OS kernel-level testing. KERNMIRI enhances Miri by simulating physical memory and implementing basic paging systems. These improvements enable KERNMIRI to accurately interpret operating systems that require fine memory management. Additionally, KERNMIRI introduces extra padding to synchronize the operating system state, which can be used to indicate to KERNMIRI to activate page tables for specific addresses or notify KERNMIRI of changes in the state of physical pages, such as transitions between typed and untyped states. KERNMIRI can cover 93% of ASTERINAS code and is capable of performing undefined behavior fixes, ensuring the safety of OSTD.
For detailed content, please refer to
Y. Peng, H. Tian, J. Zhang, R. Li, C. Chen, J. Jiang, J. Xian, X. Wang, C. Xu, D. Zhou, Y. Luo, S. Yan, and Y. Zhang. “ASTERINAS: A Linux ABI-Compatible, Rust-Based Framekernel OS with a Small and Sound TCB.” In Proceedings of the 2025 USENIX Annual Technical Conference (USENIX ATC ’25), Boston, MA, USA, July 2025. ISBN 978-1-939133-48-9.