Yu Ruiji
First-year graduate student at the School of Computer Science, Nanjing University, open-source intern for the rk8s project, involved in the development of image building tools and file system direction in rk8s, currently mainly responsible for testing and maintaining libfuse-fs.
libfuse-fs （https://crates.io/crates/libfuse-fs） is a FUSE (Filesystem in Userspace) library developed in Rust, aimed at providing developers with a simple, easy-to-use, and efficient file system development API.
The core idea of FUSE is to allow developers to implement file system logic in user space without writing kernel modules. File operations initiated by applications (such as read, write, rename) first reach the kernel’s VFS, and then the FUSE kernel module forwards them to user space processes, where the user program decides how these operations are actually executed.
Currently, libfuse mainly includes two core file system implementations:
·Passthrough File System: Acts as a “proxy”, forwarding user file operation requests unchanged to the underlying real file system.
·Overlay File System: One of the core technologies for container image building. It uses union mount technology to overlay multiple directories (the top layer is writable, while the other layers are read-only), presenting a unified merged view to the user.
The Linux kernel has provided an implementation of the Overlay file system, and the comparison with the user-space implementation of the Overlay file system is as follows:
The design of libfuse was inspired by fuse-backend-rs（https://github.com/cloud-hypervisor/fuse-backend-rs）, and we designed an asynchronous FUSE API based on it. Currently, libfuse has been implemented in several of our core projects:
·mega/Scorpio：A modern Git client based on FUSE, aimed at monorepo.（https://github.com/web3infra-foundation/mega/scorpio）
·rk8s：Our recently developed lightweight Kubernetes distribution, where libfuse is responsible for the image building process.（https://github.com/rk8s-dev/rk8s）
From the early prototype to now handling actual business, we have encountered many bugs and have navigated various “pits” in FUSE development during the implementation process. It is these challenges that have driven the continuous iteration of libfuse, evolving it from a “barely usable” library into a robust infrastructure component.
During this period, we focused on improving the following features:
· Designed and implemented a hard link counting mechanism to ensure the correctness of file link logic.
· Introduced a file handle caching mechanism to enhance I/O performance.
· Added a uid/gid mapping mechanism to support permission management in containerized scenarios.
·…
[A Real Debugging Experience]
Recently, we successfully implemented the image building function in rk8s using libfuse. This process involved a large number of concurrent read and write operations and synchronization of underlying files, which fully validated the robustness of libfuse in complex scenarios.
During the process of fixing bugs and iterating features, we realized that merely covering business scenarios is not enough. We need a complete regression testing suite to conduct a “comprehensive examination” of libfuse, in order to uncover those corner cases that are difficult to trigger in daily use. After research, we selected IOR, fio, and xfstests as our “exam subjects”:
·IOR：A standard I/O stress testing tool in the high-performance computing (HPC) field, adept at testing sequential read/write, large file throughput, and high concurrency access scenarios.
·fio：A general-purpose I/O performance testing tool that can simulate almost all real application disk access patterns (sequential/random, synchronous/asynchronous, mixed load, etc.).
·xfstests：The “gold standard” in the file system field. It includes unit tests and integration tests, and is the preferred tool for all mainstream Linux file system developers to check correctness, stability, and boundary conditions.
Among the above tools, xfstests is undoubtedly the most rigorous. It has accumulated a vast number of test cases over years of development, capable of extremely detailed verification of file system consistency. It is worth mentioning that fuse-backend-rs also considers it an important part of its integration testing.
Due to the characteristics of the FUSE protocol itself, there are a few known incompatible cases in xfstests. After excluding about 10 such cases, libfuse successfully passed the remaining over 700 test cases. The test results are as follows:
Although we passed the rigorous tests of xfstests, our refinement of libfuse has not stopped. Currently, we are advancing the integration of libfuse with the newly developed distributed file system slayerfs. In the future, we hope libfuse can further optimize performance and improve mechanisms, providing reliable underlying support for more projects.