Fjall – A Safe Rust KV Storage Engine
Fjall is an embeddable, LSM-based, forbid-unsafe Rust key-value storage engine. Its goal is to be a reliable and predictable yet high-performance general-purpose KV storage engine suitable for small datasets, especially those larger than memory size. I have just released version 1.0, which stabilizes its data format for all future 1.x.x versions.
Its design is significantly influenced by the LevelDB/RocksDB architecture and generally offers similar performance. It has similar features, including:
- Range and prefix searches, as well as forward and backward iteration
- Data partitioning (referred to as “Column Families” in RocksDB)
- Cross-partition atomic write batching
- MVCC and snapshot reads
- Adjustable durability guarantees
- Size-tiered (write-optimized), level-tiered (read-optimized), and FIFO compression (for temporary data)
Repo https://github.com/fjall-rs/fjall
Lessons Learned from Developing Distributed Systems with Rust
Codethink is a company dedicated to promoting safe and reproducible software. When building distributed systems, they chose Rust as their tool because Rust offers multiple safety guarantees and reasonable package management. Here are some lessons they learned while building a 3-node Rust distributed system:
-
Rust Does Not Slow Down Development Speed:
- The explicit error return types in Rust (as opposed to exceptions) greatly reduce hidden failure points.
- The error propagation mechanism allows us to handle all errors in one place, making reasoning easier.
- The development speed in Rust is not slower than in other languages (like Python or C).
Challenges of Asynchronous Rust:
- Using asynchronous programming ensures that nodes can perform CPU-intensive work while keeping multiple communication channels open with other nodes.
- The complexity of asynchronous programming in Rust is partly to avoid sacrificing performance.
- Careful handling of shutdown and cancellation safety is required to ensure that services can stop or restart without corrupting data.
In summary, Rust performs excellently in building distributed systems, and although asynchronous programming can be challenging, the end result is successful. 🚀🦀
Blog https://www.codethink.co.uk/articles/2024/distributed_system_rust/
—
From the Daily Report Team Mike
Community Learning Exchange Platform Subscription:
- Rustcc Forum: Supports RSS
- WeChat Official Account: Rust Language Chinese Community