Source: Internet
Robyn is a fast, high-performance Python web framework with a Rust runtime. It aims to provide near-native Rust throughput while benefiting from code written in Python. It has over 200k installations on PyPi.
Robyn can compete with other products such as Flask, FastAPI, Django, and preferred web servers. One of Robyn’s main advantages is that it can be put into production without an external web server, making it more efficient and streamlined.
Since all other/popular Python frameworks are written in Python or CPython, they are not friendly to concurrency due to the notorious GIL and slow execution speed. Robyn uses a Rust runtime and connected web server, attempting to move out of the GIL to improve runtime performance in various ways. While having a decoupled web server has several advantages, Robyn’s integrated web server allows us to better control performance execution.
We love to see performance data. This is our core goal. We are excited to see another milestone in our ongoing efforts: the write latency of the data pipeline has been reduced by 4 times, from 120 milliseconds to 30 milliseconds! This improvement is the result of transitioning from a C library accessed through a Python application to a fully Rust-based implementation.
This is a brief introduction to our architectural changes, actual results, and their impact on system performance and user experience.
Switching from Python to Rust So why are we switching from Python to Rust? Our data pipeline is used by all services!
Our data pipeline is the backbone of our real-time communication platform. Our team is responsible for replicating event data from all APIs to all internal systems and services. Data processing, event storage and indexing, connection status, etc. Our main goal is to ensure the accuracy and reliability of real-time communication.
Before the migration, the old pipeline used a C library accessed through Python services, which buffered and bundled data. This was indeed a key factor contributing to our latency. We wanted to optimize and knew it was achievable.
We explored the transition to Rust because we had previously seen performance, memory safety, and concurrency benefits. It was time to do it again!
High Emphasis on Rust’s Performance and Asynchronous IO Advantages Rust excels in performance-intensive environments, especially when combined with asynchronous IO libraries like Tokio. Tokio supports a multithreaded, non-blocking runtime for writing asynchronous applications in the Rust programming language. Migrating to Rust allows us to fully leverage these capabilities for high throughput and low latency, all with compile-time memory and concurrency safety.
Memory and Concurrency Safety Rust’s ownership model provides compile-time guarantees for memory and concurrency safety, avoiding the most common issues such as data races, memory leaks, and invalid memory access. This is beneficial for us.
Looking ahead, we can confidently manage the lifecycle of our codebase. If ruthless refactoring is needed later, it can be done. And there will always be “later needs”.
Technical Implementation of Architectural Changes, Service-to-Service, and Messaging Using MPSC and Tokio The previous architecture relied on a service-to-service messaging system, which introduced significant overhead and latency. Python services used a C library to buffer and bundle data. When messages were exchanged between multiple services, latency occurred, increasing system complexity. The buffering mechanism in the C library was a major bottleneck, leading to an end-to-end latency of about 120 milliseconds. We thought this was optimal because our average latency per event was 40 microseconds. While this looked good from the old Python service perspective, downstream systems were affected during unbundling, leading to higher overall latency.
When we deployed, the average latency per event increased from the original 40 microseconds to 100 microseconds. This did not seem optimal.
However, when we looked back at the reasons, we could see what was happening. The good news is that downstream services can now use events faster one by one without unbundling.
Overall end-to-end latency has the opportunity to improve significantly from 120 milliseconds to 30 milliseconds.
-
The new Rust application can trigger events concurrently immediately.
This approach was not possible in Python, as using different concurrency models would also require a rewrite. We might be able to rewrite it in Python. If a rewrite is necessary, why not do it best in Rust!
Resource Reduction in CPU and Memory: Our Python services consumed over 60% of kernel resources. The new Rust services consume less than 5% of resources across multiple cores. Memory reduction is also significant, with the Rust runtime occupying about 200MB, while Python requires several GB of memory.
New Architecture Based on Rust:
-
The new architecture leverages Rust’s powerful concurrency mechanisms and asynchronous IO capabilities.
-
Service-to-service messaging has been replaced by multiple instances of multi-producer, single-consumer (MPSC) channels.
-
Tokio is built for efficient asynchronous operations, reducing blocking and increasing throughput.
-
Our data flow has been simplified by eliminating the need for intermediate buffering stages, opting for concurrency and parallelism instead.
These measures improve performance and efficiency.
Rust Application Example This code is not a direct copy; it is just an alternative example to simulate the functionality of our production code. Additionally, this code only shows one MPSC, while our production system uses multiple channels.
-
Cargo.toml: We need to include dependencies for Tokio and any other crates we might use (e.g., asynchronous channels for events).
-
Event Definition: The event type is used in the code but not defined, as we have many types not shown in this example.
-
Event Stream: event_stream is referenced, but the way it is created is different from many streams. Depending on your approach, the example remains simple.
Here is a Rust example with code and a Cargo.toml file. There are also event definitions and event stream initialization.
Cargo.toml
[package]
name = "tokio_mpsc_example"
version = "0.1.0"
edition = "2021"
[dependencies]
tokio = { version = "1", features = ["full"] }
main.rs
use tokio::sync::mpsc;
use tokio::task::spawn;
use tokio::time::{sleep, Duration};
// Define the Event type
#[derive(Debug)]
struct Event {
id: u32,
data: String,
}
// Function to handle each event
async fn handle_event(event: Event) {
println!("Processing event: {:?}", event);
// Simulate processing time
sleep(Duration::from_millis(200)).await;
}
// Function to process data received by the receiver
async fn process_data(mut rx: mpsc::Receiver) {
while let Some(event) = rx.recv().await {
handle_event(event).await;
}
}
#[tokio::main]
async fn main() {
// Create a channel with a buffer size of 100
let (tx, rx) = mpsc::channel(100);
// Spawn a task to process the received data
spawn(process_data(rx));
// Simulate an event stream with dummy data for demonstration
let event_stream = vec![
Event { id: 1, data: "Event 1".to_string() },
Event { id: 2, data: "Event 2".to_string() },
Event { id: 3, data: "Event 3".to_string() },
];
// Send events through the channel
for event in event_stream {
if tx.send(event).await.is_err() {
eprintln!("Receiver dropped");
}
}
}
Rust Example Files
Cargo.toml: – Specifies the package name, version, and edition. – Includes the dependencies required for the “full” feature set of tokio.
main.rs:
-
Defines the event structure.
-
Implements the handle_event function to process each event.
-
Implements the process_data function to receive and process events from the channel.
-
Creates an event_stream with dummy data for demonstration purposes.
-
Uses the Tokio runtime to spawn a task that processes events and sends them through the channel in the main function.
Benchmark To validate our performance improvements, we conducted extensive benchmarking in development and staging environments. We used tools like hyperfine and criterion.rs to collect latency and throughput metrics. We simulated various scenarios to mimic production-like loads, including peak traffic periods and extreme cases.
Production Validation To assess the actual performance in a production environment, we implemented continuous monitoring using Grafana and Prometheus. This setup allows tracking key metrics such as write latency, throughput, and resource utilization. Additionally, alerts and dashboards were configured to promptly identify any deviations or bottlenecks in system performance, ensuring potential issues can be addressed in a timely manner. Of course, we cautiously deployed to a low traffic percentage over several weeks. The charts you see are from our full deployment after the validation phase.
Benchmarks Alone Are Not Enough Load testing proved the improvements. While yes, testing does not prove success, as it provides evidence. Write latency has consistently decreased from 120 milliseconds to 30 milliseconds. Response times have been enhanced, and end-to-end data availability has accelerated. These advancements significantly improve overall performance and efficiency.
Before and After In the old system, service-to-service messaging was buffered through a C library. This involved multiple services in the messaging loop, and the C library increased latency through event buffering. Due to Python’s Global Interpreter Lock (GIL) and its inherent operational overhead, Python services added an extra layer of latency. These factors led to higher end-to-end latency, complex error handling and debugging processes, and limited scalability due to bottlenecks introduced by event buffering and the Python GIL.
After implementing Rust, intermediary services were eliminated by passing messages directly through channels, while Tokio enabled non-blocking asynchronous IO, significantly increasing throughput. Rust’s strict compile-time guarantees reduced runtime errors, and we achieved powerful performance. Observed improvements include end-to-end latency reduced from 120 milliseconds to 30 milliseconds, enhanced scalability through efficient resource management, and improved error handling and debugging through Rust’s strict typing and error handling model. It is hard to argue for using anything else besides Rust.
Deployment and OperationsMinimal Operational Changes The deployment underwent minimal modifications to accommodate the migration from Python to Rust. The same deployment and CI/CD processes were used. Configuration management continued to leverage existing tools (like Ansible and Terraform), facilitating seamless integration. This allowed us to transition smoothly without disrupting existing deployment processes. This is a common approach. You want to make as few changes as possible during migration. This way, if issues arise, we can isolate footprints and find problems faster.
Monitoring and Maintenance Our applications integrate seamlessly with the existing monitoring stack, including Prometheus and Grafana, for real-time metric monitoring. Rust’s memory safety features and reduced runtime errors significantly lower maintenance overhead, resulting in a more stable and efficient application. It is gratifying to see our build system working correctly, and even better, we can capture errors during the development process on our laptops, allowing us to catch errors before pushing commits that could lead to build failures.
Real Impact on User Experience Improved data availability and faster write operations enable near-instantaneous data reads and indexing readiness, enhancing user experience. These enhancements include reduced data retrieval latency, resulting in more efficient and responsive applications. Real-time analytics and insights are also better. This provides businesses with up-to-date information for making informed decisions. Additionally, faster dissemination of updates across all user interfaces ensures users always have access to the latest data, enhancing collaboration and productivity for teams using our provided APIs. From an external perspective, the latency is evident. Combined APIs ensure data is now available and faster.
Enhanced System Scalability and Reliability Enterprises focusing on Rust will gain significant advantages. They will be able to analyze large volumes of data without slowing down the system. This means you can keep up with user loads. And let’s not forget the additional benefits of a more resilient system and less downtime. We run a business with a billion connected devices, and interruptions are absolutely unacceptable; continuous operation is a must.
The transition to Rust not only significantly reduced latency but also laid a solid foundation for future enhancements in performance, scalability, and reliability. We provide the best experience for our users.
Rust aligns with our commitment to providing the best API services for billions of users. Our experience enables us to meet and exceed the real-time communication needs of today and the future.
Original article: https://www.jdon.com/60376.html
Why is n8n the automation choice for tech teams? A complete guide from installation to practical use!
Abandon Flask, embrace FastAPI, one word: refreshing!
What is uv, the best tool in the Python ecosystem for the past decade, that has exploded in the entire community?