Source | Tencent Cloud Developer
Author | Yu Yulong
This article mainly interprets and analyzes content related to Rust, hoping to provide some experience and help for developers interested in this area.
About RustRust is a strongly typed, compiled, and memory-safe programming language. The earliest version of Rust was originally a private project of a Mozilla Foundation employee named Graydon Hoare. In 2009, Mozilla began sponsoring the development of the project, and in 2010, Rust achieved self-hosting, using Rust to build its own compiler.Mozilla applied Rust in building the next-generation browser layout engine, Servo, which integrated its CSS engine into Firefox starting in 2017.Originally intended as a memory-safe language to replace C++ or C for building large low-level projects like operating systems and browsers, Rust has also gained attention in the frontend industry due to Mozilla’s involvement, and its ecosystem has gradually flourished.
Memory Safety – A Major Selling Point of Rust
As we all know, mainstream programming languages can generally be divided into two categories: one category uses automatic garbage collection (GC), such as Golang, Java, JavaScript, etc., while the other category includes C++ and C, where users need to manage memory manually.
Most languages have similar memory models.
When code is executed, the values corresponding to variables are pushed onto the stack sequentially. When the code execution ends for a certain scope, the values corresponding to the variables are popped from the stack. The stack, being a Last In First Out (LIFO) structure, fits the scope of programming languages well – the outermost scope is declared first and ends last. However, the stack cannot insert values in between, so it can only store values that, once declared, do not change in size, such as int, char, or fixed-length arrays. Other values, such as variable-length arrays (vector) and variable-length strings (String), cannot be stored in the stack.
When a programming language needs a space of unknown size, it requests the operating system to allocate a block of memory, returning the memory address – a pointer – to the program. Thus, the programming language successfully stores this data in the heap and keeps the pointer in the stack – because the size of the pointer is fixed; a pointer in a 32-bit program is always 32 bits, and in a 64-bit program, it is 64 bits.
Data in the stack does not require memory management. As the code executes, it is easy to determine whether a variable is still in use – once the variable’s scope ends, it can no longer be accessed, indicating that the variable is no longer useful. Memory management in the stack is sufficient through continuous push and pop operations without programmer intervention.
However, data in the heap is different. The program only has a memory pointer, and the actual memory block is not in the stack and cannot be automatically destroyed with the stack. The program cannot automatically clean up the memory corresponding to the pointer when the pointer variable in the stack is destroyed, as multiple variables may hold pointers pointing to the same memory block, leading to unexpected situations when cleaning up that memory block.
Based on this, some programs come with a very complex GC algorithm, such as reference counting, which counts how many variables point to a memory block. When the reference count reaches zero, it indicates that all pointers to this location have been destroyed, and the memory block can be cleaned. Other programs require manual memory management, where any space allocated in the heap must be manually cleaned up.
Both methods have their pros and cons. The former requires the program to carry a runtime that contains the GC algorithm, increasing the program’s size, while the latter leads to memory unsafety, as the responsibility for memory management falls on the programmer, greatly affecting the safety of the code. Forgetting to reclaim memory can lead to increased memory usage, and incorrect reclamation can lead to deleting data that should not be deleted. Additionally, modifying data through pointers can cause overflows into other blocks, leading to unintended data modifications.
Rust, on the other hand, adopts a completely new memory management approach. This approach can be simply summarized as: the programmer and compiler reach a certain agreement, and the programmer must write code according to this agreement. When the programmer adheres to this agreement, it becomes very clear whether a memory block is still in use, so clear that it can be determined at compile time without running the program. The compiler can then insert memory reclamation code at specific locations in the code to achieve memory recovery. In other words, Rust essentially avoids situations where it is difficult to determine whether a memory address is still in use by restricting the use of references, leaving the remaining situations easy to determine – simple enough that a compiler can handle it without needing a professional programmer.
The major benefits of this approach are:
- No need for GC algorithms and runtime. It is essentially still manual reclamation, but the compiler inserts the manual reclamation code, relieving the programmer from having to write it.
- As long as compilation passes, it is guaranteed to be memory safe.
Implementation Principles
The memory safety mechanism of Rust can be said to be unique. It has a simple and easy-to-understand mechanism called the ownership system, which involves two core concepts: ownership and borrowing.
Ownership
Any value, including pointers, must be bound to a variable, which we refer to as the variable owning the value. For example, in the following code, the variable str owns the value “hello”.
let str = "hello"When the scope of str ends, the value of str will be cleaned up, and str will no longer be valid. This is consistent with almost all mainstream languages and is easy to understand.
However, note that Rust distinguishes between fixed-length strings and variable-length strings. The previous example is a fixed-length string, as its length is immutable and can be stored on the stack. Therefore, the following code can execute correctly, just like in almost all mainstream languages:
let str = "hello world";
let str2 = str;
println!("{}", str);
println!("{}", str2);
But if we introduce a variable-length string stored on the heap, let’s take a look at the same code:
fn main() {
let str = String::from("hello world");
let str2 = str;
println!("{}", str);
println!("{}", str2);
}At this point, we would be surprised to find that the code throws an error. Why is that?
The reason is that in the first piece of code, the value of the variable str is stored on the stack, and str owns the string “hello world” itself. So if we assign str2 = str, it effectively creates another variable str2 that also owns the same string. This is a “memory copy”. Both variables own their respective copies of “hello world”, but they are not the same instance.
In the second piece of code, what we have is essentially just a pointer to a memory block. When we assign str2 = str, we are actually assigning the address value of str to str2. In other languages, this would likely not be a problem, but str and str2 will point to the same memory address, and modifying str will also change str2. However, in Rust, the same value can only be bound to one variable, meaning that a variable has ownership of a value, just like an item can only belong to one person at a time! When we assign str2 = str, the address value stored in str is no longer owned by str; it now belongs to str2. This is called “ownership transfer”. Therefore, str becomes invalid, and using an invalid value will naturally result in an error.
The following situations can lead to ownership transfer:
The assignment operation mentioned above:
let str = String::from(“hello world”); let str2 = str; // str loses ownership!
Passing a value into another scope, such as a function:
let str = String::from(“hello world”); some_func(str); // At this point, str becomes invalid.
Thus, we can easily see that for the same memory address, it can only be stored in one variable at a time. If this variable goes out of scope and cannot be accessed anymore, then this memory block can be released. This judgment process is very simple and can be implemented at the static check stage by the compiler, allowing Rust to easily achieve memory safety.
However, the above writing style can be quite counterintuitive. While it indeed solves the memory safety issue, it can be cumbersome. For example, I want to pass str to a method for some logical operations, and after the operations, I still hope to read str, similar to the following code:
fn main() {
let mut str1 = String::from("hello world"); // Here, mut just indicates that this variable is mutable, not a constant.
add_str(mut str1, "!!!");
println!("{}", str1);
}
fn add_str(str_1: String, str_2: &str) {
str_1.push_str(str_2);
}
We expect to operate on str, adding three exclamation marks and then printing it out. This code will certainly be incorrect because when str is passed into the add_str method, ownership is transferred to the variable str_1 within the method, and it no longer has ownership, thus cannot be used. This situation is quite common, and the simple ownership mechanism complicates the issue. Therefore, Rust has another mechanism to solve the following problem: References and Borrowing.
Borrowing:Although a value can only have one variable owning its ownership, just like a person can lend their belongings to others, a variable can also lend the value it owns to others. The above code can be slightly modified:
fn main() {
let mut str1 = String::from("hello world");
add_str(&mut str1, "!!!");
println!("{}", str1);
}
fn add_str(str_1: &mut String, str_2: &str) {
str_1.push_str(str_2);
}
The add_str function now receives &mut str1 instead of mut str, which means it borrows the data from mut str1 for use, but the ownership remains with str1. The conditions for memory block reclamation are still based on the scope of str1 finishing execution and the memory address it holds being popped from the stack and destroyed.The essence of these two mechanisms is that counting references to a memory block becomes exceptionally simple. As long as the variable corresponding to this memory address is in the heap, the reference count is 1; otherwise, it is 0. There are only these two situations. There is absolutely no case where multiple variables point to the same memory address, drastically reducing the complexity of the reference counting GC algorithm. This reduction allows for static checks to identify all necessary GC timings and perform GC, eliminating the need for a complex runtime.
Other Features of Rust
With the aforementioned features, Rust has become a perfect alternative to C++. Currently, in the frontend field, there are two directions for using Rust: one is to create higher-performance frontend tools with Rust, and the other is to use it as a programming language for WASM, compiling into WASM modules that can run in the browser.
High-Performance Tools
Previously, if the frontend wanted to create a high-performance tool, the only choice was gyp, using C++ to write code and compiling it into an API callable by Node.js. Well-known libraries like saas-loader are implemented this way. However, in many cases, most frontend tools are written in JavaScript without much concern for performance, such as Babel, ESLint, webpack, etc. A significant reason for this is that C++ is quite difficult to learn. With dozens of versions of C++ features, it takes a lot of time to learn, and even after learning, a lot of experience is required to manage memory effectively and avoid memory leaks. Rust, on the other hand, is young, has no dozens of versions of standards, and features a modern package manager similar to npm. More importantly, it does not leak memory, which has led to a surge of high-performance tools written in Rust, despite its short history. Examples include:
-
SWC, a Rust-written library that wraps a Node.js API, functions similarly to Babel but achieves 40 times the performance thanks to Rust.
-
Rome, also based on Rust, was created by the author of Babel, Sebastian. Rome encompasses tools for compilation, code detection, formatting, packaging, and testing frameworks, aiming to be a comprehensive tool for handling JavaScript source code.
-
RSLint, a Rust-written lint tool for JS code, aims to replace ESLint.
As frontend complexity increases, we will inevitably pursue better-performing toolchains. In a few years, we may see projects using the stable versions of SWC and Rome running in production environments.
WASM
Furthermore, with the advent of WASM, the frontend is also searching for the most suitable language to support WASM, and currently, Rust seems to be a strong candidate. For WASM, languages with runtimes are unacceptable, as they would increase the package size by including runtime code along with business logic, which is detrimental to user experience. By excluding languages with runtimes, the frontend’s options become limited to C++ and Rust, with Rust’s advantages making it more appealing. Additionally, Rust provides good support in this area; its official compiler supports compiling Rust code into WASM code, and tools like wasm-pack enable quick construction of WASM modules. Here’s a simple demonstration with code extracted from the previously mentioned SWC:
#![deny(warnings)]
#![allow(clippy::unused_unit)]
// Import other packages or standard libraries, external libraries
use std::sync::Arc;
use anyhow::{Context, Error};
use once_cell::sync::Lazy;
use swc::{
config::{ErrorFormat, JsMinifyOptions, Options, ParseOptions, SourceMapsConfig},
try_with_handler, Compiler,
};
use swc_common::{comments::Comments, FileName, FilePathMapping, SourceMap};
use swc_ecmascript::ast::{EsVersion, Program};
// Import WASM-related libraries
use wasm_bindgen::prelude::*;
// Use the wasm_bindgen macro; this means that this method will be compiled to WASM, and its name will be transformSync,
// with TypeScript type transformSync
#[wasm_bindgen(
js_name = "transformSync",
typescript_type = "transformSync",
skip_typescript
)]
#[allow(unused_variables)]
// Define a method that is public and can be called externally. The purpose of this method is to transform high-version JS into low-version JS.
// We won't bother with the internal logic.
pub fn transform_sync(
s: &str,
opts: JsValue,
experimental_plugin_bytes_resolver: JsValue,
) -> Result<JsValue, JsValue> {
console_error_panic_hook::set_once();
let c = compiler();
#[cfg(feature = "plugin")]
{
if experimental_plugin_bytes_resolver.is_object() {
use js_sys::{Array, Object, Uint8Array};
use wasm_bindgen::JsCast;
// TODO: This is probably very inefficient, including each transform
// deserializes plugin bytes.
let plugin_bytes_resolver_object: Object = experimental_plugin_bytes_resolver
.try_into()
.expect("Resolver should be a js object");
swc_plugin_runner::cache::init_plugin_module_cache_once();
let entries = Object::entries(&plugin_bytes_resolver_object);
for entry in entries.iter() {
let entry: Array = entry
.try_into()
.expect("Resolver object missing either key or value");
let name: String = entry
.get(0)
.as_string()
.expect("Resolver key should be a string");
let buffer = entry.get(1);
// https://github.com/rustwasm/wasm-bindgen/issues/2017#issue-573013044
// We may use https://github.com/cloudflare/serde-wasm-bindgen instead later
let data = if JsCast::is_instance_of::(&buffer) {
JsValue::from(Array::from(&buffer))
} else {
buffer
};
let bytes: Vec<u8> = data
.into_serde()
.expect("Could not read byte from plugin resolver");
// In here we 'inject' externally loaded bytes into the cache, so
// remaining plugin_runner execution path works as much as
// similar between embedded runtime.
swc_plugin_runner::cache::PLUGIN_MODULE_CACHE.store_once(&name, bytes);
}
}
}
let opts: Options = opts
.into_serde()
.context("failed to parse options")
.map_err(|e| convert_err(e, ErrorFormat::Normal))?;
let error_format = opts.experimental.error_format.unwrap_or_default();
try_with_handler(
c.cm.clone(),
swc::HandlerOpts {
..Default::default()
},
|handler| {
c.run(|| {
let fm = c.cm.new_source_file(
if opts.filename.is_empty() {
FileName::Anon
} else {
FileName::Real(opts.filename.clone().into())
},
s.into(),
);
let out = c
.process_js_file(fm, handler, &opts)
.context("failed to process input file")?;
JsValue::from_serde(&out).context("failed to serialize json")
})
},
)
.map_err(|e| convert_err(e, error_format))
}
We can see that as long as there is an existing Rust library, transforming it into WASM is very simple. Readers can also try to experiment with Golang and C++ for WASM and find that the entire process with Rust is significantly simpler.
Are There Any Issues?
While I’ve mentioned many benefits of Rust, I faced some challenges while learning it, largely because Rust is quite unique.For example, in most programming languages, declaring variables and constants has different syntax, such as JavaScript distinguishing between let and const, Go distinguishing between const and var, or Java using final to declare constants. But Rust is special; by default, variables declared are constants, and mutable variables need to be explicitly declared with let mut a=1.There are many such unique aspects in Rust. While most of them are just different design philosophies and do not imply superiority or inferiority, such designs can indeed impose a cognitive burden on developers from other languages.From my learning experience, the difficulty of learning Rust is not low; it may not be easier than C++. Some in the community suggest that to truly understand Rust’s elegance, one should first learn C++. Those wanting to learn Rust should be prepared for this.
Yu Yulong
Author of Tencent Cloud Developer Community [Technical Insights – Original Collection by Tencent Tech].Tencent Frontend Development Engineer, graduated from Xiangtan University, currently responsible for frontend development of medical SaaS products under Tencent Medical Health Studio, and involved in the development and maintenance of some frontend toolchains within the team.
Click the video below
Follow 【51CTO Technical Stack】 video account
Leave a comment in the video comments section
Three lucky users will be drawn to receive fan benefits
● Maimao dismissed employees for absenteeism, ordered to pay 240,000; IntelliJ IDEA 2022.2.2 released; Adobe announces acquisition of Figma | T News
● The game