Rust: The Future of Systems Programming, C as New Assembly

Until now, OSTS has been a closed event for Intel, where business and technical leaders gather to discuss various trends, technologies, and innovations that help shape the open-source ecosystem. However, this year was different; for the first time, they invited non-Intel attendees, including media, partners, and developers.

The conference featured keynote speeches, over 50 technical sessions, panels, and demonstrations covering all open-source technologies Intel is involved in, including integrated software stacks (edge computing, AI, infrastructure), firmware, embedded and IoT projects, and cloud system software.

Systems programming is about developing and managing software that serves as a platform for other software. To achieve the necessary performance and abstraction, system software has a direct or closely related interface with computer hardware. Unlike application programming, the software it provides does not serve the user but is software that serves the computer hardware.

Triplett broadly defines systems programming as “any software that is not an application.” This includes BIOS, firmware, bootloaders, operating system kernels, embedded and similar low-level code, and virtual machine implementations. Triplett also considers web browsers as system software because they are “not just applications”; they are, in fact, “platforms for websites and web applications.”

Previously, most system software, including BIOS, bootloaders, and firmware, was written in Assembly. In the 1960s, experiments began to support hardware in high-level languages, creating languages such as PL/S, BLISS, BCPL, and extended ALGOL.

In the 1970s, Dennis Ritchie created the C programming language for the Unix operating system. It originated from the typeless B programming language, with C packing powerful high-level features and specific characteristics well-suited for writing operating systems. Many components of UNIX, including its kernel, were eventually rewritten in C. Many other system software, including the Oracle database, a large part of the Windows source code, and the Linux operating system, are written in C.

At this point, C was widely adopted. However, what exactly made developers comfortable migrating to C? Triplett believes that two things must make developers feel at ease for a migration from one language to another: features and equivalence.

• First, the language should provide “sufficiently compelling” features. “It can’t just be a little better. It must be significantly better to justify the effort and engineering time required for the migration,” he added. Compared to Assembly, C offers more features. It provides a certain level of type safety, portability, productivity through high-level constructs, and more readable code.

• Secondly, the language must provide equivalence, meaning developers must be able to believe that its capabilities are on par with Assembly. He stated, “It can’t just be better; it also can’t be worse.” Aside from being faster and able to represent any data type that Assembly can, it also has what Triplett calls an “escape hatch.” This means you can migrate gradually and, if needed, use Assembly together.

Triplett believes that C is now becoming like Assembly was a few years ago. “C is the new Assembly,” he summarized. Developers are looking for a high-level language that can not only solve the problems that C cannot but also leverage other exciting features provided by these languages. This language, aimed at migrating developers away from C, should be memory-safe, provide automatic memory management, safety, and so on.

“Any language that wants to be better than C, if it truly is a compelling alternative, must provide more than just preventing buffer overflows. People care about usability and productivity. They care about writing self-explanatory code and accomplishing more with less code. It also needs to address safety issues. Usability and productivity are closely related to safety. The less code you need to write to get the job done, the smaller the chance of bugs and security vulnerabilities,” he explained.

As early as 2006, Mozilla employee Graydon Hoare began working on Rust, initially as a personal project. In 2009, Mozilla began sponsoring the project, expanding the team and further advancing the development of the language.

One of the reasons Mozilla was interested is that Firefox is written in over 4 million lines of C++ code, with many very serious vulnerabilities. Rust was designed with safety and concurrency in mind, making it the best choice for rewriting many Firefox components under the Quantum project. They also used Rust to develop Servo, an HTML rendering engine that will eventually replace Firefox’s rendering engine. Many other companies have also begun using Rust in their projects, including Microsoft, Google, Facebook, Amazon, Dropbox, Fastly, Chef, Baidu, and others.

Rust addresses the memory management issues in C. It provides automatic memory management, so developers do not have to manually call free on every object. What distinguishes it from other modern languages is that it has no garbage collector or runtime system of any kind. Instead, Rust has the concepts of ownership, borrowing, references, and lifetimes. “Rust has a system where it can declare whether a specific use of an object is that object’s owner or just temporarily borrowing it. If you are just borrowing an object, the compiler tracks it. As long as you reference it, it stays the same. Rust ensures that the owner of the object releases it when done; it inserts free calls at compile time, with no additional runtime overhead,” Triplett explained.

Having no runtime is also an advantage of Rust. Triplett believes that languages with runtime are difficult to use as systems programming languages. He added, “You must initialize the runtime before you can call any code, and you must use that runtime to call functions, and the runtime itself may run additional code in the background at unexpected times.”

Rust also aims to provide safe concurrent programming. This is the same feature that enables memory safety; it tracks which thread owns which object, which objects can be passed between threads, and which objects need to acquire locks.

These features are compelling enough to attract developers to choose Rust for systems programming. However, when it comes to the second criterion, Rust is still unable to achieve equivalence with C.

Triplett’s first contribution to the Rust programming language was RFC 1444, which began in 2015 and was accepted in 2016. This RFC proposed native support for C-compatible unions in Rust, which would be defined by a new “context keyword” union. Triplett recognized the necessity of this proposal because he wanted to build a virtual machine in Rust, and the Linux kernel interface /dev/kvm needed unions for this purpose.

“I worked with the Rust community and language team to introduce unions into Rust, and because of this work, I am now a member of the Rust language governance team, helping to evaluate and guide other changes to the language,” he added.

In 2016, he discussed this RFC in detail at the first RustConf.

Another feature Triplett researched was supporting unnamed structs and union types in Rust. This has been a widely used extension in C compilers for decades and is also included in the C11 standard. This allows developers to group and layout fields in any way to match C data structures used in Foreign Function Interfaces (FFI). When this proposal is implemented, Rust will be able to represent this type with the same name as structs without artificially inserting field names, making it less confusing for users of existing platform-complete interfaces.

Systems programming often involves low-level operations and handling processor-specific details, such as privileged instructions. To this end, Rust supports inline assembly using the “asm!” macro. However, it is currently only available in nightly builds of the compiler and is not stable. Triplett is working with other Rust developers to draft a proposal to introduce more robust syntax for inline assembly. For more details on inline assembly support, check out this pre-RFC.

Many Intel processors, including Xeon Scalable ‘Cooper Lake-SP’, now support BFLOAT16, a new floating-point format. This is a 16-bit version of the 32-bit IEEE 754 single-precision floating-point format, truncated for use primarily in deep learning. This format is also used in machine learning libraries that handle large datasets, such as TensorFlow. It also makes interoperability with existing systems, functions, and storage easier. This is why Triplett is working to add BFLOAT16 support in Rust, so developers can utilize the full capabilities of their hardware.

This is one of Triplett’s significant announcements. He initiated a working group focused on achieving complete equivalence with C. In this group, his goal is to collaborate with the Rust community and other Intel developers to define the specifications for remaining features that need to be implemented in Rust for systems programming. This group will also focus on supporting systems programming with stable versions of Rust, not just experimental nightly builds of the compiler.

In a recent Reddit discussion, Triplett shared the current status of the working group, stating, “Before answering questions, let me clarify: the FFI/C equivalence working group is in the startup process and is not fully operational yet. Once fully operational, I will post relevant information and its initial goals here and elsewhere.”

If interested, you can watch Josh Triplett’s complete OSTS video speech to learn more about Intel’s contributions to Rust.

Original link:

https://hub.packtpub.com/rust-is-the-future-of-systems-programming-c-is-the-new-assembly-intel-principal-engineer-josh-triplett/