AI Empowerment: Accelerating Linux Kernel Maintenance

AI Empowerment: Accelerating Linux Kernel Maintenance

AUTOSEL is a tool that utilizes large language models (LLM) to assist in the maintenance of the Linux kernel, particularly excelling in repetitive tasks such as patch porting, code generation, and CVE classification. It alleviates the workload of maintainers and enhances efficiency and consistency through techniques like semantic code analysis, cross-validation, and retrieval-augmented generation.

Translated from:How AI Helps Maintain the Linux Kernel[1]

Author: Joab Jackson

Meet AUTOSEL, a Linux maintainer that helps keep the kernel stable.

AUTOSEL is a script that uses large language models (LLM) to accomplish its tasks.

It performs a thankless job that all kernel maintainers dread: porting patches.

Porting patches is a “very tedious and frustrating process” and “not scalable,” said NVIDIA distinguished software engineer Sasha Levin during his Open Source Summit[2] talk.The presentation[3] focused on how AI is beginning to be used in the Linux kernel community to help keep up with the daunting task of maintaining the Linux kernel.

While it may not yet be writing exciting new features for the open-source operating system kernel, AI excels at those mind-numbingly repetitive but still necessary tasks. In other words, AI is already making life easier for Linux kernel developers, helping to maintain the Linux Kernel Stable and LTS trees, Levin said.

Patch Review

As the largest open-source project in the world, the Linux kernel is constantly updated and upgraded… a lot.

This pace has remained constant: up to 10,000 new patches can enter the mainline kernel in a 10-week period.

Stable and long-term support (LTS) kernel maintainers typically review about 100 patches daily, every day, including weekends and holidays.

Only a handful, about 5-10, ultimately fit for porting.

Levin’s AUTOSEL[5] written in Rust first checks incoming submissions for similarities with past porting decisions and only suggests those that seem worthy of further review to human submitters.

AUTOSEL is built from multiple LLMs, each used for specific strengths, and for cross-validation to reduce errors and hallucinations.

For each submission, the tool creates a mathematical representation (or “embedding”) of the text, preserving the semantic meaning of each submission in the kernel’s history, making them easy to compare.

For human maintainers, the tool reduces the number of submissions they must review. It even explains its reasoning via email.

Understanding Your Tools

Like any other tool, the value of LLMs depends on the user’s understanding of them.

You can think of large language models[6] as the next-generation compiler, providing a leap in productivity for developers, Levin said. They act like “large state machines,” but their uniqueness lies in their probabilistic rather than deterministic state transitions.

They excel at matching patterns given a large number of parameters and user-provided inputs. The “temperature” parameter controls the probabilistic nature of the LLM, or how freely it interprets its material.

Other Uses

Like any other technology, LLMs are first tested on secondary tasks.

LLMs perform excellently in “small, well-defined tasks,” Levin said.

One such use is code generation and refactoring. Well-defined bug fixes or converting code to other forms (e.g., standard APIs) are good tasks.

For the 6.15 kernel version, Levin had the LLM write a patch that converted the open hash table implementation to a standard API.

Linux kernel 6.16 includes <span>git-resolve</span>. This tool can resolve incomplete or incorrect commit IDs, which is a headache for core developers[7], but occurs infrequently enough that it is not worth spending a lot of time manually writing a tool to figure out which incomplete SHA-1 actually connects to which commit.

Levin spent 20 minutes collaborating with the LLM to create that tool.

It would take an engineer about half a day to create such a tool, and given the relative rarity of the problem it solves, it is not worth the effort. Additionally, the LLM did a lot of extra work: it created a set of self-tests and even wrote documentation, which a human engineer would be reluctant to write even if they would.

Levin said there are endless cleanup tasks that can be accomplished in the kernel. LLMs can help non-native English speakers write descriptive commit messages.

CVE Classification

Another tedious task is classifying security vulnerabilities (CVE[8]), a task the Linux kernel community is undertaking in 2024.

This work involves reviewing submissions to see if they address security issues.

Initially, a set of “hacky Bash scripts” was written to assist.

LLMs were used to replace these scripts with a more refined set of tools written in Rust, which includes a complete suite of testing tools and documentation.

Starting with AUTOSEL, the CVE classifier uses LLMs to identify submissions that address security issues and then continues to check if the vulnerability has been resolved in previous patches. This is a daunting task for humans, given that the code comprising the Linux kernel[9] consists of 40 million lines.

LLMs can understand the semantic meaning of submissions, providing a more comprehensive matching capability. Retrieval-augmented generation[10] (RAG) loops extract the kernel’s development history and documentation (e.g., Git repositories) to minimize hallucinations.

Levin noted that LLMs effectively act as AI agents. They can run git commands[11] directly against the code repository, such as <span>git blame</span>[12], to learn from the history of kernel development itself.

In summary, so far, AI has helped Linux expand its maintenance work while enhancing consistency and reducing the manual labor of tedious tasks.

Reference Links

<span>[1]</span> How AI Helps Maintain the Linux Kernel:https://thenewstack.io/how-ai-helps-maintain-the-linux-kernel/<span>[2]</span>Open Source Summit:https://ossna2025.sched.com/?utm_source=the+new+stack&utm_medium=referral&utm_content=inline-mention&utm_campaign=tns+platform<span>[3]</span>Talk:https://ossna2025.sched.com/event/1zffD/ai-for-kernel-engineers-sasha-levin-nvidia<span>[4]</span>Presentation:https://lwn.net/Articles/1026558/<span>[5]</span>**AUTOSEL**:https://git.sr.ht/~sashal/autosel<span>[6]</span>Large Language Models:https://thenewstack.io/how-to-generate-ai-from-a-database-bruce-momjian/<span>[7]</span>A headache:https://lwn.net/Articles/1001526/<span>[8]</span>CVE:https://thenewstack.io/vulnerability-management-best-practices-for-patching-cves/<span>[9]</span>Linux Kernel:https://thenewstack.io/learning-linux-start-here/<span>[10]</span>Retrieval-Augmented Generation:https://thenewstack.io/navigating-the-nuances-of-graphrag-vs-rag/<span>[11]</span>git commands:https://thenewstack.io/how-to-make-git-a-developers-bff/<span>[12]</span>`git blame`:https://git-scm.com/docs/git-blame

Leave a Comment