Abstract
Recently, Gabriel Krisman Bertazi from SUSE submitted a set of RFC patches aimed at enhancing the performance of single-threaded tasks running on high-core-count CPUs (many-core CPUs). The core of the optimization lies in restructuring the <span><span>rss_stat</span></span> structure used in the kernel to record the Resident Set Size (RSS) of processes. The author noted that allocating per-CPU (pcpu) memory for <span><span>rss_stat</span></span> in systems with a large number of cores incurs significant overhead, especially in scenarios involving fork/exec or a large number of short-lived processes. This patch employs a hybrid strategy of “local counters + atomic counters” for single-threaded processes, avoiding the frequent overhead of remote updates, resulting in speedups of 6%–15% and approximately 1.5% in synthetic and real-world benchmark scenarios, respectively. The article analyzes the implementation approach, applicable scenarios, and potential trade-offs, and provides suggestions for future mainline integration and optimizations for EPYC/Threadripper.
Background: Why Modify <span><span>rss_stat</span></span>?
Modern servers and high-end desktops (e.g., AMD EPYC, Threadripper) have hundreds of CPU logical cores. In the Linux kernel, to reduce contention on shared counters, per-CPU counters (pcpu — one data structure per CPU) are commonly used to cache frequently updated statistics, which are then periodically merged (or aggregated during reads). <span><span>rss_stat</span></span> is the statistical structure used to record the RSS of a process or address space, which includes per-CPU counts for quick updates.
The issue is that on platforms with extremely high core counts, the cost of allocating pcpu memory for each object that requires <span><span>rss_stat</span></span> is not trivial, especially in workloads that frequently create and destroy processes (fork-intensive), where this overhead is magnified. Observed impacts include system time regression, affecting the performance of I/O/CPU mixed tools like Git.
Core Insight: Lightweight Approach for Single-Threaded Tasks
Identifying Targets: Single-Threaded vs Multi-Threaded
The key insight of the patch set is to distinguish between single-threaded processes and multi-threaded processes that frequently perform cross-CPU updates. For single-threaded processes, there is virtually no “remote update” scenario (i.e., other CPUs updating the RSS count of that process), so it is unnecessary to retain a complete per-CPU update path for such scenarios.
Implementation Approach
- For single-threaded objects, use a local counter as the normal update path: the process updates a local variable on its own CPU, avoiding triggering pcpu memory allocation and remote cache-line writes.
- For rare remote updates that do occur, use an atomic counter as a fallback path to ensure statistical correctness and concurrency safety.
- Simplify or delay the allocation of pcpu resources as needed, reducing allocation overhead for short-lived processes like fork/exec.
In short: make the “common path” extremely lightweight (local fast path) and handle the “rare path” with more expensive but correct atomic operations.
Details of the Patch Content
This patch series consists of 4 patches mainly containing the following:
- Introduction of a new generic counter API: A new
<span><span>include/linux/lazy_percpu_counter.h</span></span>file introduces the lazy per-CPU counter, designed to allow delayed initialization/upgrading of per-CPU counters. - Modification of the mm (memory-management) subsystem: The original per-CPU
<span><span>rss_stat</span></span>counter is replaced with this lazy-initialized counter. For single-threaded mm_struct (representing address space/process memory context), delay/avoid per-CPU memory allocation. Only upgrade to a full per-CPU counter when it is detected that the mm is shared by multiple threads (i.e., spawning a second thread/fork + thread scenario). - Separation of update paths: Updates to mm counters (e.g., when RSS changes) distinguish between the “fast/common path” (local/atomic updates) and the “slow/fallback path” (per-CPU updates + aggregation). This significantly reduces the overhead for ordinary single-threaded tasks.
- Adjustments to fork/exec/exit paths: Since the initialization/release of mm_struct does not necessarily require immediate allocation of per-CPU data, this reduces lock contention and memory allocation overhead during the creation/destruction processes like fork/exec.
The patch notes also mention that this new lazy-counter API, besides <span><span>rss_stat</span></span>, can also be used for other per-CPU statistical purposes — paving the way for future optimizations of various per-CPU statistics.
Effects: Gains in Synthetic and Real Benchmarks
The author provides two types of benchmark results in the article:
- Synthetic microbenchmark (artificially created, fork-intensive, e.g., looping calls to
<span><span>/bin/true</span></span>): On a 256-core system, wall-clock time decreased by 6%–15%, with specific values varying with the number of cores. This type of benchmark amplifies the overhead of pcpu allocation and remote updates, thus reflecting the optimization benefits more clearly. - More realistic benchmarks (e.g.,
<span><span>kernbench</span></span><code><span><span>)</span></span>: Showed an improvement of approximately 1.5% in elapsed time. Although the magnitude is smaller, such improvements are still worth pursuing in large server or latency-sensitive scenarios.
These data indicate that on highly parallel machines, micro-optimizations targeting single-threaded scenarios can translate into measurable system-level benefits, especially in workloads that create a large number of short-lived processes or high concurrency.
Applicable Scenarios and Impact Assessment
Suitable Scenarios
- Build/Continuous Integration Systems (CI/build farm): Significant pcpu allocation overhead when many short-lived processes (compilation, testing commands) run concurrently.
- Container-Intensive Hosts (container host): Creation/destruction of many lightweight processes.
- Desktop Environments and Applications Critical for Single-Thread Optimization: For example, certain I/O-intensive but single-threaded toolchain tasks.
- High-Core Workstations/Servers (EPYC, Threadripper): The more cores, the more pronounced the relative cost of pcpu allocation.
Risks and Trade-offs
- Increased Complexity: Introducing two sets of update paths (local + atomic + per-CPU fallback) requires careful verification of aggregation timing and consistency to prevent statistical bias or ABA-type issues.
- Adaptation Boundaries: How to accurately determine whether a task is “single-threaded” or “will perform remote updates”? The determination strategy must be cautious (e.g., changes in thread count, subsequent spawning of threads by processes, etc.).
- Long-term Maintenance Costs: Adding special branches to the kernel statistical subsystem may incur long-term maintenance burdens, requiring good documentation and regression testing.
Insight Analysis
Why is this Optimization Worthwhile?
On modern many-core platforms, the traditional strategy of “copying data structures to each CPU to avoid contention” incurs its own costs — pcpu memory management and initialization are no longer trivial at large scales. Kernel design has long balanced between “reducing lock contention” and “reducing remote writes,” and this patch reflects another dimension: using different data paths for different workload categories. This “workload-aware” lightweight strategy can significantly reduce path overhead while maintaining correctness.
Scalable Design Patterns
- On-Demand Degradation (optimistic fast-path + fallback): Use a faster but assumption-based implementation for the common path (e.g., local counter), and a more general but expensive fallback for uncommon cases (e.g., atomic++ / per-CPU + aggregation).
- Workload Classification: Kernel subsystems can consider typical workloads (single-threaded short-lived vs long-lived multi-threaded) more in their design to adopt different resource allocation strategies.
- Delayed Resource Allocation: Delaying the allocation of expensive per-CPU resources on demand can significantly reduce the overhead for short-lived objects.
Feasibility and Recommendations for Upstream Mainline
- If this RFC receives extensive testing (especially on platforms like AMD EPYC/Threadripper and common CI/build systems), it has high value for mainline integration.
- It is recommended to increase the coverage of regression tests and benchmarks (including real workloads with different core counts).
- Additionally, the logic for determining single-threaded status should be clarified, or support for rapidly switching from “local only” to “per-CPU + aggregation” strategies at runtime should be provided (e.g., enabling the full path when cross-CPU updates are detected).
- Clearly document design assumptions and boundary conditions in commit messages and code to facilitate future maintenance.
Conclusion
This patch series represents a fine-grained optimization of the Linux kernel in the face of the many-core and heterogeneous workload trends. It does not pursue major architectural changes but instead utilizes per-CPU statistics + lazy initialization + on-demand upgrades strategies to create lightweight paths for single-threaded/short-lived scenarios, thereby reducing unnecessary overhead. Such optimizations have real significance for modern multi-core servers and high-concurrency short-task environments.
At the same time, it reminds us that:kernel design cannot assume all processes are “multi-threaded and long-lived”. There should be dedicated optimization paths for lightweight, short-lived, single-threaded tasks. Similar ideas — optimizing paths based on workload — may become increasingly important in future kernel subsystems (memory/scheduler/accounting/trace).
Reference Link:
https://lore.kernel.org/all/20230608111408.s2minsenlcjow7q3@quack3/