Essential Course on Performance Optimization: In-Depth Analysis of Embedded Linux CPU Scheduling

In embedded Linux systems, CPU scheduling directly determines system performance, task response speed, and real-time performance. Especially in resource-constrained scenarios such as industrial, communication, automotive, and smart devices, reasonable configuration and optimization of CPU scheduling are core aspects of enhancing embedded system performance. This article will provide a complete guide to CPU scheduling optimization for embedded developers, covering CPU scheduling principles、types of scheduling strategies、multi-core optimization、real-time task optimization、scheduling tools and practical applications to case studies and optimization methodologies.

1. Overview of CPU Scheduling Principles

1. Role of the Scheduler

The CPU scheduler is one of the core modules of the Linux kernel, responsible for allocating CPU time among multiple processes or threads, ensuring timely system responses and efficient resource utilization.

• Goals：

• High CPU utilization

• Task fairness

• Guarantee of real-time performance

• Low latency

• Concerns for Embedded Systems：

• Limited number of CPU cores, significant multi-task contention

• Task response latency directly affects device stability

• Real-time tasks need to be prioritized for execution

2. Core Components of the Scheduler

The main scheduler in the embedded Linux kernel is CFS (Completely Fair Scheduler), suitable for ordinary tasks; it also supports real-time schedulers (RT Scheduler), including SCHED_FIFO and SCHED_RR.

Core data structures of CFS:

• runqueue: the run queue for each CPU

• task_struct: process/thread control block

• vruntime: virtual run time used for fair scheduling

+--------------------+         +--------------------+
|   CPU Core 0       |         |   CPU Core 1       |
|  runqueue0         |         |  runqueue1         |
|  task1, task2...   |         |  task3, task4...   |
+--------------------+         +--------------------+
            \                         /
             \                       /
              \                     /
               \                   /
                \-----------------/
                       CFS Scheduler

The kernel polls the runqueue of each CPU core through the scheduler, fairly distributing tasks while considering task priority and real-time policies.

2. Types of Scheduling Strategies

1. Normal Scheduling (CFS)

• SCHED_NORMAL / SCHED_OTHER：

• Default strategy, suitable for general applications

• Sorted by vruntime using a red-black tree to ensure fairness

• Characteristics：

• Fair allocation of CPU time

• Automatic load balancing

• May delay response for real-time tasks

Flowchart：

New task joins runqueue
        |
        v
Calculate vruntime
        |
        v
Insert into red-black tree
        |
        v
Select task with minimum vruntime for execution

2. Real-Time Scheduling

• SCHED_FIFO (First-Come, First-Served)：

• Tasks are executed based on priority

• Tasks with the same priority are executed in the order they arrive

• SCHED_RR (Round Robin)：

• Tasks with the same priority are executed in a time-slice round-robin manner

Characteristics of Real-Time Scheduling：

Feature	SCHED_FIFO	SCHED_RR
Task Preemption	High	High
Time Slice	None	Fixed
Priority Response	Fast	Fast
Applicable Scenarios	High-priority tasks	High-priority round-robin tasks

3. Principles for Choosing Scheduling Strategies

• Normal background tasks → CFS

• Delay-sensitive or high-priority tasks → SCHED_FIFO

• Multiple real-time tasks → SCHED_RR

3. Multi-Core Optimization

In embedded multi-core systems, effectively utilizing each core is a key focus for performance optimization.

1. CPU Affinity

• Binding tasks to specific cores to reduce cache misses

• API Example:

cpu_set_t mask;
CPU_ZERO(&mask);
CPU_SET(1, &mask); // Bind task to CPU1
sched_setaffinity(0, sizeof(mask), &mask);

2. CPU Isolation (isolcpus)

• Isolating critical cores to prevent ordinary tasks from preempting

• Kernel boot parameters:

isolcpus=2,3 nohz_full=2,3 rcu_nocbs=2,3

Flowchart：

+---------------------+
| CPU Core 0          | Ordinary Task
| CPU Core 1          | Ordinary Task
+---------------------+
| CPU Core 2          | Critical Task (Isolated)
| CPU Core 3          | Critical Task (Isolated)
+---------------------+

3. Load Balancing Optimization

• Avoiding hotspot tasks concentrating on a single core

• Adjusting scheduler parameters to improve performance

• Using <span>taskset</span> tool to verify task distribution

4. Real-Time Task Optimization

Embedded systems often have hard real-time or soft real-time tasks.

1. Priority Design

• High-priority tasks → Real-time scheduling strategy

• Medium and low-priority tasks → Normal strategy

• Avoiding priority inversion

2. Avoiding Blocking Operations

• Blocking can delay high-priority tasks

• Recommendations:

• Use non-blocking I/O

• Consider priority inheritance when using locks

3. Task Switching Optimization

• Reducing the frequency of context switches

• Prioritize using CPU pinning

5. Scheduling Tools and Analysis

1. perf

• Function: CPU usage, hotspot function analysis

• Example:

perf top
perf record -g ./your_app

2. ftrace

• Function: Function call tracing

• Kernel command:

echo function > /sys/kernel/debug/tracing/current_tracer
cat /sys/kernel/debug/tracing/trace

3. taskset and sched_setaffinity

• Function: Binding tasks to specific CPUs

• Practical example:

taskset -c 2 ./your_app

6. Performance Optimization Case Studies

Case Study 1: Multi-Core Video Processing Platform

• Problem: High CPU usage, frame loss

• Optimization:

• Binding video processing tasks to isolated cores

• Real-time scheduling strategy SCHED_FIFO

• Effect: Latency reduced by 30%, frame rate stabilized

Case Study 2: Embedded Communication Gateway

• Problem: Slow response due to multi-task preemption

• Optimization:

• Adjusting CPU affinity

• Isolating critical task cores

• Effect: Significant reduction in real-time task latency, improved system stability

7. Scheduling Optimization Methodology

1. Analyze the Current Situation：

• Using tools like perf, ftrace, htop, etc.

Strategy Selection：

• Ordinary tasks → CFS

• Real-time tasks → SCHED_FIFO / SCHED_RR

Multi-Core Planning：

• CPU affinity + isolcpus

Priority Design：

• Reasonable layering of high-priority and low-priority tasks

Iterative Validation：

• Testing latency and CPU usage before and after optimization, adjusting parameters

Flowchart：

Performance Analysis --> Strategy Selection --> Core Planning --> Priority Optimization --> Tool Validation --> Continuous Iteration

8. Conclusion

Through this article, you have mastered:

• The principles of the Linux kernel CPU scheduler and the CFS mechanism

• Various scheduling strategies and their application scenarios

• Multi-core optimization, CPU affinity, and core isolation methods

• Real-time task optimization techniques

• Scheduling performance analysis tools and practices

• Methodology for scheduling optimization in embedded systems

CPU scheduling optimization is fundamental to enhancing embedded performance. By combining tools, parameter tuning, and multi-core optimization, significant improvements in system response speed and stability can be achieved, laying a solid foundation for subsequent I/O, memory, and overall performance tuning.