Why the C++ STL Does Not Include a Built-in Thread Pool? Exploring the Design Philosophy of the Standard Library and the Evolution of Concurrency Click the blue text to follow the author

1. Introduction

In the era of multi-core processors, to efficiently utilize computing resources, the thread pool has become an important concurrency model, suitable for server-side applications, high-performance computing, and various scenarios that need to handle a large number of concurrent tasks. By pre-creating and managing a set of threads, it avoids the overhead of frequently creating and destroying threads, thereby improving system performance and response speed.

So why doesn’t the C++ Standard Library (Standard Template Library, STL) provide a built-in thread pool implementation? Currently, using a thread pool relies on third-party libraries such as Boost.Asio and Intel TBB, or requires implementing one from scratch.

This article explores the deeper reasons why the C++ Standard Library does not include a thread pool.

2. The Design Philosophy of C++

To understand why the C++ Standard Library does not have a built-in thread pool, we need to delve into the design philosophy of the C++ language and its standard library. C++, as a system-level programming language, has one of its core principles as providing powerful low-level control capabilities, adhering to the principle of “zero-cost abstraction.” The introduction of language features and library components must not impose additional runtime or memory overhead.

The design goal of the C++ Standard Library (STL) is to provide a set of basic, general building blocks, rather than high-level encapsulations tailored for specific application scenarios. C++ tends to provide a series of “tools” that developers can flexibly combine and build the desired “products.”

In the field of concurrency, C++11 introduced std::thread for creating and managing individual threads, std::mutex for mutually exclusive access to shared resources, std::condition_variable for condition waiting and notification between threads, and std::future and std::promise for asynchronous result passing. These are low-level concurrency primitives that do not constitute a complete thread pool but are the foundation for building any complex concurrency model (including thread pools).
They provide developers with great flexibility and control. Developers can precisely control the lifecycle of threads, synchronization mechanisms, and task scheduling strategies, allowing for extreme performance optimization or unique behavior tailored to specific application scenarios. If the standard library provided a fixed thread pool implementation, it would not meet the needs of all scenarios, and inherent design choices would introduce unnecessary overhead or limitations.

C++ largely inherits the characteristics of the C language: pursuing direct control over hardware and extreme runtime efficiency. This obsession with performance and control makes the C++ Standard Library very cautious when introducing new features, especially those with high abstraction levels, uncertain performance, or that limit flexibility. The thread pool, as a higher-level abstraction involving complex scheduling and resource management, has various implementation methods and potential performance differences, making it very difficult to standardize as a universal component.

The complexity and diversity of thread pool implementations are more suitable as upper-level applications built on these foundational tools rather than being directly included in the core of the standard library.

3. The Complexity and Diversity of Thread Pool Implementations

A “thread pool” is not a single, fixed concept, but a pattern that encompasses various design choices and trade-offs. There is no “one-size-fits-all” thread pool model that can perfectly adapt to all application scenarios, making it very difficult to standardize a universal thread pool.

A typical thread pool consists of a collection of threads, a task queue, and a task scheduling mechanism. The specific implementation of these components varies widely, and each choice can have different impacts on performance, resource consumption, and behavior. It is challenging for the standards committee to select a single design as the official standard.

The design of a thread pool must be highly dependent on the application scenarios and task characteristics it serves.

Task Types:

CPU-bound tasks: Tasks primarily perform computations and should utilize as many CPU cores as possible. In this case, the size of the thread pool is related to the number of CPU cores, avoiding excessive context switching overhead.
I/O-bound tasks: Tasks primarily wait for external resources. These tasks release the CPU while waiting, so the thread pool can accommodate more threads, increasing throughput.
Mixed tasks: Tasks that involve both computation and I/O. Balancing the number of threads to maximize efficiency is a complex issue.

Thread Count Management:

Fixed-size thread pool: The number of threads is determined at creation, which is simple to implement but cannot adapt to load changes.
Dynamic thread pool: Dynamically adjusts the number of threads based on factors such as the length of the task queue and system load. However, this adds complexity to the implementation (how to determine when to increase/decrease threads, how to safely destroy threads?), with the only benefit being better adaptability.
Determining the optimal number of threads for a thread pool is itself a challenge, requiring benchmarking and tuning based on specific applications.

Task Queue:

Bounded queue: The queue has a limited capacity. When the queue is full, how to handle new submitted tasks? Should it block the submitter, reject the task, or discard the task? Each strategy has different applicable scenarios and pros and cons.
Unbounded queue: Theoretically grows infinitely. Simplifies task submission, but in reality, can we have infinite memory?
Task scheduling strategy: The order in which tasks are taken from the queue for execution. Common strategies include first-in-first-out (FIFO), last-in-first-out (LIFO), and priority-based scheduling.

Exception Handling: How to handle uncaught exceptions thrown by tasks in the thread pool? Should it terminate the entire thread pool, only terminate that thread and recreate it, or report the exception to the submitter?

Task Cancellation Mechanism: How to cancel a task that has been submitted but not yet executed? How to interrupt a task that is currently executing? This requires the task itself to support cooperative cancellation points, increasing the complexity of task implementation.

thread_local variable issues: This is a significant technical barrier. The lifecycle of thread_local variables is bound to threads. If threads in the thread pool are reused to execute different tasks, the state of thread_local variables may “leak” between tasks, or their destructors may not be called when tasks end, which contradicts the expected behavior of thread_local variables. For example, if a task stores resources in a thread_local variable, when the thread is reused to execute the next task, that resource may not be released in time. Microsoft encountered issues with the destructor of thread_local variables not conforming to standards when implementing std::async.

Why the C++ STL Does Not Include a Built-in Thread Pool? Exploring the Design Philosophy of the Standard Library and the Evolution of Concurrency

4. The Consensus Dilemma in the Standardization Process

The C++ Standards Committee (ISO) considers multiple factors when deciding whether to include a feature in the standard library, including generality, stability, impact on existing code, and potential for future evolution.

The implementation details of thread pools vary widely, and there is no “best” solution that can meet all needs. It is very difficult for committee members to reach a consensus on a universal thread pool design.

Different members have different preferences regarding thread management strategies, task queue types, exception handling methods, and task cancellation mechanisms based on their experiences and application scenarios.

The goal of the standard library is to provide general, widely usable components. A thread pool that is too generic may be inefficient due to a lack of specificity, while a thread pool optimized for specific scenarios may lose its generality.

The C++ Standards Committee adopts a cautious approach when introducing new features, avoiding “over-design” or leading to “library bloat.”

The standard library consists of core or foundational features of the language that are widely applicable. Although thread pools are common, as a high-level concurrency model, they are closer to application-level or domain-specific library functions rather than being foundational to the language or standard library.
Once a feature is included in the standard, it must be maintained long-term to ensure backward compatibility. A complex thread pool with multiple variants would have high maintenance costs and significant limitations for future evolution. The committee certainly does not want to introduce a component that may become a burden in the future.

The C++ Standards Committee has also attempted higher-level abstractions in the concurrency domain. C++11 introduced std::async, which can asynchronously execute a function and return a std::future. The implementation of std::async may use an internal thread pool (or similar mechanism). However, std::async has also exposed some issues in practical use:

launch strategy uncertainty:std::async‘s launch strategy (std::launch::async or std::launch::deferred) defaults to std::launch::async | std::launch::deferred, allowing runtime to decide whether to start a new thread immediately or delay execution until get() or wait() is called. It is difficult to predict when a task will actually start executing.
Previously mentioned thread_local variable lifecycle issues: If std::async internally uses a thread pool and threads are reused, the destruction timing of thread_local variables may not meet expectations, leading to resource leaks or abnormal behavior. This has indeed been a problem in some compiler implementations.

Standardizing a thread pool not only has to overcome technical complexities but also requires reaching a consensus within the committee, learning from past experiences. Rather than forcibly standardizing a potentially unsatisfactory thread pool, it is better to leave the choice and flexibility to developers to select or implement the most suitable solution.

5. The Evolution of the C++ Concurrency Model

C++11 is a milestone in C++ concurrent programming. It officially incorporates multi-threading support into the standard library, providing a series of core concurrency primitives that lay the foundation for building various concurrent structures:

std::thread: Creates and manages independent execution threads. This is the most basic unit of concurrency.
std::mutex, std::recursive_mutex, std::timed_mutex, etc.: Provide mechanisms for mutually exclusive access to shared resources, preventing data races.
std::condition_variable: Waiting and notification between threads, implementing complex synchronization logic.
std::future and std::promise: Provide a mechanism for asynchronously obtaining operation results, allowing tasks to execute in another thread and retrieve results through future upon completion.
std::atomic: Atomic operations ensure that reads and writes to shared variables are uninterruptible, allowing simple data synchronization without mutexes.

These primitives provide fine-grained control and minimal overhead, essential “LEGO blocks” for building more complex concurrent structures like thread pools.

C++11 also introduced the std::async function template, providing a simpler way to execute tasks asynchronously and return a std::future. A key feature of std::async is that it may internally use a thread pool, thereby hiding some of the thread management details. As mentioned earlier, std::async also has some issues.

As the C++ language continues to evolve, the focus of the standards committee has gradually shifted from direct thread management to higher-level “tasks” and “execution models.” The core idea of this shift is to focus on how to define and compose tasks rather than how to manage underlying threads.

Parallel Algorithms (C++17): C++17 introduced parallel versions of STL algorithms, providing execution policies (std::execution::par) that instruct the compiler and runtime library to automatically parallelize these algorithms without manually creating and managing threads. This is a very high-level concurrency abstraction, completely delegating thread management and scheduling to the library implementation.
std::jthread (C++20): C++20 introduced std::jthread, a “joinable thread.” jthread automatically calls join() in its destructor, avoiding the common issue of “forgetting to join a std::thread leading to program crashes.” It also supports cooperative interruption, providing a more elegant way to manage threads. Although jthread is still an abstraction for a single thread, it marks the standard library’s efforts to provide safer and easier-to-use concurrency tools.

The future direction of the C++ concurrency model includes:

Executors: This is an important feature currently being actively discussed and proposed by the C++ Standards Committee. Executors provide a unified interface for submitting tasks to different execution contexts. These execution contexts can be thread pools, single threads, asynchronous I/O event loops, GPUs, etc. By using executors, the task logic is decoupled from the execution method, enabling more flexible and portable concurrent code. A standard executor framework can build thread pools that conform to standard interfaces, making it easy to switch between different thread pool implementations.
Coroutines (C++20): Coroutines, introduced in C++20, are another powerful concurrency feature. Functions can pause and resume during execution, enabling non-blocking asynchronous programming. Coroutines themselves are not a concurrency mechanism, but when combined with executors, they can achieve very efficient concurrent and asynchronous operations. Coroutines can effectively utilize thread pools by scheduling their execution on threads within the thread pool, avoiding the overhead of thread context switching.

The evolution of the C++ standard library’s concurrency model is a gradual process from low-level primitives to higher-level abstractions. It tends to provide powerful tools and flexible mechanisms for building concurrent solutions rather than a single, fixed “finished product” thread pool.

6. Solution Choices

Based on the concurrency primitives provided by the C++ standard library (std::thread, std::mutex, std::condition_variable, std::queue, etc.), implementing a thread pool from scratch is a common choice.

Advantages: Complete control over the behavior of the thread pool, highly customizable, allowing seamless integration with other parts of the application for optimal performance.
Challenges: Implementing a robust, efficient, deadlock-free, and data race-free thread pool is quite complex, especially when dealing with dynamic thread scaling, task cancellation, exception safety, and thread_local variable issues.

There are many mature, proven third-party libraries and frameworks with thread pool implementations that have been validated through extensive practice, reducing the risk of errors.

Boost.Asio: The Boost library is the de facto standard extension for C++, and its Boost.Asio is a cross-platform asynchronous I/O library that provides a powerful I/O service (io_context or io_service) that can be combined with a thread pool to handle asynchronous operations. Although it is not a direct “thread pool” library, the work queue and executor model can be conveniently used to build a thread pool.
TBB (Threading Building Blocks): Intel’s TBB is a C++ template library for parallel programming that provides advanced parallel algorithms and data structures, internally managing a thread pool to execute parallel tasks. TBB focuses on task parallelism rather than the typical thread pool task submission model, but its internal mechanisms are quite similar to those of a thread pool.
folly (Facebook Open-source Library): The folly library, open-sourced by Facebook, includes a series of high-performance C++ components, including a rich and efficient thread pool implementation.
Qt Concurrent: The Qt framework provides a module called Qt Concurrent for writing multi-threaded applications, which also uses a thread pool to manage the parallel execution of tasks.
PPL and Concurrency Runtime (Microsoft): Provides advanced parallel programming models, including task parallelism and data parallelism, relying on efficient thread management mechanisms.

7. Conclusion

Although the C++ Standard Library does not provide an out-of-the-box thread pool, this is not a flaw.

The thread pool, as a high-level concurrency model, has its best implementation highly dependent on specific application scenarios and task characteristics. A standardized universal thread pool would be unable to meet the needs of all scenarios due to inherent policy choices, potentially introducing unnecessary overhead or limitations, which contradicts the core spirit of C++.

Rather than forcibly standardizing a potentially unsatisfactory thread pool, it is better to leave the choice and flexibility to developers.

Previous tutorials

C++ Training Camp: Empowering Growth, Making You No Longer Just Theoretical!

C++

STL Algorithms: Master These, and Your Code Efficiency Will Increase Tenfold!

Building

Using Google Test for C++ Unit Testing

Applications

Thoroughly Understand Asynchronous and Multithreading: Concepts, Principles, and Application Scenario Comparisons

Why the C++ STL Does Not Include a Built-in Thread Pool? Exploring the Design Philosophy of the Standard Library and the Evolution of Concurrency Lion Welcome to follow my public account for learning technology or submissions