Click the blue text to follow us
Author | daydreamer
In internet services, C++ is often used to build high-performance, high-concurrency, high-traffic, and low-latency backend services. How to allocate memory reasonably to meet the high-performance requirements of the system is a frequent and important topic, and due to the characteristics of memory itself and the complexity of practical problems, many challenges arise.
We can classify memory in various ways, based on the size of memory allocation:
-
Small object allocation: Memory allocation less than 4 times the page size, under a 4KiB page size, <16KiB is considered small object allocation;
-
Large object allocation: Memory allocation greater than or equal to 4 times the page size, under a 4KiB page size, >=16KiB is considered large object allocation.
From the duration of memory holding:
-
Memory allocation and release within a single backend request or even shorter time
-
Memory holding and updating within any time window
-
Memory holding and updating nearly as long as the application process
-
Memory holding and releasing that remains meaningful after a process terminates for a period of time
Of course, further classification can also be based on memory allocation and release frequency, read and write frequency.
Memory management serves application systems, aiming to assist the system in better solving bottleneck issues. For example, regarding ‘how to reduce backend response latency and improve stability’, memory management may need to consider:
-
Handling memory read and write concurrency (frequent reads or frequent writes) to reduce response time and CPU consumption
-
Pooling and reusing memory at the application layer
-
Memory block size requested from the system and memory fragmentation
Each of these issues could be a significant topic on its own. This article serves as the opening of a series titled ‘Exploring C++ Memory Management’, introducing the theoretical foundation of memory management in Linux C++ programs. In subsequent articles, we will continue to unveil the implementation principles of commonly used memory management libraries in C++, including ptmalloc, jemalloc, tcmalloc, etc., explaining how popular memory allocators manage memory in C++ programs. Understanding the principles of memory allocators will help engineers reduce the cost of handling memory usage issues in practice and tailor the application-level memory management system to the system’s needs.
1. Linux Memory Management
GEEK TALK
Linux can roughly be divided from the bottom up into:
-
Hardware (Physical Hardware)
-
Kernel Space
-
User Space
△Figure 1: Linux Structure
Kernel modules run in kernel space, while applications run in user space, with non-overlapping memory address spaces. This method ensures that applications running in user space have a consistent view of the hardware, independent of the hardware platform. User space interacts with kernel services in a controlled manner through system calls, such as trapping into kernel mode and handling page faults.
The Linux memory management system can be roughly divided from the bottom up as follows:
-
Kernel-level memory management: In the Linux kernel, memory is managed through memory allocation functions:
-
kmalloc()/__get_free_pages(): Allocate smaller memory (kmalloc() in bytes, __get_free_pages() in pages of 128K), the allocated memory is located in the mapped area of physical memory and is physically contiguous, with a fixed offset from the actual physical address.
-
vmalloc(): Allocate larger memory, providing a contiguous memory area in virtual memory space, but not guaranteeing physical memory continuity, with overhead far greater than __get_free_pages(), requiring a new page table to be established.
-
User-level memory management: Implements commonly used memory management interfaces (malloc, free, realloc, calloc) through system call functions (brk, mmap, etc.); classic memory management libraries include ptmalloc2, tcmalloc, jemalloc.
-
Applications allocate memory through memory management libraries or directly call system memory management functions, using them according to the characteristics of the application itself, such as: memory allocation and release for single variables, memory pooling and reuse, etc.
At this point, a single process can run smoothly using the memory divisions provided by Linux. From the perspective of user programs, the memory model of Linux processes is roughly as follows:
△Figure 2: Linux Process Memory Model
-
Stack: Stores local variables and function parameters during program execution, growing from high addresses to low addresses
-
Heap: Dynamic memory allocation area, managed through functions such as malloc, new, free, and delete
The standard C library provides malloc/free functions to allocate and release memory, which are fundamentally implemented based on system calls like brk/mmap. Referring to Figure 2:
-
brk(): Used to allocate and release small memory. The end of the data segment, the start of heap memory, is called brk (program break). By setting the end address of the heap, the address can be moved up or down to expand or shrink heap memory. Low address memory must be released after high address memory is released; the low address marked as free cannot be merged. If subsequent memory requests exceed this free area, it will become a memory hole. By default, when the free memory at the highest address space exceeds 128K (which can be adjusted by the M_TRIM_THRESHOLD option), memory compaction (trim) is performed.
-
mmap(): Used to allocate large memory. mmap (memory map) is a method of memory-mapping files, mapping a file or other object into the process’s virtual address space (the Memory Mapping Segment between the heap and stack), creating a one-to-one mapping relationship between file disk addresses and a segment of virtual addresses in the process’s virtual address space. After establishing this mapping relationship, the process can read and write this segment of memory using pointers, while the system automatically writes back dirty pages to the corresponding file disk, and modifications to this area in kernel space directly reflect in user space, enabling file sharing between different processes. Memory larger than 128 K is allocated using the mmap() system call. Unlike brk() memory allocation, memory allocated by mmap() can be released independently.
-
munmap(): Releases the memory space created by mmap().
However, for multiple concurrently running processes, the system still needs to handle limited physical memory and increasing memory addresses. So, when Linux has multiple concurrently running processes, what processes does a single memory allocation go through? The main process of memory allocation in modern Linux systems is as follows:
-
Applications call memory allocation functions, using system calls brk or mmap to allocate virtual memory address space.
-
The mapping process from virtual memory to physical memory is handled by requesting the MMU allocation unit, calculating which page the virtual address belongs to, and then calculating the physical address of the page table entry based on the starting address of the page mapping table. The content of this table entry is searched in the TLB (Translation Lookaside Buffer); if the table entry is not in the TLB, its content is loaded from memory into the TLB.
△Figure 3: Linux Memory Allocation Mechanism (Virtual + Physical Mapping)
In the memory allocation process, further analysis involves tools:
-
Virtual Memory: A technology commonly used by modern operating systems, where each process has its own independent logical address space, and memory is divided into equal-sized blocks called pages. Each page is a segment of contiguous addresses corresponding to a block in physical memory called a page frame, usually with equal sizes for pages and page frames. Virtual memory allows multiple virtual pages to share the same physical page while isolating the kernel and user processes, as well as different user processes.
-
MMU (Memory-Management Unit): The memory management unit is responsible for managing the mapping from virtual addresses to physical addresses, ensuring that each user process has its own independent address space, providing hardware mechanisms for memory access permission checks, and protecting the memory used by each process from being destroyed by other processes.
-
PageTable: A storage unit for the mapping relationship between virtual memory and physical memory pages.
-
TLB (Translation Lookaside Buffer): A high-speed cache for virtual address mapping, primarily to improve the efficiency of MMU address mapping processing by adding a caching mechanism. If present, it can directly retrieve the mapped address for use.
It is important to mention a significant concept: memory lazy allocation, where physical mapping for an address is only established when the address is actually accessed. This is one of the fundamental ideas of Linux memory management. When the user requests memory, the Linux kernel only allocates virtual memory, not actual physical memory. When the user first uses this memory, the kernel triggers a page fault, allocates physical memory, and establishes the mapping relationship between virtual and physical memory. When a process experiences a page fault, it enters kernel mode and performs the following operations:
-
Check if the virtual address to be accessed is valid
-
Find/allocate a physical page
-
Fill in the physical page content
-
Establish the mapping relationship (from virtual address to physical address)
-
Re-execute the instruction that triggered the page fault
If the process of filling the physical page requires reading from disk, this page fault is a major fault (majflt); otherwise, it is a minor fault (minflt). We need to pay special attention to the value of majflt, as it can severely impact performance; a random read from disk takes several milliseconds, while minflt only significantly affects performance when occurring in large quantities.
2. Summary
GEEK TALK
Through the introduction of Linux memory management, we can see the issues that memory management needs to address:
-
Using the limited interfaces provided by the system to operate virtual memory read and write
-
Weighing the costs of allocating large memory at once versus allocating smaller memory multiple times: controlling page faults (especially majflt) vs. excessive memory usage by processes
-
Reducing memory fragmentation
-
Reducing the additional overhead brought by memory management libraries themselves
In the upcoming articles, we will further discuss classic memory management libraries such as ptmalloc, jemalloc, and tcmalloc, exploring their implementation principles for commonly used memory management in C++ programs.
END
#Previous Recommendations #
Starry Sky Project · The third wave (technical articles launched) event is now open! The first issue of ‘The Art of Code’ series discusses software engineering capabilities. The second issue of ‘The Art of Code’ series covers the art of coding.