Hello, everyone, I am Programmer MM.
This article is about 2700 words long. Today, I took a day off and finished reading Chapter 8 of ‘Linux Device Drivers (3rd Edition)’ – Memory Allocation. This chapter systematically explains various mechanisms of kernel space memory management, which is core knowledge that driver developers must master, directly affecting the correctness, performance, and stability of drivers.
Follow our public account to obtain e-books related to Linux (including ‘Linux Device Drivers’) and commonly used development tools. A document list is provided at the end of this article.
1. Basics of Memory Allocation
The functions used for memory allocation are kmalloc and kfree.
kmalloc is the most commonly used function for allocating small blocks of memory in the kernel, and its behavior is similar to the user-space malloc, but with key differences.
Function prototype:
#include <linux/slab.h>void *kmalloc(size_t size, int flags);Core features>>>
Physically contiguous: The allocated memory area is contiguous in physical memory, which is crucial for Direct Memory Access (DMA).
Retain original value: The memory content is not automatically zeroed after allocation.
Size limit: There is an upper limit on allocation; for portability, memory larger than 128KB should not be allocated. Internally, a “slab allocator” predefines a series of fixed-size memory pools to handle requests, so the actual allocated memory may be slightly more than the requested value.
Key parameters: Allocation flags (flags)
The flags determine how the kernel interacts during allocation, which is key to correctly using kmalloc.
Common flags
Application scenarios and descriptions
GFP_KERNEL // General use. Allocation may block (sleep), waiting for the kernel to free memory. Can only be used in process context (e.g., within system calls), and the function must be reentrant. GFP_ATOMIC // Used in atomic context. For interrupt handlers, tasklets, holding spinlocks, etc., where sleeping is not allowed. Never sleeps, but may fail due to insufficient memory. The kernel reserves some memory for its use. GFP_DMA // Requests memory from the segment available for DMA (usually the first 16MB of physical memory) to ensure that peripherals can access it directly.Memory release: Use void kfree(const void *ptr); to free memory allocated by kmalloc.
2. Page Allocation: get_free_page and alloc_pages
When larger blocks (one page or multiple pages) of contiguous physical memory need to be allocated, page-oriented allocation functions are a more efficient choice.
Core functions:
get_zeroed_page(unsigned int flags) // Allocates a page and zeroes it. __get_free_page(unsigned int flags) // Allocates a page but does not zero it. __get_free_pages(unsigned int flags, unsigned int order) // Allocates 2^order contiguous pages, returning a pointer to the first byte, not zeroed. alloc_pages // This series of functions returns a pointer to struct page rather than a virtual address. This is particularly useful when dealing with high memory (physical memory that the kernel cannot directly map on 32-bit systems). struct page *alloc_pages(unsigned int flags, unsigned int order); __free_page(struct page *page); // Used for freeing3. High Memory and vmalloc
High Memory: On 32-bit architectures, the kernel’s virtual address space is limited (usually 1GB) and cannot permanently map all physical memory. Physical memory beyond the direct mapping range is called high memory, which needs to be allocated using alloc_pages with the __GFP_HIGHMEM flag and accessed by temporarily mapping it into the kernel address space using functions like kmap. 64-bit systems typically do not have this limitation.
vmalloc: void *vmalloc(unsigned long size); Allocates a segment of virtual address space that is contiguous but may not be contiguous in physical address.
Characteristics and overhead: Requires a dedicated page table, and the allocation overhead is larger than kmalloc and page allocators.
Application scenarios: Suitable for very large contiguous virtual memory buffers used only by software (e.g., during module loading). Cannot be used for DMA operations, as DMA typically requires contiguous physical addresses.
Release: Use vfree(void *addr).
ioremap: Similar to vmalloc, it also establishes a new page table but does not actually allocate physical memory. It is specifically used to map a device’s physical memory address range into the kernel’s virtual address space for driver access to device registers or memory buffers. Use iounmap to unmap.
4. Specialized Mechanisms: Slab Caches, Memory Pools, and Per-CPU Variables
1. Slab Allocator
When a driver needs to frequently allocate and free a large number of objects of the same size, using the Slab allocator can avoid the overhead and fragmentation caused by repeatedly requesting and releasing memory.
// Create cache kmem_cache_t *kmem_cache_create(const char *name, size_t size, size_t align, unsigned long flags, ...); // Allocate object void *kmem_cache_alloc(kmem_cache_t *cache, int flags); // Free object and cache void kmem_cache_free(struct kmem_cache *cachep, void *objp); void kmem_cache_destroy(struct kmem_cache *cachep);2. Memory Pool (mempool)
A memory pool (mempool) is a mechanism provided by the Linux kernel that ensures successful memory allocation in emergencies. Its core idea is to pre-allocate a certain number of memory objects and keep them in a “pool.” When the system cannot allocate memory through normal means (like kmalloc) due to insufficient memory, the driver can obtain a pre-allocated object from this pool, ensuring successful allocation.
The main design goal is to provide a layer of assurance for memory allocation in code paths where allocation failures are not allowed (e.g., in interrupt contexts, holding spinlocks, or to ensure the integrity of specific core data structures).
The memory pool works through two key components: a pre-allocated object pool and an allocation strategy that provides backup assurance in emergencies.
Basic data structure: The memory pool is represented by the mempool_t structure, created using the mempool_create function.
Pre-allocation: When creating a memory pool, the kernel will use the allocation function you provide (e.g., mempool_alloc_slab) to pre-allocate a specified number (min_nr) of objects. These objects constitute the “backup force” of the memory pool.
Allocation strategy: When you call mempool_alloc to attempt to allocate an object, the memory pool operates in the following order:
First, it tries to allocate memory from the system using the allocation function you specified when creating it (e.g., based on slab mempool_alloc_slab).
If the above allocation fails, the memory pool will take an object from its pre-allocated pool for you.
Release strategy: When you call mempool_free to release an object:
If the current number of objects in the memory pool is less than its minimum number (min_nr), this object will be returned to the pre-allocated list of the memory pool.
If the number of objects in the memory pool has reached or exceeded min_nr, this object will be truly returned to the system using the release function you specified when creating it (e.g., mempool_free_slab).
Using a memory pool mainly involves creating, allocating, releasing objects, and destroying.
Creating a memory pool
Typically, we create a memory pool based on slab cache for higher efficiency. mempool_alloc_slab and mempool_free_slab are kernel-provided helper functions used in conjunction with slab caches.
#include <linux/mempool.h> /* First create a slab cache */ kmem_cache_t *my_cache = kmem_cache_create("my_cache", size, 0, SLAB_HWCACHE_ALIGN, NULL, NULL); /* Then create a memory pool based on that slab cache, setting the minimum reserved object count to 20 */ mempool_t *pool = mempool_create(20, mempool_alloc_slab, mempool_free_slab, my_cache); /* Allocate an object, using GFP_KERNEL flag */ void *element = mempool_alloc(pool, GFP_KERNEL); /* After use, free the object */ mempool_free(element, pool);It is important to note that although the memory pool attempts to ensure successful allocation, the allocation flags passed to mempool_alloc (such as GFP_KERNEL or GFP_ATOMIC) will affect its internal first step (attempting to allocate from the system).
Resizing and destroying
You can use mempool_resize to adjust the minimum number of pre-allocated objects in the memory pool.
Before destroying the memory pool, ensure that all objects allocated from that pool have been returned, and then destroy the custom slab cache (if used).
/* Resize the memory pool, changing the minimum object count to 30 */ mempool_resize(pool, 30, GFP_KERNEL); /* Destroy the memory pool */ mempool_destroy(pool); /* Then destroy the slab cache */ kmem_cache_destroy(my_cache);Points to note:
While memory pools can guarantee successful allocation, they also bring some overhead and risks, so they should be used with caution.
[1]. Memory waste: This is the most significant issue. Pre-allocated objects in the memory pool that may not be used for a long time will occupy memory, and even when system memory is very tight, they cannot be repurposed, potentially degrading overall system performance.
[2]. Risk of misuse: Not all situations require a memory pool. It should only be considered in specific scenarios where allocation failures cannot be handled (e.g., certain interrupt handlers or operations requiring atomicity). For most drivers, handling normal memory allocation failures (e.g., returning error codes for upper layers to retry) is usually a better choice.
[3]. Allocation delay: In cases where the pre-allocated objects in the memory pool are exhausted, mempool_alloc may block, waiting for other objects to be released. Therefore, when calling mempool_alloc in atomic contexts, the GFP_ATOMIC flag must be used to prevent blocking in non-sleepable contexts.
3. Per-CPU Variables
To address data contention issues between processors in SMP (Symmetric Multi-Processing) systems and avoid performance losses caused by using locks, Per-CPU variables can be used. Each CPU has an independent copy of this variable, and accessing the copy of the current CPU does not require locking.
Definition and usage:
static DEFINE_PER_CPU(int, my_percpu_var); // Static definition
int *ptr = get_cpu_var(my_percpu_var); // Get the address of the variable for the current CPU and disable preemption
(*ptr)++;
put_cpu_var(my_percpu_var); // Restore preemption
5. DMA and Coherent Memory
Direct Memory Access (DMA) requires specific memory areas.
Coherent DMA mapping: Use the dma_alloc_coherent() function to allocate a segment of memory that is directly accessible by the device and coherent with the CPU cache. It returns a physical address usable for DMA and the corresponding kernel-side virtual address.
Streaming DMA mapping: For ordinary memory buffers obtained using kmalloc or page allocators, temporary mappings need to be established using functions like dma_map_single() to inform the kernel and device to perform DMA transfers. After the transfer is complete, the mapping must be removed.
Based on the comparison of memory allocation methods in the table below, we can choose which method to use for memory allocation in our development scenarios:
Important principles:
[1]. Check return values: Kernel memory allocation may fail, and it is essential to check whether the return value is NULL.
[2]. Match context: In atomic contexts (interrupts, soft interrupts, etc.), only GFP_ATOMIC can be used; in process contexts, prefer using GFP_KERNEL.
[3]. Who allocates, who frees: Ensure that allocation and release functions are paired and clean up all allocated memory upon module exit to prevent memory leaks.
The above is the full content.
Previous articles (welcome to subscribe to the technical sharing column for all articles):[Project Practice] Locating the issue of Flash entering hardware write protection in embedded software systems and the implementation method to unlock the write lock using software[Project Practice] A nanny-level tutorial: PWM control of white light lamps and brightness adjustment[Project Practice] Self-review and experience summary of front-line programmers[Project Practice] Troubleshooting and resolution of audio intercom function delay and stuttering under Linux
“Thank you for reading this far”
This is the notebook of a female programmer
15+ years embedded software engineer and a mother of two
Sharing reading insights, work experiences, self-growth, and lifestyle.
I hope my words can be helpful to you