Follow to see more great articles like this~
Multi-size THP creation, two different ways
By Jonathan CorbetFebruary 13, 2025Gemini-1.5-flash translationhttps://lwn.net/Articles/1009039/
Huge pages (巨页) can enhance the performance of many programs, but they can also bring adverse performance impacts. In recent years, multi-size transparent huge pages (mTHPs) (多尺寸透明巨页) have increasingly been seen as a compromise that offers the advantages of huge pages at a lower cost. However, if the system does not create mTHP, it cannot benefit from it; two developers have released patches to enable the creation of mTHP in the background.
Huge pages (巨页)
Huge pages (巨页) are a feature implemented by the CPU and its memory-management unit (内存管理单元); they allow the system to operate with mixed page sizes. However, these sizes are limited in number and are entirely defined by the CPU’s memory architecture; they correspond to levels in the page-table hierarchy (页表层级). As a reminder, the hierarchy roughly looks like this (from this 2013 article):
(More modern systems can also have another layer below the top of the hierarchy).
On x86 systems, the base-page (基本页) size is 4KB. The first level of huge pages (巨页) is implemented by marking entries in the page middle directory (PMD, 页面中间目录), which are the entries in the layer just above the bottom level of page-table entries (PTE, 页表条目). The PMD entries then directly reference pages that cover a larger portion of the address space rather than the PTE table. This results in a page size of 2MB; these pages are commonly referred to as PMD huge pages (PMD 巨页). Moving up to the next level in the page upper directory (PUD, 页面上级目录) produces 1GB huge pages (巨页).
Huge pages (巨页) implemented in this way have some significant advantages. A single PMD huge page (PMD 巨页) replaces 512 base pages (基本页), significantly reducing the overhead of memory management and the number of page faults (缺页中断) the system must handle. It also frees up to 512 entries in the translation lookaside buffer (TLB), which is a scarce processor resource and can often be a performance bottleneck. Therefore, programs using huge pages (巨页) can run faster. Initially, using huge pages (巨页) required special efforts from developers and system administrators; the addition of transparent huge pages (THPs, 透明巨页) in 2011 allowed the kernel to coalesce the base pages (基本页) of processes into huge pages (巨页) in the background, allowing the entire system to benefit from the performance advantages of huge pages (巨页).
Unfortunately, huge pages (巨页) are not without cost. The cost of using huge pages (巨页) can be internal fragmentation (内部碎片); if a process only uses part of a huge page (巨页), the rest will be wasted. Therefore, enabling transparent huge pages (透明巨页) can significantly increase the memory usage of workload (工作负载); if the system is close to its limits without huge pages (巨页), enabling them may push it over the limit. Consequently, distributors often disable this feature, and it is easy to find recommendations online to keep it that way.
The multi-size option (多尺寸选项)
Multi-size THPs (多尺寸 THP), as the name suggests, extend the concept of huge pages (巨页) to more sizes, specifically those that do not correspond to the page-table hierarchy (页表层级). The mTHP concept is enabled by transition to folios (转换为页), as a way to manage memory in the kernel; it can now represent groups of physically contiguous pages of arbitrary (though powers of two) sizes. Using mTHP will again reduce memory management overhead, as they can be managed as a unit rather than as separate base pages (基本页). Updated CPUs have added TLB-coalescing (TLB 合并) capabilities that allow an mTHP of 8 pages (x86) or 16 pages (Arm) to be represented by a single TLB entry, thus enhancing the performance benefits that mTHP can bring.
Therefore, using mTHP can yield performance improvements similar to traditional huge pages (巨页), but with lower costs of internal fragmentation (内部碎片). For the kernel, allocating mTHP is also much easier than allocating a full PMD or PUD huge page (巨页), especially when the system has been running for a while and memory has become fragmented. Therefore, in recent years, considerable effort has been made to enhance the kernel’s ability to utilize mTHP, which is not surprising.
The THP page mechanism can inject huge pages (巨页) into the process’s address space in several ways. One way is at the initial allocation of pages. For example, the kernel can allocate a huge page (巨页) and read everything in a single operation, rather than using base pages (基本页) to satisfy a page fault (缺页中断). Another method is to check the process’s address space in the background and identify ranges of virtual contiguous pages that can be replaced with huge pages (巨页). When such a range is found, a huge page (巨页) will be allocated, the base pages (基本页) copied into it, and then they will be released after replacing them in the page table of the huge page (巨页). The latter task is performed by the <span>khugepaged</span> kernel thread, which benefits from being able to observe the actual memory pages used by the process. However, the <span>khugepaged</span> thread can only create PMD huge pages (PMD 巨页); it predates mTHP and does not create them. The two problematic patch sets are aimed at filling this gap.
Before examining them, there are some useful background knowledge associated with these two methods. There is a sysctl knob (sysctl 旋钮) <span>max_ptes_none</span> that controls how many new pages <span>khugepaged</span> can allocate to create a huge page (巨页). For example, if a suitable range of address space contains 500 active pages (活动页面), then <span>khugepaged</span> must allocate 12 additional pages to have the complete 512 pages that make up a PMD huge page (PMD 巨页). If <span>max_ptes_none</span> is set to at least 12, <span>khugepaged</span> will proceed; otherwise, it will abandon that huge page (巨页). Creating mTHP in <span>khugepaged</span> increases the amount of thought that must go into setting this knob (旋钮).
Improving <span>khugepaged</span> (改进 khugepaged)
The first patch set from Nico Pache changes the core <span>khugepaged</span> scan loop (currently known as <span>hpage_collapse_scan_pmd()</span> in the current kernel, although this series renames it to <span>khugepaged_scan_pmd()</span>). The current implementation stops scanning candidate huge pages (候选巨页) immediately if it finds <span>max_pte_none</span> unmapped pages (未映射页面). In the modified version, it will scan the entire address range of huge pages (巨页) and create a bitmap (位图) marking where mTHP can be created in that page. The minimum size considered is order 3 (8 pages); this may be changed in the future to allow a minimum order 4 on Arm systems, matching the size of TLB entries that can be merged.
When determining whether a given chunk of size mTHP can be converted into an actual mTHP, this patch accordingly scales the <span>max_pte_none</span> value. When looking at an 8-page chunk, <span>max_pte_none</span> will be divided by 64 (512/8) to determine how many pages must actually exist.
After scanning the PMD, <span>khugepaged</span> will first consider whether the entire content can be converted into a PMD huge page (PMD 巨页). If that fails, it will consult the bitmap created during scanning to try to create as large an mTHP as possible, depending on the actual pages resident in memory. After processing the bitmap and creating any mTHP, <span>khugepaged</span> will continue scanning the next PMD.
This approach has a problem that both patch sets must address; it is known as creep (蔓延). Suppose <span>max_pte_none</span> is set to 511, which is exactly its default value. Under this setting, even a single resident page (常驻页面) is enough to cause <span>khugepaged</span> to fill the rest of the huge page (巨页), which is likely what the user wants. However, when dealing with mTHP, things change a bit. When scanning PMD, each single page will lead to the creation of an 8-page mTHP, as expected. However, when the next scan occurs, these mTHP will each be converted into 16-page mTHP, since each mTHP contains enough pages to allow <span>khugepaged</span> to elevate them to the next size.
Over time, these mTHP will all creep to PMD size, which is precisely the result the patch set aims to avoid. To prevent this from happening, the administrator needs to set <span>max_pte_none</span> to a value less than half of the PMD huge page (PMD 巨页) size, that is, 255 or smaller.
The second patch series from Dev Jain takes a slightly different approach. <span>khugepaged</span> first scans in chunks of PMD size as before, but if it cannot collapse the given PMD range, it will drop to a smaller order and try again. The code implementing this algorithm consists of a bunch of <span>goto</span> statements, which can be a bit hard to follow, but the goal is to scan each memory range at a size that successfully creates mTHP. After completing the PMD entries, the order will reset to the maximum to start on the next entry.
This series takes a stricter approach to the creep problem; it will only perform mTHP collapse (合并) when <span>max_ptes_none</span> is set to zero or 511. In the former case, mTHP will only be created if all base pages (基本页) being replaced already exist; in the latter case, creep to the full PMD size is explicitly allowed.
Pache has released another patch set that can be used with either <span>khugepaged</span> mTHP implementation. In the current kernel, various sizes of mTHP can be explicitly enabled or disabled through a set of sysctl knobs (sysctl 旋钮) (documented in Documentation/admin-guide/mm/transhuge.rst). Setting <span>always</span> causes the kernel to attempt to create that size of mTHP, while <span>never</span> disables that size. Pache added a new <span>defer</span> setting that causes the kernel to not use the given size to respond to page faults (缺页中断), but will allow <span>khugepaged</span> to collapse to that size. The aim here is to restrict mTHP allocations to memory that has proven useful, hoping to further reduce the costs of using mTHP while enjoying the advantages of mTHP.
As of the time of writing, both patch sets have just been released; aside from Andrew Morton requesting benchmark results, there has not been any real review discussion generated. Therefore, it is difficult to predict which (if any) patch set will be accepted, or how many changes may be required before that happens. However, it is clear that there is a desire to better utilize mTHP, so the objectives behind this work are easy to understand; sooner or later, the kernel will almost certainly acquire some capability to create mTHP in the background while processes are running.
The end of the articleLWN articles follow the CC BY-SA 4.0 license.
Feel free to share, reprint, and create derivative works based on existing agreements~Long press the QR code below to follow for in-depth articles from LWN and various recent discussions in the open-source community~