Analysis of High Memory Usage by Buffers and Cache in Linux

1. Origin of the Problem: When using a Linux system, we often find that the cache occupies a large amount of memory. For example, when checking memory status with the `free` command, we may see that `buff/cache` has already taken up 2.1G of memory. Since the Linux 2.4 version, “buffer” and “cache” have been unified as page cache, resulting in a high overall memory usage, but the actually used memory (used) is only 282M. This indicates that a large amount of memory is occupied by the cache rather than actual usage.2. Linux Cache Content: The cache is the page cache for file data, primarily used to accelerate file read and write operations. When using technologies such as MMap, Buffered I/O, and Read-Ahead, the kernel generates page cache.However, if using Raw I/O or Direct I/O, it is possible to bypass the cache and perform I/O operations directly on the disk or partition, avoiding the use of cache.3. Tools to View Linux Cache: To understand which files are cached, tools such as `fincore`, `pcstat`, `hcache`, and `vmtouch` can be used. Among them, `fincore` is no longer maintained, and here we mainly introduce `pcstat`, `hcache`, and `vmtouch`. These tools can help analyze cache usage to identify the underlying issues.4. Using `pcstat`: `pcstat` is a tool for analyzing cache, and its installation and usage methods are as follows. First, the `go` language package needs to be installed, then clone the source code, compile the tool, and run it to verify its functionality. It is worth noting that different operating systems and CPU architectures require compiling the corresponding version of the `pcstat` executable to ensure compatibility and optimal performance.5. `hcache` Tool: `hcache` is also a tool for cache analysis, and its installation method is similar to that of `pcstat`. `hcache` provides some additional features, such as summation calculations and the `–top` option to display the files or processes that occupy the most cache, although the accuracy of this feature may be limited.6. `vmtouch` Tool: `vmtouch` is a tool written in C for understanding and controlling file system cache in Unix and Unix-like systems. It offers a range of functionalities, including querying files in cache, preloading files, clearing cache, and locking file pages to prevent them from being swapped out to disk.7. Conclusion: Through the aforementioned tools, we can effectively analyze and manage the cache usage in Linux systems, thereby optimizing memory usage and improving system performance. By combining these tools, we can more intuitively address issues, such as identifying files and processes that occupy a large amount of cache and taking corresponding measures to reduce cache usage and enhance system efficiency.

Leave a Comment