Buffer and Cache are present in the output of the free command:
$ free
total used free shared buff/cache available
Mem: 8169348 263524 6875352 668 1030472 7611064
Swap: 0 0 0
Note that the output of free may vary between different versions.
This output includes specific usage statistics for physical memory Mem and swap partition Swap, such as total memory, used memory, cache, available memory, etc. The cache is the sum of Buffer and Cache. Today, we will focus on understanding the origins and meanings of Buffer and Cache.
By consulting the Linux manual man free, we can see the descriptions of Buffer and Cache:
- Buffers are memory used by the kernel for buffering, corresponding to the Buffers value in /proc/meminfo.
- Cache is memory used for the kernel page cache and Slab, corresponding to the sum of Cached and SReclaimable in /proc/meminfo.
These values come from /proc/meminfo, and by checking man proc:
/proc is a special filesystem provided by the Linux kernel, serving as an interface for users to interact with the kernel. For example, users can query the kernel’s running status and configuration options from /proc, check the running status and statistics of processes, and of course, you can also modify kernel configurations through /proc.
- Buffers are temporary storage for raw disk blocks, used to cache disk data, typically not very large (around 20MB). This allows the kernel to consolidate scattered writes and optimize disk writing, such as merging multiple small writes into a single large write.
- Cached is the page cache for files read from the disk, used to cache data read from files. This way, when accessing these file data next time, it can be quickly retrieved from memory without needing to access the slow disk again.
- SReclaimable is part of Slab. Slab consists of two parts, with the reclaimable part recorded by SReclaimable, and the unreclaimable part recorded by SUnreclaim.
According to research:
- Buffer can be used as both “a cache for data to be written to disk” and “a cache for data read from disk”.
- Cache can be used as both “a page cache for data read from files” and “a page cache for writing files”.
In simple terms, Buffer is a cache for disk data, while Cache is a cache for file data. They can be used in both read and write requests.
About the difference between disk and filesDisk is a block device that can be partitioned into different sections; on top of the partitions, a filesystem is created and mounted to a directory, after which files can be read and written in that directory.In fact, in Linux, “everything is a file”, and the “files” mentioned in this article refer to ordinary files, while disks are block device files. You can execute “ls -l <path>” to see their differences (if you don’t understand the output, please refer to man ls).When reading and writing ordinary files, it goes through the filesystem, which is responsible for interacting with the disk; while reading and writing disks or partitions skips the filesystem, known as “raw I/O”. The caching used in these two read/write methods is different, which is the distinction between Cache and Buffer discussed in this article.
Additional Information
Theoretically, reading a file first goes to Block Buffer, then to Page Cache. Page Cache exists because of the filesystem. In older versions of Linux, these two caches were separate. Thus, file data would be cached twice. This approach, while simple, was inefficient. Later, Linux unified these two caches. For files, Page Cache points to Block Buffer, while for non-files, it is Block Buffer. This means that for file operations, only Page Cache is affected, while raw operations only affect Buffer. For example, a VM virtual machine would bypass the filesystem and only operate on the disk, commonly referred to as Direct I/O.
Common Observation Commands
(1) vmstat
#vmstat 1
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
r b swpd free buff cache si so bi bo in cs us sy id wa st
00 07499460 1344230484 0 0 0 0 291450010000
10 07338088 1752390512 0 0 488 0 395580475300
10 07158872 1752568800 0 0 0 4 303761504900
10 06980308 1752747860 0 0 0 0 243600505000
00 06977448 1752752072 0 0 0 0 291380010000
00 06977440 1760752080 0 0 0 152 42212019910
...
- buff and cache correspond to the Buffers and Cache we saw earlier, measured in KB.
- bi and bo represent the size of block device reads and writes, measured in blocks per second. Since the block size in Linux is 1KB, this unit is equivalent to KB/s.
(2) Clearing Cache
echo 3 > /proc/sys/vm/drop_caches
Here, /proc/sys/vm/drop_caches is an example of modifying kernel behavior through the proc filesystem. Writing 3 indicates clearing various caches such as file pages, directory entries, Inodes, etc.
(3) Statistics of Physical Memory Usage by All Processes
This is done by summing the Pss in /proc//smaps, as Pss is the portion of private memory + shared memory that belongs to the process proportionally. For example, if private memory is 200k, and shared memory is 500k shared with 4 other processes, then Pss would be 200k + (500/(1+4)) = 200k + 100k = 300k.Thus, summing the Pss of all processes avoids concerns of double counting, as the shared memory portion is already accounted for in Pss.
Reference command:
awk '/Pss:/{ sum += $2 } END { print sum }' /proc/$$/smaps