1. Types of Disk I/O Tests

In daily testing, the main types of disk I/O tests are as follows:

Random Read, Random Write, Random Read/Write
Sequential Read, Sequential Write, Sequential Read/Write

2. Introduction to FIO

FIO is an open-source IOPS testing tool in Linux, primarily used for stress testing and performance validation of disks. By using FIO, multiple threads or processes can be created to perform specific types of I/O operations, either by writing configuration files or executing test actions directly from the command line, effectively serving as a multi-threaded I/O generation tool. The FIO tool can generate various I/O patterns to test the performance of disk devices. Its open-source address is: https://github.com/axboe/fio

Using the FIO tool, you can set read/write ratios according to testing needs, such as 80% read and 20% write, etc.

2.1 FIO Installation

Taking Ubuntu 22.04 as an example, the installation command is as follows:

sudo apt-get update && sudo apt-get install -y fio

2.2 Common Parameters

Parameter Name	Function	Example Value
bs	Defines the block size for I/O operations, with units in k, K, m, and M, etc. The default I/O block size is 4KB.	4KB
ioengine	Defines how FIO issues I/O requests, typically using synchronous I/O and asynchronous I/O.– Synchronous I/O can only issue one I/O request at a time, waiting for the kernel to complete before returning. This means that the I/O queue depth for a single thread is always less than 1, but can be filled by concurrent execution of multiple threads. Typically, 16 to 32 threads are used to fill the I/O queue depth.– Asynchronous I/O usually uses methods like libaio to submit a batch of I/O requests at once, then wait for the batch to complete, reducing the number of interactions and increasing efficiency.	libaio
iodepth	Defines the I/O queue depth during testing, default is 1.	1
direct	Direct mode – 1: Indicates the use of direct I/O, bypassing I/O cache, writing data directly.– 0: Indicates the use of buffered I/O.– Default is True.	1
rw	Read/write mode, possible values include:– Sequential Read: read– Sequential Write: write– Random Read: randread– Random Write: randwrite– Mixed Random Read/Write: randrw– Mixed Sequential Read/Write: readwrite	read
time_based	Specifies the use of time-based mode. No parameter value needs to be set; FIO runs based on time.
runtime	Specifies the duration of the test, i.e., the duration FIO runs, in seconds.	600
refill_buffers	FIO will refill the I/O buffer on each submission; the default setting is to fill it only at the beginning and reuse that data.
norandommap	When performing random I/O, FIO will overwrite each block of the file. If this parameter is given, it will select a new offset without looking at the I/O history.
randrepeat	Whether the random sequence is repeatable.– True or 1: Indicates the random sequence is repeatable.– False or 0: Indicates the random sequence is not repeatable.– Default is True.	1
group_reporting	Defines the display mode for test results; group_reporting summarizes statistics for each process rather than displaying information aggregated by different jobs.
name	Defines the name of the test task.
filename	Defines the name of the test file/device.	/dev/vdb
size	Defines the amount of data for the test I/O operations. If parameters like runtime are not specified, FIO will read/write the specified amount of data completely before stopping the test.– The value of this parameter can be a number with units, such as size=10G, indicating that the amount of data read/written is 10GB; it can also be a percentage, such as size=20%, indicating that the amount of data read/written occupies 20% of the total file space on the device.	500GB
rwmixwrite	In mixed read/write mode, the write proportion.	30
numjobs	Defines the number of concurrent threads for the test.

2.3 Using FIO

2.3.1 Running via Command Line

Testing Random Write IOPS

sudo fio -direct=1 -iodepth=32 -rw=randwrite -ioengine=libaio -bs=4k -size=20G -numjobs=16 -runtime=600 -group_reporting -filename=/${MOUNT_PATH}}$/${TEST_FILE_NAME} -name={RAND_WRITE_TEST}

Testing Random Read IOPS

sudo  -direct=1 -iodepth=32 -rw=randread -ioengine=libaio -bs=4k -size=20G -numjobs=16 -runtime=600 -group_reporting -filename=/dev/{DEVICE_NAME} -name={RAND_READ_TEST}

Testing Write Throughput

sudo fio -direct=1 -iodepth=64 -rw=write -ioengine=libaio -bs=4k -size=20G -numjobs=16 -runtime=600 -group_reporting -filename=/${MOUNT_PATH}}$/${TEST_FILE_NAME} -name={WRITE_TEST}

Testing Read Throughput

sudo fio -direct=1 -iodepth=64 -rw=read -ioengine=libaio -bs=4k -size=20G -numjobs=16 -runtime=600 -group_reporting -filename=/dev/{DEVICE_NAME} -name={READ_TEST}

2.3.2 Running via Configuration File

2.3.2.1 General Configuration

FIO supports running based on configuration files in addition to command line execution, with a classic ini format as follows:

Use [ ] to define the job name, but do not use <span>global</span>
Comments use ; or #

; -- start job file --
[global]
rw=randread
size=128m

[job1]

[job2]

; -- end job file --

The above configuration file has the same effect as the following command:

# fio --name=global --rw=randread --size=128m --name=job1 --name=job2

Now let’s look at the following example:

; -- start job file --
[random-writers]
ioengine=libaio
iodepth=4
rw=randwrite
bs=32k
direct=0
size=64m
numjobs=4
; -- end job file --

The above configuration file has the same effect as the following command:

# fio --name=random-writers --ioengine=libaio --iodepth=4 --rw=randwrite --bs=32k --direct=0 --size=64m --numjobs=4

Additionally, FIO supports fast testing through shared test files, with the configuration file written as follows:

; -- start job file including.fio --
[global]
filename=/tmp/test
filesize=1m
; Global included test file
include glob-include.fio

[test]
rw=randread
bs=4k
time_based=1
runtime=10
include test-include.fio
; -- end job file including.fio --

2.3.2.2 Using Environment Variables

In configuration files, FIO also supports the use of environment variables, with the reference format in the configuration file as <span>${VAR_NAME}</span>, as shown in the following example:

; -- start job file --
[random-writers]
rw=randwrite
size=${SIZE}
numjobs=${NUMJOBS}
; -- end job file --

Execute the following command:

SIZE=64m NUMJOBS=4 fio jobfile.fio

After using environment variables, the equivalent configuration file is:

; -- start job file --
[random-writers]
rw=randwrite
size=64m
numjobs=4
; -- end job file --

2.3.3 Practice

1. Write Configuration File

[global]
ioengine=libaio
direct=1
iodepth=1
bs=4k
numjobs=32
runtime=120
size=5G
randrepeat=0
group_reporting

[rand-read-test]
rw=randread
filename=/dev/sde

[rand-write-test]
rw=randwrite
filename=/home/data4/fio.randwrite.test

[rand-read-write-test]
rw=randrw
filename=/home/data4/fio.rand-read-write.test

2. Run

sudo fio fio.conf

3. Result Analysis

The output results are as follows:

# sudo fio fio.conf
rand-read-test: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1
...
rand-write-test: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1
...
rand-read-write-test: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1
...
fio-3.28
Starting 96 processes
Jobs: 96 (f=96): [r(32),w(32),m(32)][100.0%][r=151MiB/s,w=104MiB/s][r=38.6k,w=26.7k IOPS][eta 00m:00s]
rand-read-test: (groupid=0, jobs=96): err= 0: pid=3058638: Wed Aug 20 10:47:05 2025
read: IOPS=41.0k, BW=160MiB/s (168MB/s)(18.8GiB/120002msec)
    slat (usec): min=2, max=84346, avg=237.56, stdev=917.90
    clat (nsec): min=882, max=105729k, avg=934390.79, stdev=2021217.42
     lat (usec): min=3, max=110005, avg=1172.49, stdev=2263.52
    clat percentiles (usec):
     |  1.00th=[    3],  5.00th=[  184], 10.00th=[  285], 20.00th=[  404],
     | 30.00th=[  490], 40.00th=[  578], 50.00th=[  685], 60.00th=[  832],
     | 70.00th=[ 1020], 80.00th=[ 1254], 90.00th=[ 1598], 95.00th=[ 1893],
     | 99.00th=[ 2966], 99.50th=[ 5997], 99.90th=[28705], 99.95th=[50594],
     | 99.99th=[77071]
   bw (  KiB/s): min=93770, max=203572, per=100.00%, avg=164165.63, stdev=311.35, samples=15296
   iops        : min=23442, max=50892, avg=41034.97, stdev=77.90, samples=15296
  write: IOPS=25.6k, BW=99.9MiB/s (105MB/s)(11.7GiB/120002msec); 0 zone resets
    slat (usec): min=6, max=99557, avg=865.97, stdev=1946.04
    clat (nsec): min=1438, max=97151k, avg=997772.53, stdev=1607120.25
     lat (usec): min=56, max=105108, avg=1864.40, stdev=2650.19
    clat percentiles (usec):
     |  1.00th=[  161],  5.00th=[  285], 10.00th=[  371], 20.00th=[  498],
     | 30.00th=[  603], 40.00th=[  701], 50.00th=[  807], 60.00th=[  922],
     | 70.00th=[ 1057], 80.00th=[ 1237], 90.00th=[ 1598], 95.00th=[ 2040],
     | 99.00th=[ 3490], 99.50th=[ 5407], 99.90th=[20841], 99.95th=[33162],
     | 99.99th=[64750]
   bw (  KiB/s): min=52498, max=131745, per=100.00%, avg=102401.73, stdev=240.76, samples=15296
   iops        : min=13117, max=32932, avg=25595.54, stdev=60.25, samples=15296
  lat (nsec)   : 1000=0.01%
  lat (usec)   : 2=0.55%, 4=0.68%, 10=0.15%, 20=0.01%, 50=0.01%
  lat (usec)   : 100=0.27%, 250=4.76%, 500=20.56%, 750=23.99%, 1000=16.98%
  lat (msec)   : 2=27.58%, 4=3.76%, 10=0.37%, 20=0.13%, 50=0.13%
  lat (msec)   : 100=0.04%, 250=0.01%
  cpu          : usr=0.89%, sys=3.70%, ctx=13650352, majf=0, minf=6861
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=4917306,3068451,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
   READ: bw=160MiB/s (168MB/s), 160MiB/s-160MiB/s (168MB/s-168MB/s), io=18.8GiB (20.1GB), run=120002-120002msec
  WRITE: bw=99.9MiB/s (105MB/s), 99.9MiB/s-99.9MiB/s (105MB/s-105MB/s), io=11.7GiB (12.6GB), run=120002-120002msec

Disk stats (read/write):
  sde: ios=4798086/3074895, merge=44/137845, ticks=4552421/2533315, in_queue=7085984, util=99.97%

I/O statistics for each execution task’s data direction.

IOPS related metrics can be found in the IOPS=*** content.
Throughput related metrics can be found in the BW=*** content.
Latency related metrics can be found in the lat related content.
Submission latency related metrics can be found in the slat related content.
Completion latency related metrics can be found in the clat related content.
CPU usage related metrics can be found in the CPU related content.
IO depths indicate the distribution of I/O during the job lifecycle.

Run status group x represents the meaning of values:

bw: Total bandwidth and minimum and maximum bandwidth.
io: The cumulative I/O executed by all threads in this group.
run: The minimum and maximum runtime of threads in this group.

Disk stats represent the meaning of values:

ios: The number of I/O operations for all groups.
merge: The total number of merges performed by the I/O scheduler.
ticks: The number of ticks that kept the disk busy.
in_queue: The total time spent in the disk queue.
util: Disk utilization.

2.4 Others

1. View Supported ioengine

# fio --eng help

The output results are as follows:

Available IO engines:
 libpmem
dev-dax
pmemblk
rbd
rados
rdma
libaio
io_uring
sg
mtd
gfapi_async
gfapi
splice
e4defrag
falloc
posixaio
exec
filedelete
filestat
filecreate
ftruncate
net
netsplice
null
sync
psync
vsync
pvsync
pvsync2
mmap
cpuio

Official help documentation: https://fio.readthedocs.io/en/latest/fio_doc.html#command-line-options

2. View Option Descriptions

# fio --cmdhelp

The output results are as follows:

description             : Text job description
name                    : Name of this job
wait_for                : Name of the job this one wants to wait for before starting
filename                : File(s) to use for the workload
  lockfile              : Lock file when doing IO to it
directory               : Directory to store files in
filename_format         : Override default $jobname.$jobnum.$filenum naming
unique_filename         : For network clients, prefix file with source IP
opendir                 : Recursively add files from this directory and down

Linux Disk Performance Testing Tool – FIO

1. Types of Disk I/O Tests

2. Introduction to FIO

2.1 FIO Installation

2.2 Common Parameters

2.3 Using FIO

2.3.1 Running via Command Line

2.3.2 Running via Configuration File

2.3.2.1 General Configuration

2.3.2.2 Using Environment Variables

2.3.3 Practice

2.4 Others

Leave a Comment Cancel reply

1. Types of Disk I/O Tests

2. Introduction to FIO

2.1 FIO Installation

2.2 Common Parameters

2.3 Using FIO

2.3.1 Running via Command Line

2.3.2 Running via Configuration File

2.3.2.1 General Configuration

2.3.2.2 Using Environment Variables

2.3.3 Practice

2.4 Others

Related posts

Leave a Comment Cancel reply