Using the diff Command in the Linux Terminal to Compare Two Files

Last week, I encountered a strange issue. I modified a YAML file locally, and it worked fine in the simulated environment. I planned to deploy it to the integration environment for formal testing. However, I wanted to conduct a small-scale test myself before the formal testing to minimize the chances of bugs being discovered and reduce deployment frequency. Since the changes were minimal, I directly modified the same YAML file in the integration environment. But when I started the service in the integration environment, it threw an error:

Error: error parsing xxxconfig.yaml:  ...  mapping values are not allowed in this context  ....

A YAML syntax error? Impossible, it worked fine locally! When I reverted to the previous version, the service started successfully. My local service also started without issues. I even opened the configuration files from both local and integration environments and compared them visually several times; they looked identical, and I couldn’t find any mistakes. This was puzzling. Later, I uploaded the modified YAML file from my local environment to the integration environment and used diff to compare them. I discovered that the line causing the error in the integration environment had one less space in its indentation. It turned out that I had manually modified the YAML file in the integration environment, adding an extra space (possibly due to automatic formatting by the editor)!!! One space caused the YAML parser to interpret it as a nested structure, resulting in an error.

If I had known this earlier, I wouldn’t have been lazy and foolish; I should have uploaded the modified YAML directly for testing. When comparing file differences, I shouldn’t have relied on my eyes instead of using the excellent diff tool, wasting over ten minutes of my time. But then again, who hasn’t had moments of laziness and foolishness?

Back to the main topic, speaking of file comparison tools, I will summarize the usage of the commonly used comparison tools diff and cmp.

We often modify a file, especially configuration files, and compare the inconsistencies before and after the changes. I generally use the diff command. The diff command has three output formats:

1. Default format

$ diff test1.conf test2.conf3c3< database_host=192.168.1.10---> database_host=10.0.0.50

I do not recommend using the default format because it only shows the differences without context, making it unclear which line has changed.

2. Unified format, using the parameter “-u”

$ diff -u test1.conf test2.conf@@ -3,7 +3,7 @@ # Database Configuration-database_host=192.168.1.10+database_host=10.0.0.50 database_port=3306

From the comparison output above, you can see that the result using -u is similar to the format we commonly use with git diff. It shows the context and indicates which line has changed. It also outputs +/- symbols, clearly indicating what was added and what was removed. I personally highly recommend using “-u” to output file differences.

3. Side-by-side output format, using the parameter “-y”

$ diff -y test1.conf test2.confdatabase_host=192.168.1.10  | database_host=10.0.0.50database_port=3306             database_port=3306

From this example, you can see that using “-y” outputs the results with differing lines side by side, which is very intuitive. However, if the content of the file lines is long, it can be difficult to read. Therefore, it is more suitable for simple comparisons between short files.

Sometimes, when comparing files, some differences are expected (such as timestamps, domain names, ports, etc.), and we want to ignore these expected differences. We can use the parameter “-I” followed by the line to ignore.

# Use diff -I to ignore lines containing "time_stamp"$ diff -u -I "time_stamp" test1.conf test2.conf

Alternatively, we can use grep -v to ignore lines containing “time_stamp”:

$ diff -u test.conf prod.conf | grep -v "database_host"

We can also use -w to ignore whitespace characters and -i to ignore case sensitivity.

With diff, we can also compare entire directories by using the parameter “-r”. If multiple files have been modified, comparing them one by one can be cumbersome. The following command can recursively compare two directories:

$ diff -r project_backup/ project_current/# Output: Only in project_current/: new_file.pydiff -r project_backup/main.py project_current/main.py45a46> print("Debug")

From the output above, it can be seen that a new file new_file.py was added, and line 46 of main.py added debugging code.

In addition to diff, I also often use the tool vimdiff in the Linux server terminal. Compared to diff, vimdiff has a more intuitive interactive graphical interface, allowing direct navigation to the differences and enabling direct editing and merging of differences, making it very convenient. The command is simple:

vimdiff test1.conf test2.conf

When mentioning diff, we cannot forget another commonly used file comparison tool in the Linux terminal, cmp.

Diff is a more advanced comparison tool that splits text by lines and generates readable differences, making it suitable for comparing text files.

If it is a binary file, cmp is more suitable. Cmp reads files directly by byte stream and compares them byte by byte, accurately locating the differences.

Of course, in practical work, cmp can be used in conjunction with diff. For example, sometimes we back up files, and to quickly verify if the backup is consistent, even for text files, we can first use cmp for a quick comparison.

$ cmp test.conf test.conf.backup

If they are identical, the output will be 0. If they are not identical, we can use diff or vimdiff to check where the changes occurred.

Thank you for reading this far. If this article was helpful to you, feel free to follow, like, share, and subscribe. For more Linux command usage, please refer to the links below:

The Five Major Application Scenarios and Usage of the find Command in Linux

How to Quickly Find the Directories Taking Up the Most Disk Space in Linux?

A Detailed Explanation of File Upload with curl

A Detailed Explanation of the Usage of the wget Tool for File Download in Linux

How to Efficiently Use Commands like sed, awk, grep to Delete Specific Lines from Files