Finding Logs in Linux

In Linux systems, log files are critical resources that record the system’s operational status, error messages, and various events. Properly searching and analyzing logs is essential for system maintenance and troubleshooting. Here are some commonly used methods and techniques for searching logs.

There are various commands to view logs in Linux: grep, tail, head, cat, tac, less, etc. This article will introduce a few commonly used methods.

1. grep Command

In Linux systems, grep is a very powerful text search tool used to search for lines in a file that match a specific pattern. If you want to use grep to find content in log files, you can follow these common methods:

1. Search for lines containing specific text

Assuming you want to search for all lines containing “error”, you can use:

grep "error" filename.log

Where filename.log is the name of the log file you want to search.

2. Search case-insensitively

Using the -i option allows for case-insensitive searching:

grep -i "error" filename.log

3. Count matching lines

Using the -c option counts the number of matching lines:

grep -c "error" filename.log

4. Display matching lines with line numbers

Using the -n option displays the line numbers of matching lines:

grep -n "error" filename.log

5. Search in multiple files

You can search for the same pattern in multiple files by listing the filenames as arguments:

grep "error" file1.log file2.log file3.log

6. Recursively search in directories

Using the -r or -R option allows for recursive searching in a directory and its subdirectories:

grep -r "error" /path/to/logs/

7. Use regular expressions

grep supports regular expressions, which can be used for more complex searches:

grep "^ERROR" filename.log  # Matches lines starting with ERROR
grep "ERROR [0-9]\{4\}" filename.log  # Matches lines with ERROR followed by four digits

8. Combine with other commands

You can combine grep with other commands (like awk, sed, cut, etc.) to process and transform search results. For example, using awk to print detailed information of matching lines:

grep "error" filename.log | awk '{print $0, NR}'  # Prints line content and line number

9. Use egrep or grep -E for extended regular expression searches

If you need to use extended regular expressions (like | for “or”), you can use egrep or grep -E:

grep -E "error|warning" filename.log  # Searches for lines containing error or warning

10. Ignore case and count matches

Combine the -i and -c options:

grep -ic "error" filename.log  # Case insensitive and counts matches

With these basic and advanced techniques, you can efficiently use the grep command to search and analyze information in log files. You can also continue to filter results using pipes or redirection commands based on the query results.

2. tail Command

In Linux operating systems, the tail command is used to output the last part of a file. It is very useful, for example, when viewing log files, as you only need to focus on the latest log entries.

By default, tail displays the last 10 lines of data. If multiple files are specified, it will prepend the filename to the content of each displayed file for distinction.

1. Display the end of a file

-n or --lines=[+]K: Displays the last K lines of the file. If a + precedes K, it starts displaying from the K-th line.

tail -n 5 filename  # Displays the last 5 lines of the file
tail -n +5 filename  # Starts displaying from the 5th line

-c or --bytes=[+]K: Displays the last K bytes of the file. If a + precedes K, it starts displaying from the K-th byte.

tail -c 100 filename  # Displays the last 100 bytes of the file
tail -c +100 filename  # Starts displaying from the 100th byte

2. Real-time display of the end of a file

-f displays new content added to the file in real-time, commonly used for monitoring log files.

tail -f filename

-F is similar to -f but continues to track new files if the file is renamed or deleted.

tail -F filename

3. Do not display filenames

-q or --quiet or --silent does not display filenames (when processing multiple files).

tail -q filename1 filename2

4. Always display filenames

-v or --verbose: Always display filenames (even when processing a single file).

tail -v filename

5. Combine with process monitoring

Used with the -f option, the tail command will automatically exit when the specified process ID terminates.

tail -f --pid=PID filename

6. Refresh at intervals

-s or --sleep-interval=S used with -f specifies the interval time (in seconds) for checking file changes.

tail -f filename -s 5  # Refreshes every 5 seconds

File size: For very large files, using the tail command usually does not cause performance issues, as it only reads the end part of the file.Permission issues: If the file does not have read permissions, the tail command will report an error. Ensure you have the appropriate permissions to read the file.Output redirection: Using > to redirect output will overwrite the contents of the target file. Using >> to append content will add to the end of the target file.Real-time monitoring: When using the -f or -F options, you can terminate real-time monitoring by pressing Ctrl+C.

3. head Command

The head command is used in Linux systems to display the beginning part of a file.

By default, the head command displays the first 10 lines of a file. If the file contains fewer than 10 lines, it will display all content.

1. Display the beginning of a file

-n or --lines=[-]K: Specifies the number of lines to display from the file. If K is negative, it starts displaying after skipping K lines from the beginning of the file.

head -n 5 filename  # Displays the first 5 lines of the file
head -n -5 filename  # Displays all content except the last 5 lines

-c or --bytes=[-]K: Specifies the number of bytes to display from the file. If K is negative, it starts displaying after skipping K bytes from the beginning of the file.

head -c 100 filename  # Displays the first 100 bytes of the file
head -c -100 filename  # Displays all content except the first 100 bytes

2. Multiple files without displaying filenames

-q or --quiet: Does not display filenames. This is useful when processing multiple files to avoid filename information in the output.

head -q file1 file2 file3

3. Always display filenames

-v or --verbose: Always display filenames, even when processing a single file.

head -v filename

NotesFile size: For very large files, using the head command usually does not cause performance issues, as it only reads the beginning part of the file.Permission issues: If the file does not have read permissions, the head command will report an error. Ensure you have the appropriate permissions to read the file.Output redirection: Using > to redirect output will overwrite the contents of the target file. Using >> to append content will add to the end of the target file.

4. cat Command

cat is a very commonly used command in Linux systems, short for “concatenate”, mainly used to view, concatenate, and create files. Here are the main functions and usage of the cat command.

1. View file content

Basic usage

cat filename  # Displays the content of the specified file to the terminal

Display line numbers: Using the -n option adds line numbers to each output line, including empty lines; while the -b option only numbers non-empty lines.

cat -n filename
cat -b filename

Display special characters: The -E option displays a $<code> symbol at the end of each line, the <code>-T option displays tabs as ^I, and the -v option displays non-printing characters.

cat -E filename
cat -T filename

3. Concatenate files

Merge files

cat file1 file2 > combined_file  # Merges the contents of file1 and file2 into combined_file

Display multiple files

cat file1 file2

Create files: You can create files from standard input, and press Ctrl+D to save and exit after inputting.

cat > test.txt

Append content: You can append the content from standard input to the end of a file.

cat >> filename

4. Combine with pipes

Combine with other commands

ls -l | cat

View the last few lines of a file

cat filename | tail -n 10  # Displays the last 10 lines of the file.

5. tac Command

tac outputs the contents of a file line by line in reverse order, which can be seen as the reverse version of the cat command. It supports custom delimiters (like regular expressions) to flexibly handle non-newline-separated text.

1. Display file content in reverse

tac /etc/passwd

2. Specify delimiters for text processing

-s or --separator: Specify a custom delimiter (default is newline \n).

tac -s '#' data.txt  # Outputs content in reverse using `#` as the delimiter

3. Regular expression delimiters

-r or --regex: Enables regular expression matching for delimiters (must be used with -s).

tac -sr '^[0-9]+' log.txt   # Outputs in reverse using lines starting with numbers as delimiters

6. less Command

1. Basic usage

It supports scrolling up and down, displaying text by pages, and is more flexible and powerful than the more command.

less filename.txt

2. Common operations

Move the cursor: Use the arrow keys ↑ and ↓ to move up and down one line; Page Up and Page Down keys to scroll up and down a page.

Search text: Type / followed by the search term, and press Enter to search down; type ? followed by the search term, and press Enter to search up. Press n to find the next match, and press N to find the previous match.

Exit command: Press q to exit the less command.

3. Common options

-N: Displays line numbers, making it easier to view and locate text content.

less -N filename.txt

-S: When text lines are too long, it truncates the part that exceeds the screen width to avoid line wrapping affecting viewing.

less -S filename.txt

-m: Displays the percentage position of the file and the range of currently displayed lines, helping to understand viewing progress.

less -m filename.txt

4. Advanced features

Mark and jump: Use m followed by a letter to set a mark, and use ' followed by the letter to jump to the marked position.

View multiple files: You can open multiple files simultaneously, using :n to switch to the next file, and :p to switch to the previous file.

less -Nm filename1.txt filename2.txt

Continuous monitoring of files: Use the F option, similar to tail -f, to continuously monitor and display new content at the end of the file.

less -F filename.txt

5. Applicable scenarios

View large log files: Quickly locate error messages or specific event records.

Read long documents: Such as program code, configuration files, etc., for easy browsing and searching.