(Click the public account above to quickly follow)

Source: Wu Qin (Tyler)

Link: http://www.cnblogs.com/skynet/archive/2010/12/25/1916873.html

Working in Linux, some commands can greatly improve efficiency. This article introduces the find and grep commands, which are essential Linux commands that I use almost every day. The structure of this article is as follows:

• find command

• General form of the find command

• Common options and examples of the find command

• find and xargs

• grep command

• General form of the grep command

• Basic set of regular expression metacharacters for grep

• Common options and examples of the grep command

1. find command

The find command is an omnipresent command and one of the most useful commands in Linux. The find command is used to search for files in a directory (and its subdirectories). You can specify matching criteria such as by filename, file type, user, or even timestamp. Let’s experience the power of the find command through examples.

1.1 General form of the find command

The general form of the find command given in the man documentation is:

find [-H] [-L] [-P] [-D debugopts] [-Olevel] [path…] [expression]

In fact, the options [-H] [-L] [-P] [-D debugopts] [-Olevel] are not commonly used (at least not in my daily work), so the common form of the find command can be simplified to:

find [path…] [expression]

• path: The directory path that the find command searches. For example, use . to represent the current directory and / to represent the system root directory.

• expression: The expression can be divided into -options [-print -exec -ok …]

• -options: Specifies the common options for the find command, which will be introduced in detail in the next section.

• -print: The find command outputs the matching files to standard output.

• -exec: The find command executes the shell command specified by this parameter on the matching files. The corresponding command format is ‘command’ { } ;, note the space between { } and ;. For example, find ./ -size 0 -exec rm {} ; deletes files with a size of zero (it can also be done like this: rm -i find ./ -size 0 or find ./ -size 0 | xargs rm -f &). To list the matched files using the ls -l command, you can place the ls -l command in the find command’s -exec option: find . -type f -exec ls -l { } ;. To find and delete files in the /logs directory that were modified more than 5 days ago: find /logs -type f -mtime +5 -exec rm { } ;.

• -ok: Functions the same as -exec, but executes the shell command specified by this parameter in a safer mode, prompting the user for confirmation before executing each command. For example, find . -name “*.conf” -mtime +5 -ok rm { } ; searches for all files ending with .LOG in the current directory that were modified more than 5 days ago and prompts before deleting them.

Some people summarize the structure of the find command as follows:

find start_directory test

options

criteria_to_match

action_to_perform_on_results

1.2 Common options and examples of the find command

• -name: Search for files by filename. find /dir -name filename searches for a file named filename in the /dir directory and its subdirectories. find . -name “*.c” searches for any files with a .c extension in the current directory and its subdirectories (represented by “.”).

• -perm: Search for files by file permissions. find . -perm 755 -print searches for files with permission bits set to 755 in the current directory, meaning the file owner can read, write, and execute, while other users can read and execute the file.

• -prune: This option allows the find command to not search in the specified directory. If used with the -depth option, then -prune will be ignored by the find command. find /apps -path “/apps/bin” -prune -o -print searches for files in the /apps directory but does not search in the /apps/bin directory. find /usr/sam -path “/usr/sam/dir1” -prune -o -print searches for all files in the /usr/sam directory that are not within the dir1 subdirectory.

• -user: Search for files by file owner. find ~ -user sam -print searches for files owned by user sam in the $HOME directory.

• -group: Search for files by file group. find /apps -group gem -print searches for files belonging to the gem user group in the /apps directory.

• -mtime -n +n: Search for files by modification time, where -n indicates files modified within the last n days, and +n indicates files modified more than n days ago. find / -mtime -5 -print searches for files modified within the last 5 days in the system root directory. find /var/adm -mtime +3 -print searches for files modified more than 3 days ago in the /var/adm directory.

• -nogroup: Search for files without a valid group, meaning the file’s group does not exist in /etc/groups. find / -nogroup -print

• -nouser: Search for files without a valid owner, meaning the file’s owner does not exist in /etc/passwd. find /home -nouser -print

• -newer file1 ! file2: Search for files that are newer than file1 but older than file2.

• -type: Search for a specific type of file, such as: b – block device file. d – directory. c – character device file. p – pipe file. l – symbolic link file. f – regular file. find /etc -type d -print searches for all directories in the /etc directory. find . ! -type d -print searches for all types of files except directories in the current directory. find /etc -type l -print searches for all symbolic link files in the /etc directory.

• -size n:[c] Search for files with a length of n blocks, where c indicates the file length in bytes. find . -size +1000000c -print searches for files larger than 1 MB in the current directory. find /home/apache -size 100c -print searches for files exactly 100 bytes in length in the /home/apache directory. find . -size +10 -print searches for files larger than 10 blocks in the current directory (one block equals 512 bytes).

• -depth: When searching for files, first search for files in the current directory, then in its subdirectories. find / -name “CON.FILE” -depth -print will match all files first and then enter subdirectories to search.

• -mount: When searching for files, do not cross filesystem mount points. find . -name “*.XC” -mount -print searches for files ending with XC in the current filesystem starting from the current directory (without entering other filesystems).

• -follow: If the find command encounters a symbolic link file, it will follow the link to the file it points to.

1.3 find and xargs

When using the -exec option of the find command to process matched files, the find command passes all matched files to exec for execution. However, some systems have a limit on the length of commands that can be passed to exec, which can lead to overflow errors after the find command runs for a few minutes. The error message is usually “argument list too long” or “argument list overflow”. This is where the xargs command comes in handy, especially when used with the find command.

The find command passes the matched files to the xargs command, which only takes a portion of the files at a time instead of all of them, unlike the -exec option. This way, it can first process the first batch of files, then the next batch, and so on.

In some systems, using the -exec option will initiate a corresponding process for each matched file, rather than executing all matched files as parameters at once; this can lead to too many processes and decreased system performance, thus being inefficient.

Using the xargs command, however, only one process is created. Additionally, when using the xargs command, whether to obtain all parameters at once or in batches, and how many parameters to obtain each time, will depend on the options of the command and the corresponding adjustable parameters in the system kernel.

Let’s see how the xargs command is used together with the find command, along with some examples.

find . -type f -print | xargs file searches for every regular file in the system and then uses the xargs command to test which category they belong to.

find / -name “core” -print | xargs echo “” >/tmp/core.log searches for memory dump files (core dump) throughout the system and saves the results to /tmp/core.log.

find . -type f -print | xargs grep “hostname” searches for the word hostname in all regular files.

find ./ -mtime +3 -print | xargs rm -f -r deletes everything older than 3 days (find . -ctime +3 -exec rm -rf {} ;).

find ./ -size 0 | xargs rm -f & deletes files with a size of zero.

The find command, when used with exec and xargs, allows users to execute almost any command on the matched files.

2. grep command

grep (global search regular expression (RE) and print out the line) is a powerful text search tool that can search text using regular expressions and print the matching lines.

2.1 General options and examples of the grep command

grep [OPTIONS] PATTERN [FILE…] grep [OPTIONS] [-e PATTERN | -f FILE] [FILE…]

The grep command is used to search for the pattern specified by the Pattern parameter and write each matching line to standard output. These patterns are limited regular expressions that use ed or egrep command styles. If multiple names are specified in the File parameter, the grep command will display the names of the files containing matching lines. Characters with special meanings in the shell ($, *, [, |, ^, (, ), ) must be enclosed in double quotes when they appear in the Pattern parameter.

If the Pattern parameter is not a simple string, the entire pattern usually needs to be enclosed in single quotes. In expressions like [a-z], the – (hyphen) can specify a range based on the current sorting sequence. The sorting sequence can define equivalent classes for use in character ranges. If no file is specified, grep assumes standard input.

2.2 Basic set of regular expression metacharacters for grep

^ Anchors the start of a line, e.g., ‘^grep’ matches all lines starting with grep.

$ Anchors the end of a line, e.g., ‘grep$’ matches all lines ending with grep.

. Matches a single character that is not a newline, e.g., ‘gr.p’ matches gr followed by any character and then p.

* Matches zero or more of the preceding character, e.g., ‘ *grep’ matches all lines with one or more spaces followed by grep. .* together represents any character.

[] Matches a character within a specified range, e.g., ‘[Gg]rep’ matches both Grep and grep.

[^] Matches a character not in the specified range, e.g., ‘[^A-FH-Z]rep’ matches a line starting with a letter not in A-F and H-Z, followed by rep.

(..) Marks matching characters, e.g., ‘(love)’ marks love as 1.

> Anchors the end of a word, e.g., ‘grep>’ matches lines containing words ending with grep.

x{m} Matches character x repeated m times, e.g., ‘o{5}’ matches lines containing five consecutive o’s.

x{m,} Matches character x repeated at least m times, e.g., ‘o{5,}’ matches lines with at least five consecutive o’s.

x{m,n} Matches character x repeated at least m times but no more than n times, e.g., ‘o{5,10}’ matches lines with five to ten consecutive o’s.

w Matches a word and digit character, i.e., [A-Za-z0-9], e.g., ‘Gw*p’ matches G followed by zero or more word or digit characters and then p.

W The inverse of w, matches a non-word character, such as a period. W* can match multiple.

b Word boundary, e.g., ‘bgrepb’ matches only the word grep, with spaces on both sides.

2.3 Common options and examples of the grep command

-?

Simultaneously display the matching line and the lines above and below it, e.g., grep -2 pattern filename displays the two lines above and below the matching line.

-b, –byte-offset

Print the block number of the line before the matching line.

-c,–count

Only print the number of matching lines, without showing the matching content.

-f File,–file=File

Extract templates from a file. An empty file contains 0 templates, so nothing matches.

-h,–no-filename

When searching multiple files, do not display the matching file name prefix.

-i,–ignore-case

Ignore case differences.

-q,–quiet

Suppress output, only return the exit status. 0 indicates that matching lines were found.

-l,–files-with-matches

Print a list of files with matching templates.

-L,–files-without-match

Print a list of files without matching templates.

-n,–line-number

Print the line number before the matching line.

-s,–silent

Suppress error messages about nonexistent or unreadable files.

-v,–revert-match

Reverse search, only display non-matching lines.

-w,–word-regexp

If quoted, treat the expression as a word search.

-V,–version

Display software version information.

=====

ls -l | grep ‘^a’ filters the output of ls -l through a pipe, displaying only lines starting with a.

grep ‘test’ d* displays all lines containing test in files starting with d.

grep ‘test’ aa bb cc displays lines matching test in the files aa, bb, and cc.

grep ‘[a-z]’ aa displays all lines containing at least five consecutive lowercase characters.

grep ‘w(es)t.*’ aa if west is matched, then es is stored in memory and marked as 1, then searches for any characters (.*) followed by another es(), displaying that line. If using egrep or grep -E, you can write it as ‘w(es)t.*’ without quotes.

grep -i pattern files: searches without case sensitivity. By default, it is case-sensitive.

grep -l pattern files: only lists the names of matching files.

grep -L pattern files: lists the names of non-matching files.

grep -w pattern files: only matches whole words, not parts of strings (e.g., matches ‘magic’, not ‘magical’).

grep -C number pattern files: displays [number] lines of context around the match.

grep pattern1 | pattern2 files: displays lines matching either pattern1 or pattern2.

grep pattern1 files | grep pattern2: displays lines matching both pattern1 and pattern2.

References:

• Detailed introduction to using the Linux grep command: http://fanqiang.chinaunix.net/system/linux/2007-03-15/5110.shtml

• Detailed explanation of the Linux file search commands find and xargs: http://www.linuxsir.org/main/?q=node/137#1.1

• man documentation (man find, man grep)

Follow “Linux Enthusiasts”

See more Linux technical sharing

↓↓↓