Understanding the Execution Mechanism of find . -exec ... {} + in Linux

Continuing from the previous article on the execution mechanism of find . -exec ... {} ;.

This article mentions two usage methods of -exec:

# Ends with 
find . -name "*.txt" -exec ls -l {} 
# Or ends with +
find . -name "*.txt" -exec ls -l {} +

man manual states that the command must end with ; or +.

; needs to be escaped (written as “;” or ‘;’),
while + does not need to be escaped.

1. Using ;: Executes the command once for each matching file found.

find . -name "*.txt" -exec ls -l {}

2. Using +: Passes as many matching files as possible as arguments to the command at once, similar to xargs. This greatly improves efficiency by reducing the number of process startups.

find . -name "*.txt" -exec cp {} /backup/ +

# This command will execute like this: cp file1.txt file2.txt ... fileN.txt /backup/

Then the question arises:

A reader pointed out:find . -name "*.txt" -exec cp {} /backup/ + executed with an error.
Error message: find: -exec: no terminating ";" or "+

Honestly, I was a bit confused when I saw the reader’s feedback. I thought: the command find . -name "*.txt" -exec ls -l {} + has no issues, and after specifying -I {} in xargs, {} can be placed anywhere… I quickly opened my computer to test it, and indeed there was a problem! Please forgive me, this command was written based on existing knowledge, and I had never executed it…

Over the past few days, I looked for the reason. Please continue reading.

1. The basic commands of macOS come from the `BSD` version

Linux commonly uses commands like ls, mv, mv,dir, du, etc., which come from the GNU coreutils^[1] software package. However, the basic commands included with macOS are derived from the BSD version of the tools. They indeed have significant differences.

First, let’s look at the description of the -exec option in the man find command on macOS.

-exec utility [argument ...] {} +
    Same as -exec, except that "{}" is replaced with as many pathnames as possible for each invocation of
    utility.  This behaviour is similar to that of xargs(1).  The primary always returns true; if at least
    one invocation of utility returns a non-zero exit status, find will return a non-zero exit status.

Next, let’s check the description of find^[2] in the online freeBSD man^[3] manual.

-exec utility [argument ...] {} +
    Same as -exec, except that "{}" is replaced with as many pathnames as possible for each invocation of utility.  This behaviour is similar to that of xargs(1).  The primary always returns true; if at least one invocation of utility returns a non-zero exit status, find will return a non-zero exit status.

Indeed, it is the same as freeBSD, indicating that they are homologous. Moreover, the symbol {} + must be placed at the end.

Next, I looked at the online manual for archlinux man find^[4]:

-exec command {} +
    This variant of the -exec action runs the specified command on the selected files,
    but the command line is built by appending each selected file name at the end;
    the total number of invocations of the command will be much less than the number of matched files.
    The command line is built in much the same way that xargs builds its command lines.
    Only one instance of '{}' is allowed within the command, and it must appear at the end, immediately before the `+';
    it needs to be escaped (with a `\') or quoted to protect it from interpretation by the shell.
    The command is executed in the starting directory.
    If any invocation with the `+' form returns a non-zero value as exit status, then find returns a non-zero exit status.
    If find encounters an error, this can sometimes cause an immediate exit, so some pending commands may not be run at all.
    For this reason -exec my-command ... {} + -quit may not result in my-command actually being run.
    This variant of -exec always returns true.

Finally, I checked the online manual for ubuntu man find^[5]:

-exec command {} +
    This variant of the -exec action runs the specified command on the selected files, but the command
    line is built by appending each selected file name at the end; the total number of invocations of
    the command will be much less than the number of matched files.  The command line is built in much
    the same way that xargs builds its command lines.  Only one instance of '{}' is allowed within the
    command.  The command is executed in the starting directory.  If find encounters an error, this
    can sometimes cause an immediate exit, so some pending commands may not be run at all.  This
    variant of -exec always returns true.

Finally, I checked the manual of the findutils^[6] software package:

Action: -execdir command {} +
    This works as for ‘-execdir command ;’,
    except that the result is always true, and the '{}' at the end of the command is expanded to a list of names of matching files.
    This expansion is done in such a way as to avoid exceeding the maximum command line length available on the system.
    Only one '{}' is allowed within the command, and it must appear at the end, immediately before the '+' .
    A '+' appearing in any position other than immediately after ‘{}’ is not considered to be special (that is, it does not terminate the command).

Action: -exec command {} +
    This insecure variant of the ‘-execdir’ action is specified by POSIX.
    The main difference is that the command is executed in the directory from which find was invoked,
    meaning that ‘{}’ is expanded to a relative path starting with the name of one of the starting directories,
    rather than just the basename of the matched file. The result is always true.

Finally, I found a definitive description, both the archlinux manual page and the findutils manual page mention it (highlighted in red).

We take the findutils manual page as the benchmark:

Only one ‘{}’ is allowed within the command, and it must appear at the end, immediately before the ‘+’. A ‘+’ appearing in any position other than immediately after ‘{}’ is not considered to be special (that is, it does not terminate the command).

Translated:

Only one ‘{}’ is allowed within the command, and it must appear at the end, immediately before the ‘+’. A ‘+’ appearing in any position other than immediately after ‘{}’ is not considered to be special (that is, it does not terminate the command).

Thus, the usage of -exec command {} + is very clear.

# The filename of ls is just at the end, so there is no problem.
find . -name "*.txt" -exec ls -l {} +

# Incorrect version: `find . -name "*.txt" -exec cp {} /backup/ +`
# Below is the correct version
find . -name "*.txt" -exec cp -t /backup/ {} +

There is also a pitfall here:macOS does not have the -t option for the cp command. So macOS users should still combine find + xargs.

2. How to use GNU coreutils on macOS

Since the basic commands of macOS come from the BSD version, how can we use the GNU coreutils toolkit?

It’s simple, install it via brew:

brew install coreutils

Here, the Homebrew Formulae coreutils^[7] mentions the issue of name conflicts.

The article mentions:

Commands also provided by macOS and the commands dir, dircolors, vdir have been installed with the prefix “g”. If you need to use these commands with their normal names, you can add a “gnubin” directory to your PATH with: PATH="$HOMEBREW_PREFIX/opt/coreutils/libexec/gnubin:$PATH"

macOS has built-in commands, and the dir, dircolors, vdir commands are now installed as versions with a "g" prefix. To use these commands with their native names, you can add the "gnubin" directory to the PATH environment variable: PATH="$HOMEBREW_PREFIX/opt/coreutils/libexec/gnubin:$PATH"

3. About the `-t` option of the cp command

As mentioned above, the cp command on macOS does not have the -t option. Therefore, I specifically browsed the GNU cp manual^[8] which mentions a link:Target directory^[9], which specifically discusses the origin of the -t option.

The article mentions:

The cp, install, ln, and mv commands normally treat the last operand specially when it is a directory or a symbolic link to a directory. For example, ‘cp source dest’ is equivalent to ‘cp source dest/source’ if dest is a directory. Sometimes this behavior is not exactly what is wanted, so these commands support the following options to allow more fine-grained control.

Normally, when the last operand is a directory or a symbolic link to a directory, the cp, install, ln, and mv commands treat it specially. For example, if the target path dest is a directory, then cp source dest is equivalent to cp source dest/source. Sometimes this default behavior does not fully meet operational needs, so these commands support the following options (referring to -t and -T) for more precise control.

Finally, those engaged in technology cannot be careless.

Thanks to the readers and friends.

References

[1]

GNU coreutils: https://www.gnu.org/software/coreutils/manual/

[2]

freeBSD man: https://man.freebsd.org/cgi/man.cgi

[3]

archlinux man find: https://man.archlinux.org/man/find.1

[4]

ubuntu man find: https://manpages.ubuntu.com/manpages/xenial/en/man1/find.1.html

[5]

findutils: https://www.gnu.org/software/findutils/manual/html_node/find_html/Multiple-Files.html

[6]

Homebrew Formulae coreutils: https://formulae.brew.sh/formula/coreutils

[7]

GNU cp manual: https://www.gnu.org/software/coreutils/manual/html_node/cp-invocation.html

[8]

Target directory: https://www.gnu.org/software/coreutils/manual/html_node/Target-directory.html

Understanding the Execution Mechanism of find . -exec … {} + in Linux

1. The basic commands of macOS come from the `<span>BSD</span>` version

2. How to use GNU coreutils on macOS

3. About the `<span>-t</span>` option of the cp command

References

Leave a Comment Cancel reply

1. The basic commands of macOS come from the <span>BSD</span> version

2. How to use GNU coreutils on macOS

3. About the <span>-t</span> option of the cp command

References

Related posts

Leave a Comment Cancel reply

1. The basic commands of macOS come from the `<span>BSD</span>` version

3. About the `<span>-t</span>` option of the cp command