1. Core Purpose & Concepts

• Core Purpose:Piping is a mechanism that directly connects the standard output (stdout) of one command to the standard input (stdin) of another command, allowing multiple small and specialized commands to be chained together to build powerful and complex data processing pipelines.
• Core Terminology:
• Pipe <span>|</span>: A special operator in the shell that looks like a vertical line. Its function is to take the output of the command on its left and use it as the input for the command on its right.
• Standard Output (stdout): The default location where data is output after a command executes successfully, usually your terminal screen. Its file descriptor is <span>1</span>.
• Standard Input (stdin): The default source from which a command reads data, usually your keyboard input. Its file descriptor is <span>0</span>.
• <span>tee</span> (T-junction): A command that acts like a T-junction in a pipeline. It reads data from standard input, writes it to one or more files (saving a snapshot), and continues to pass it to standard output (allowing the pipeline to continue flowing).
• <span>xargs</span> (Extended Arguments): A command that reads data from standard input (usually multi-line text) and converts it into command line arguments, which are then passed to another command for execution. This solves the problem of “some commands do not accept standard input”.

2. Key Commands & Options

Basic Pipeline Structure

The basic syntax of a pipeline is to connect commands using the <span>|</span> character.

Bash

# command1's output will become command2's inputcommand1 [options] [arguments] | command2 [options] [arguments]

Key Command Comparison

Classic Command Examples

Bash

# Example 1: Use a pipe to extract the day of the week from the output of the 'date' command# Output of date -> stdin of cut# The cut command specifies space as the delimiter with '-d' and extracts the first fielddate | cut --delimiter=' ' --field=1

# Example 2: Use tee to save the full date information while continuing the pipeline to extract the day of the week# Output of date -> stdin of tee# Tee saves data to full_date.txt while passing data to stdout -> stdin of cutdate | tee full_date.txt | cut -d' ' -f1

# Example 3: Based on Example 2, redirect the final extracted day of the week to another file# The final '>' is standard redirection, terminating the pipeline.date | tee full_date.txt | cut -d' ' -f1 > day_of_week.txt

# Example 4: Demonstrate the effect of xargs - incorrect way# The echo command does not accept standard input, so piping directly to it will have no effect.date | echo "The date is:"# Output: The date is: (output of date is ignored)

# Example 5: Demonstrate the effect of xargs - correct way# xargs turns the output of 'date' into command line arguments for 'echo'date | xargs echo "The date is:"# Output: The date is: Sun Aug 10 22:36:21 AEST 2025 (or current date)

3. Practical Use Cases

• Piping (<span>|</span>):

• Log Analysis:<span>cat app.log | grep "ERROR" | wc -l</span> (Read the log file -> filter lines containing “ERROR” -> count lines, quickly calculate the number of errors).
• Process Management:<span>ps aux | grep "chrome"</span> (List all processes -> filter processes related to “chrome”).

• <span>tee</span>:

• Script Execution and Logging: When running a long installation or deployment script, using tee allows you to see the script output in real-time on the screen while fully logging all output to a log file for later auditing.

  <span>./long_install_script.sh | tee install.log</span>

• <span>xargs</span>:

• Batch File Operations: This is the classic application scenario for xargs. It is very useful when you want to perform the same operation (like rm, cp, chmod) on a large number of files (for example, files found by find or ls commands).

  Find all .tmp files and use xargs to pass their filenames as arguments to the rm command to delete them

  <span>find . -name "*.tmp" | xargs rm</span>

4. Common Pitfalls

• Using Standard Redirection in the Middle of a Pipeline (<span>></span>)

• Error:<span>date > full_date.txt | cut -d' ' -f1</span>
• Reason: The shell processes commands such that redirection takes precedence over pipes. Therefore, the output of <span>date</span> is completely redirected to the <span>full_date.txt</span> file, resulting in no data entering the pipeline, and the <span>cut</span> command receives no input.
• Correction: Use the <span>tee</span> command.<span>date | tee full_date.txt | cut -d' ' -f1</span>

• Attempting to Pipe Data to Commands that Do Not Accept Standard Input

• Error:<span>echo "myfile.txt" | rm</span>

• Reason:<span>rm</span>, <span>cp</span>, <span>echo</span>, <span>ls</span> and many other commands are designed to get their operating objects from command line arguments (arguments) rather than standard input (stdin). Piping them directly is ineffective.

• Correction: Use xargs as a “converter” to turn standard input into command line arguments.

  <span>echo "myfile.txt" | xargs rm</span>

• Confusing <span>tee</span> and Redirection

• <span>tee</span>‘s function: “peek and pass through”, meaning save a copy but continue the data flow.
• <span>></span>‘s function: “intercept and redirect”, meaning the endpoint of the data flow, breaking subsequent pipelines.
• Key Tip: If you want to save data and continue processing, use <span>tee</span>. If you only want to save the final result to a file, use <span>></span> at the end of the entire command.