Guide to File Archiving and Compression Management in Linux

1. Basic Concepts of Archiving and Compression(1) Basic Concepts

  • Archiving: Packaging multiple files/directories into a single file without reducing size

  • Compression: Reducing file size through algorithms to save storage space and transmission time

  • Common Linux Combinations: Archive first, then compress

(2) Comparison of Common Formats

Format Type Characteristics Applicable Scenarios
.tar Archiving Only packages, does not compress Backup of file collections
.gz Compression (1) gzip compression, fast speed(2) Can operate on files and directories (when operating on directories, only compresses each file within the directory, not the directory itself)(3) By default, the original file is not retained during compression or decompression Text files, web resources
.bz2 Compression (1) bzip2 compression, high compression ratio(2) Can only operate on files, does not support directory operations(3) Use the -k option to retain the original file without deleting it after compression Large files, database backups
.xz Compression (1) xz compression, highest compression ratio(2) Can only operate on files, does not support directory operations(3) Use the -k option to retain the original file without deleting it after compression Software releases, system images
.zip Archiving + Compression Cross-platform compatibility Windows/Linux sharing
.7z Archiving + Compression High compression ratio Scenarios requiring extreme compression

2. Detailed Explanation of the tar Command(1) Basic Syntax

tar [options] archive_filename files/directories_to_archive

(2) Common Option Descriptions

# Create related -c  # Create a new archive -f  # Specify archive filename -v  # Show detailed process -p  # Preserve file permission attributes --exclude  # Exclude files/directories
# View related -t  # List archive contents -r  # Append files to archive -u  # Update files in the archive
# Extract related -x  # Extract archive files -C  # Specify extraction directory -k  # Do not overwrite existing files

(3) Basic tar Operation ExamplesExample 1: Create Archive

# Basic archiving $ tar -cvf project.tar /home/project/
# Archive and retain permissions $ tar -cpvf backup.tar /etc/
# Exclude specific files $ tar -cvf data.tar --exclude="*.tmp" /var/data/
# Archive multiple directories $ tar -cvf system_backup.tar /etc /home /var/log

Example 2: View Archive Contents

# List archive contents $ tar -tf project.tar
# List archive contents in detail $ tar -tvf project.tar
drwxr-xr-x root/root         0 2024-01-15 10:00 home/project/
-rw-r--r-- root/root      1024 2024-01-15 09:30 home/project/file1.txt
-rw-r--r-- root/root      2048 2024-01-15 09:35 home/project/file2.conf
# Search for specific files in the archive $ tar -tf project.tar | grep "config"

Example 3: Extract Archive

# Extract to current directory $ tar -xvf project.tar
# Extract to specified directory $ tar -xvf project.tar -C /tmp/extract/
# Extract specific file $ tar -xvf project.tar home/project/file1.txt
# Extract without overwriting existing files $ tar -xkvf project.tar
# Interactive extraction (ask if overwrite) $ tar -xvf project.tar -w

3. Detailed Explanation of Compression Tools(1) gzip Compression

# Compress file (original file will be deleted) $ gzip file.txt $ ls file.txt.gz
# Retain original file during compression $ gzip -c file.txt > file.txt.gz
# Decompress $ gzip -d file.txt.gz $ gunzip file.txt.gz
# Show compression information $ gzip -l file.txt.gz
compressed uncompressed ratio uncompressed_name       1024        2048  50.0% file.txt
# Force compression (even if larger) $ gzip -f file.txt
# Best compression rate (slow speed) $ gzip -9 file.txt
# Fastest compression (lower compression rate) $ gzip -1 file.txt

(2) bzip2 Compression

# Compress file $ bzip2 file.txt $ ls file.txt.bz2
# Retain original file during compression $ bzip2 -c file.txt > file.txt.bz2
# Decompress $ bzip2 -d file.txt.bz2 $ bunzip2 file.txt.bz2
# Test compression file integrity $ bzip2 -t file.txt.bz2
# Show compression information $ bzip2 -v file.txt
file.txt: 2.100:1, 3.810 bits/byte, 52.38% saved, 2048 in, 975 out.
# Set compression level (1-9, default 9) $ bzip2 -5 file.txt

(3) xz Compression

# Compress file $ xz file.txt $ ls file.txt.xz
# Retain original file during compression $ xz -c file.txt > file.txt.xz
# Decompress $ xz -d file.txt.xz $ unxz file.txt.xz
# Multi-threaded compression (utilizing multi-core CPU) $ xz -T4 file.txt
# Set compression level (0-9, default 6) $ xz -9 file.txt
# Show compression progress $ xz -v file.txt
file.txt (1/1) 100 % 97.5 KiB / 200.0 KiB = 0.487

4. Using tar with Compression(1) Automatic Compression Options

# gzip compression (.tar.gz or .tgz) $ tar -czvf project.tar.gz /home/project/
# bzip2 compression (.tar.bz2 or .tbz) $ tar -cjvf project.tar.bz2 /home/project/
# xz compression (.tar.xz or .txz) $ tar -cJvf project.tar.xz /home/project/
# Corresponding extraction commands $ tar -xzvf project.tar.gz $ tar -xjvf project.tar.bz2 $ tar -xJvf project.tar.xz

(2) Compression Level Control

# Control compression level using pipe $ tar -cf - /home/project/ | gzip -9 -c > project.tar.gz
$ tar -cf - /home/project/ | bzip2 -9 -c > project.tar.bz2
$ tar -cf - /home/project/ | xz -9 -c > project.tar.xz

(3) Performance Comparison Example

# Create test file $ dd if=/dev/zero of=testfile bs=1M count=100
100+0 records in
100+0 records out
104857600 bytes (105 MB, 100 MiB) copied, 0.082321 s, 1.3 GB/s
# Test different compression methods $ time tar -czf testfile.tar.gz testfile
real    0m1.234s
user    0m1.120s
sys     0m0.114s
$ time tar -cjf testfile.tar.bz2 testfile
real    0m2.456s
user    0m2.340s
sys     0m0.116s
$ time tar -cJf testfile.tar.xz testfile
real    0m5.678s
user    0m5.560s
sys     0m0.118s
# View compression results $ ls -lh testfile
-rw-r--r-- 1 user user 100M Jan 15 10:00 testfile
-rw-r--r-- 1 user user 101K Jan 15 10:01 testfile.tar.gz
-rw-r--r-- 1 user user 99K  Jan 15 10:01 testfile.tar.bz2
-rw-r--r-- 1 user user 97K  Jan 15 10:01 testfile.tar.xz

5. zip and unzip Commands(1) zip Compression

# Basic compression $ zip archive.zip file1.txt file2.txt
# Recursively compress directory $ zip -r project.zip /home/project/
# Split volume compression (10M per volume) $ zip -r -s 10m project.zip /home/project/
# Encrypted compression $ zip -e secret.zip confidential.txt
Enter password: Verify password: 
# Exclude files $ zip -r project.zip /home/project/ -x "*.tmp" "*.log"
# Set compression level (0-9) $ zip -r -9 maximum.zip /home/project/

(2) unzip Decompression

# Basic decompression $ unzip archive.zip
# Decompress to specified directory $ unzip archive.zip -d /tmp/extract/
# List contents of zip file $ unzip -l archive.zip
# Test integrity of zip file $ unzip -t archive.zip
# Decompress encrypted file $ unzip -P password secret.zip
# Decompress without overwriting existing files $ unzip -n archive.zip
# Quiet mode decompression $ unzip -q archive.zip

6. 7z High Compression Rate Tool(1) 7z Installation and Usage

# Install on Ubuntu/Debian $ sudo apt install p7zip-full
# Install on CentOS/RHEL $ sudo yum install p7zip
# Basic compression $ 7z a archive.7z /home/project/
# Decompression $ 7z x archive.7z
# List contents of zip file $ 7z l archive.7z
# Set compression level (0-9) $ 7z a -mx=9 maximum.7z /home/project/
# Split volume compression $ 7z a -v10m project.7z /home/project/
# Encrypted compression $ 7z a -p password secret.7z confidential.txt

7. Practical Application Scenarios(1) System Backup Script

#!/bin/bash
# System important file backup script
BACKUP_DIR="/backup"
DATE=$(date +%Y%m%d_%H%M%S)
BACKUP_FILE="system_backup_${DATE}.tar.gz"
# Backup important directories
tar -czpf "${BACKUP_DIR}/${BACKUP_FILE}" \
    --exclude="/var/cache" \
    --exclude="/tmp" \
    /etc /home /var/www /usr/local
# Log backup information
echo "Backup created: ${BACKUP_FILE}" >> "${BACKUP_DIR}/backup.log"
echo "Size: $(du -h "${BACKUP_DIR}/${BACKUP_FILE}" | cut -f1)" >> "${BACKUP_DIR}/backup.log"
# Delete backups older than 7 days
find "${BACKUP_DIR}" -name "system_backup_*.tar.gz" -mtime +7 -delete

(2) Website Data Backup

#!/bin/bash
# Website data and database backup
BACKUP_DIR="/website_backup"
DATE=$(date +%Y%m%d)
MYSQL_USER="backup_user"
MYSQL_PASS="password"
# Backup website files
tar -czf "${BACKUP_DIR}/web_files_${DATE}.tar.gz" /var/www/html
# Backup MySQL database
mysqldump -u"${MYSQL_USER}" -p"${MYSQL_PASS}" --all-databases | 
gzip > "${BACKUP_DIR}/mysql_full_${DATE}.sql.gz"
# Backup PostgreSQL database
pg_dumpall | gzip > "${BACKUP_DIR}/pgsql_full_${DATE}.sql.gz"
# Verify backup files
gzip -t "${BACKUP_DIR}/web_files_${DATE}.tar.gz" && 
gzip -t "${BACKUP_DIR}/mysql_full_${DATE}.sql.gz"

(3) Log File Archiving

#!/bin/bash
# Log file compression archiving script
LOG_DIR="/var/log"
ARCHIVE_DIR="/var/log/archive"
DATE=$(date +%Y%m)
# Compress last month's logs
find "${LOG_DIR}" -name "*.log" -mtime +30 | while read logfile; do    # Get filename (without path)
    filename=$(basename "${logfile}")
    # Create compressed file    tar -czf "${ARCHIVE_DIR}/${filename}_${DATE}.tar.gz" -C "${LOG_DIR}" "${filename}"
    # Clear original log file (service continues to write)
    > "${logfile}"
    echo "Archived: ${filename}"
done
# Delete archives older than 3 months
find "${ARCHIVE_DIR}" -name "*.tar.gz" -mtime +90 -delete

8. Advanced Techniques and Troubleshooting(1) Incremental and Differential Backups

# Create full backup $ tar -czpf full_backup_20240101.tar.gz --listed-incremental=snapshot.snar /data
# Create incremental backup (based on previous snapshot) $ tar -czpf inc_backup_20240102.tar.gz --listed-incremental=snapshot.snar /data
# Create differential backup (based on full backup) $ tar -czpf diff_backup_20240108.tar.gz -N full_backup_20240101.tar.gz /data

(2) Splitting and Merging Large Files

# Split large file (100M per chunk) $ split -b 100M large_file.tar.gz large_file_part_
# Merge split files $ cat large_file_part_* > large_file_restored.tar.gz
# Verify file integrity $ md5sum large_file.tar.gz large_file_restored.tar.gz

(3) Repairing Compressed Files

# Attempt to repair corrupted zip file $ zip -F corrupted.zip --out repaired.zip
# Use 7z to attempt repair $ 7z t corrupted.7z
# Extract as much as possible from corrupted tar file $ tar -xvf corrupted.tar --ignore-failed-read

(4) Performance Optimization Techniques

# Use pigz instead of gzip (multi-threaded compression) $ tar -cf - /large_directory | pigz -p 8 > backup.tar.gz
# Use pbzip2 instead of bzip2 (multi-threaded compression) $ tar -cf - /large_directory | pbzip2 -p4 -c > backup.tar.bz2
# Use lz4 for fast compression (speed priority) $ tar -cf - /data | lz4 - backup.tar.lz4

9. Practical Summary(1) Compression Tool Selection Guide

  • Speed Priority: gzip > bzip2 > xz

  • Compression Ratio Priority: xz > bzip2 > gzip

  • Memory Usage: gzip < bzip2 < xz

  • Compatibility: zip > gzip > bzip2 > xz

(2) Recommendations for Production Environments

# 1. Regularly verify backup integrity $ tar -tf backup.tar.gz > /dev/null && echo "Backup OK"
# 2. Retain multiple backup versions
# 3. Test recovery process
# 4. Monitor backup job status
# 5. Encrypt sensitive data backups

Related posts

  1. CS Configuration for Linux Deployment
  2. 25 Linux Server Security Tips You Can Always Use!
  3. Introduction to Linux System Users and Permissions
  4. Emergency Guide for Linux System Crashes: A Complete Practical Process from Core Dump Configuration to GDB Debugging
  5. 15 Essential Linux Commands for Programmers: A Guide to Efficiency Tools
  6. In-Depth Understanding of Linux Disk Partitioning: Enterprise-Level Partitioning Practices
  7. Goodbye Photoshop! Nano Banana Enables Everyone to Master Professional Image Editing
  8. Commands for Mounting Disks in Linux
  9. Detailed Explanation of Linux Program Compilation Process
  10. C++ Lecture 3: The ‘Secret Symbols’ that Drive Code Execution
  11. How to Say Wi-Fi Password in English
  12. In-Depth Analysis of Linux Processes (6): Low-Level Implementation of Resource Isolation (Namespaces, Cgroups, and Containerization)
  13. Linux | 17. Output Redirection
  14. An Overview of Linux Command Line Tools
  15. Linux Dressed in Windows 11: A Stunning Revelation!
  16. The Notification Chain Mechanism in the Linux Kernel
  17. The Hidden Power of Bluetooth: Unveiling Practical Functions Beyond Wi-Fi
  18. Distributed Multi-Channel Wireless Extraction Module
  19. Deployment of Linux FTP/SFTP File Services

Leave a Comment