Advanced Python Programming Techniques

16. Decorators

Decorators are Python’s “syntactic sugar” used to add additional functionality to functions without modifying the original function’s code (such as logging, timing, permission validation, etc.).

1. Basic Decorators

# Define a decorator (essentially a function that takes another function as an argument)
def log_decorator(func):
    def wrapper(*args, **kwargs):
        print(f"Starting execution of function: {func.__name__}")  # Additional functionality
        result = func(*args, **kwargs)  # Execute the original function
        print(f"Function {func.__name__} execution finished")  # Additional functionality
        return result  # Return the original function's result
    return wrapper

# Use the decorator with @ syntax
@log_decorator
def add(a, b):
    return a + b

# Call the function (will automatically trigger the decorator's functionality)
print(add(3, 5))
# Output:
# Starting execution of function: add
# Function add execution finished
# 8

2. Decorators with Parameters

Decorators can also accept parameters to achieve more flexible functionality:

def log_level(level):
    # Outer function receives decorator parameters
    def decorator(func):
        def wrapper(*args, **kwargs):
            print(f"[{level}] Executing function: {func.__name__}")
            return func(*args, **kwargs)
        return wrapper
    return decorator

# Use a decorator with parameters
@log_level("INFO")
def multiply(a, b):
    return a * b

print(multiply(2, 4))  # Output: [INFO] Executing function: multiply → 8

Decorators are extremely common in frameworks (like Django, Flask) for routing registration, permission validation, etc.

17. Advanced Context Managers

In addition to using the <span>with</span> statement for file operations, we can also customize context managers to achieve automatic resource allocation and release (such as database connections, lock mechanisms, etc.).

1. Implementing Context Managers with Classes

Must implement <span>__enter__</span> (executed when entering the context) and <span>__exit__</span> (executed when exiting) methods:

class DatabaseConnection:
    def __init__(self, db_name):
        self.db_name = db_name

    def __enter__(self):
        # Connect to the database (executed when entering the context)
        print(f"Connected to database: {self.db_name}")
        return self  # Can return an object to the with statement

    def __exit__(self, exc_type, exc_val, exc_tb):
        # Close the database (executed when exiting the context, regardless of exceptions)
        print(f"Disconnected from database {self.db_name}")
        # Return True to indicate exception is handled, do not propagate
        return False


# Use the custom context manager
with DatabaseConnection("my_db") as db:
    print("Performing database operations...")
# Output:
# Connected to database: my_db
# Performing database operations...
# Disconnected from database my_db

2. Simplifying Implementation with <span>contextlib</span>

For simple scenarios, you can quickly define a context manager using the <span>contextlib.contextmanager</span> decorator:

from contextlib import contextmanager

@contextmanager
def timer():
    import time
    start = time.time()  # Record start time when entering
    yield  #  yield before is __enter__, after is __exit__
    end = time.time()    # Calculate elapsed time when exiting
    print(f"Elapsed time: {end - start:.2f} seconds")

# Use
with timer():
    # Simulate time-consuming operation
    for i in range(10000000):
        pass
# Output: Elapsed time: 0.35 seconds (exact value depends on the computer)

18. Iterators and Iterable Objects

In Python, the <span>for</span> loop can iterate over lists, strings, etc., essentially because they are iterable objects (implementing the <span>__iter__</span> method), while the iteration process is completed by iterators (implementing the <span>__next__</span> method).

1. Iterable Objects

Definition: An object that has a <span>__iter__</span> method that returns an iterator.

# Custom iterable object (generates numbers from 1 to n)
class NumberIterable:
    def __init__(self, n):
        self.n = n

    def __iter__(self):
        # Return an iterator
        return NumberIterator(self.n)


# Iterator (implements __next__ method)
class NumberIterator:
    def __init__(self, n):
        self.n = n
        self.current = 1

    def __next__(self):
        if self.current <= self.n:
            res = self.current
            self.current += 1
            return res
        else:
            # End of iteration, raise StopIteration
            raise StopIteration


# Usage: can be directly used in for loop
for num in NumberIterable(3):
    print(num)  # 1 2 3

2. Generators are Simplified Iterators

The generators learned earlier (functions with <span>yield</span>) are essentially a simplified version of iterators, automatically implementing the <span>__iter__</span> and <span>__next__</span> methods.

19. Module and Package Management

As the amount of code increases, it is necessary to organize code using modules (.py files) and packages (folders containing <span>__init__.py</span> files).

1. Package Structure

my_project/
├── utils/               # Utility package
│   ├── __init__.py      # Package initialization file (can be empty, indicates this is a package)
│   ├── math_utils.py    # Module 1: Math utilities
│   └── str_utils.py     # Module 2: String utilities
└── main.py              # Main program

2. Importing Packages / Modules

# Import in main.py
from utils.math_utils import add  # Import specific function
from utils import str_utils       # Import entire module
import utils.math_utils as mu     # Alias

print(add(2, 3))          # 5
print(str_utils.reverse("abc"))  # cba
print(mu.multiply(2, 3))  # 6

3. The Role of <span>__init__.py</span>

Can specify the interfaces exposed by the package in <span>__init__.py</span>, simplifying imports:

# utils/__init__.py
from .math_utils import add, multiply
from .str_utils import reverse

Then in main.py, you can directly import from the package:

from utils import add, reverse

20. Regular Expressions (Handling Complex Text)

Regular expressions are the “Swiss Army knife” for handling strings, used to match, find, and replace specific formats of text (such as email addresses, phone numbers, URLs, etc.).

Python supports regular expressions through the <span>re</span> module:

1. Basic Matching

import re

# Match phone numbers (simple rule: 11 digits, starting with 1)
phone_pattern = r"^1\d{10}$"  # r indicates a raw string to avoid escape issues

print(re.match(phone_pattern, "13812345678"))  # Match success (returns match object)
print(re.match(phone_pattern, "123456789"))    # None (match failed)

2. Common Methods

• <span>re.findall(pattern, string)</span>: Find all matching substrings
• <span>re.sub(pattern, repl, string)</span>: Replace matching substrings

# Extract all numbers from a string
text = "Today's temperature is 25℃, tomorrow it will be 30℃"
nums = re.findall(r"\d+", text)  # \d+ matches one or more digits
print(nums)  # ['25', '30']

# Replace sensitive words (replace with *)
text = "This is a bad word, and a worse word"
censored = re.sub(r"bad|worse", "***", text)
print(censored)  # This is a *** word, and a *** word

The syntax of regular expressions is quite rich (e.g., <span>\w</span> matches alphanumeric characters, <span>^$</span> matches the beginning and end, etc.), and requires practice in context.

21. JSON Data Processing

JSON is a cross-language data exchange format (similar to a dictionary key-value structure), and Python processes it through the <span>json</span> module:

1. Python Object ↔ JSON String

import json

# Convert Python dictionary to JSON string
data = {
    "name": "Xiao Ming",
    "age": 18,
    "hobbies": ["basketball", "programming"]
    }
json_str = json.dumps(data, ensure_ascii=False, indent=2)
# ensure_ascii=False: keep Chinese characters; indent=2: format output
print(json_str)
# Output:
# {
#   "name": "Xiao Ming",
#   "age": 18,
#   "hobbies": [
#     "basketball",
#     "programming"
#   ]
# }

# Convert JSON string to Python dictionary
data2 = json.loads(json_str)
print(data2["hobbies"][0])  # basketball

2. Reading and Writing JSON Files

# Write to JSON file
with open("data.json", "w", encoding="utf-8") as f:
    json.dump(data, f, ensure_ascii=False, indent=2)

# Read JSON file
with open("data.json", "r", encoding="utf-8") as f:
    data3 = json.load(f)
print(data3["name"])  # Xiao Ming