Introduction

Hello everyone! I am a loyal Python enthusiast, and today we will discuss a topic that is often overlooked but extremely important: memory management and garbage collection in Python. Python’s automatic memory management allows developers to focus on business logic, but without understanding the underlying mechanisms, it can lead to performance bottlenecks and even memory leaks.

Python’s memory management consists of three parts:

1. Reference Counting: Python tracks the usage of each object through reference counting.
2. Garbage Collection: When reference counting is insufficient to free memory, Python uses the garbage collection mechanism to clear unused objects.
3. Memory Allocator: Manages the memory allocation and release of Python objects.

The core of memory management relies on the sys and gc standard libraries, which are used to view reference counts and control garbage collection behavior.

Each Python object has a reference count that records how many variables or objects reference it. When the reference count reaches 0, the memory is released.

Example Code: Check Object Reference Count

We can use sys.getrefcount() to check the reference count of an object:

import sys

a = [1, 2, 3]
print("Initial reference count:", sys.getrefcount(a))

b = a  # Increase a reference
print("Reference count after increasing:", sys.getrefcount(a))

del b  # Delete a reference
print("Reference count after deletion:", sys.getrefcount(a))

Output:

Initial reference count: 2
Reference count after increasing: 3
Reference count after deletion: 2

Note: The initial reference count is 2 because sys.getrefcount() itself also increases the reference count by one.

In addition to reference counting, Python also uses garbage collection to clean up circular references and other complex scenarios. Garbage collection is controlled by the gc module, which can be manually checked and triggered.

Example Code: Manually Trigger Garbage Collection

import gc

# Create a circular reference
class Node:
    def __init__(self, value):
        self.value = value
        self.next = None

node1 = Node(1)
node2 = Node(2)
node1.next = node2
node2.next = node1  # Circular reference

# Check for unreachable objects
print("Unreachable objects before garbage collection:", len(gc.garbage))
gc.collect()  # Manually trigger garbage collection
print("Unreachable objects after garbage collection:", len(gc.garbage))

Output:

Unreachable objects before garbage collection: 0
Unreachable objects after garbage collection: 0

Even with circular references, the garbage collector can handle them correctly.

After understanding memory management, we can optimize our code through the following methods:

4.1 Use Weak References

Weak references allow you to reference an object without increasing its reference count, suitable for caching or temporary object management.

import weakref

class MyClass:
    pass

obj = MyClass()
weak_obj = weakref.ref(obj)

print("Does the object exist:", weak_obj() is not None)  # Check if the object still exists
del obj
print("Does the object exist:", weak_obj() is not None)

Output:

Does the object exist: True
Does the object exist: False

4.2 Avoid Global Variables and Circular References

Minimize global variables and unnecessary circular references to reduce garbage collection pressure.

5. Common Memory Management Tools

Python provides several convenient libraries for debugging and optimizing memory usage:

5.1 tracemalloc

Used to trace memory allocations and help locate memory leaks.

import tracemalloc

tracemalloc.start()

# Simulate memory allocation
x = [i for i in range(1000000)]

snapshot = tracemalloc.take_snapshot()
top_stats = snapshot.statistics('lineno')

for stat in top_stats[:5]:
    print(stat)

5.2 psutil

Checks runtime memory usage.

import psutil
import os

process = psutil.Process(os.getpid())
print("Memory usage:", process.memory_info().rss / 1024 ** 2, "MB")

Conclusion

Through this article, you have learned the basic principles of Python memory management, including reference counting, garbage collection, and memory optimization techniques. Mastering this knowledge will not only help you write efficient code but also avoid common issues like memory leaks. Now, try optimizing your code!