▼ Click the card below to follow me
▲ Click the card above to follow me
The Python multiprocessing library, Multiprocessing, is simply a savior for parallel computing! Imagine you have a bunch of compute-intensive tasks; the traditional single-threaded approach is like a snail climbing a mountain, while Multiprocessing is like installing a turbocharger.
Basic Concepts of Multiprocessing
Each process is like an independent little factory, with its own memory space and execution context. Process is the core class of this library, which helps you easily create new processes. Take a look at the code below:
from multiprocessing import Process
def worker(name):
print(f"Worker {name} has started working!")
if __name__ == '__main__':
p1 = Process(target=worker, args=('Xiao Ming',))
p2 = Process(target=worker, args=('Xiao Hong',))
p1.start()
p2.start()
p1.join()
p2.join()What does this code mean? It starts two processes simultaneously, with two “workers” working in parallel, haha~
Process Pool: Batch Management of Processes
Sometimes we need to handle multiple tasks at the same time, and Pool comes to the rescue!
from multiprocessing import Pool
def calc_square(x):
return x * x
if __name__ == '__main__':
with Pool(processes=4) as pool:
results = pool.map(calc_square, [1, 2, 3, 4, 5])
print(results) # [1, 4, 9, 16, 25]The Pool is like a process factory, automatically allocating and managing processes for you.
Inter-Process Communication: Queue and Pipe
Processes need to communicate with each other, and Queue and Pipe are great helpers. Queue is like a secure mailbox where multiple processes can put in and take out data.
from multiprocessing import Process, Queue
def sender(queue):
queue.put("Hey, receiving process!")
def receiver(queue):
msg = queue.get()
print(f"Received message: {msg}")
if __name__ == '__main__':
q = Queue()
p1 = Process(target=sender, args=(q,))
p2 = Process(target=receiver, args=(q,))
p1.start()
p2.start()
p1.join()
p2.join()Process Synchronization: Lock
Simultaneous access to shared resources by multiple processes can cause problems, and Lock can help you secure it.
from multiprocessing import Process, Lock
def deposit(balance, lock):
with lock:
balance.value += 10
if __name__ == '__main__':
from multiprocessing import Value
balance = Value('i', 100)
lock = Lock()
processes = [Process(target=deposit, args=(balance, lock)) for _ in range(5)]
for p in processes:
p.start()
for p in processes:
p.join()
print(f"Final balance: {balance.value}")🔥 Friendly reminder: Processes are safer than threads because they have independent memory spaces, making it less likely to encounter race conditions.
Key Points Summary ：
- Process creates processes
- Pool manages processes in batches
- Queue facilitates inter-process communication
- Lock protects shared resources
Mastering multiprocessing will significantly boost the efficiency of your Python code!
Like and share
Let money and love flow to you