Author: Bex T.
Translator: Zhao Jiankai
Proofreader: Li Hongjun
This article is about 1900 words long, and it is recommended to read in 7 minutes.
This article takes you through examples to deeply understand advanced features of Python.
You have been programming in Python for a while, writing scripts and solving various problems. Is your level outstanding? You might just be unknowingly taking advantage of Python’s advanced features.
From closures to context managers, this article provides a list of advanced features in Python. You might find yourself thinking, “I’ve been using this all along!”.
Even if these concepts are new to you, this excellent list can elevate your skills to a new level.
1. Scope
A key aspect of advanced Python programming is a deep understanding of the concept of scope.
Scope defines the order in which the Python interpreter looks up names (which can refer to anything: variables, functions, or classes) in a program. Python’s scope follows the LEGB rule (Local, Enclosing, Global, and Built-in scope). According to this rule, when you access a name, the interpreter will look for it in the local, enclosing, global, and built-in scopes in that order.
Let’s look at some examples to better understand each level.
Example 1: Local Scope
Here, x is defined locally only within the func function and cannot be accessed from other locations in the script.
Example 2: Closure Scope
The closure scope lies between local and global definitions and arises in nested functions. In the example above, x is defined locally in the outer_func function, but the inner_func function nested within can still access the x variable. However, note that inner_func only has read-only access to x; even if x is reassigned, it only takes effect within inner_func, and the assignment of x in outer_func does not change.(Reference: https://qiwsir.github.io/2021/11/03/python-builtins-first/)
Example 3: Global Scope
Here, both the variable x and the function func are defined globally, and both can be accessed from anywhere in the script. However, to modify a global variable in a smaller scope, you need to use the global keyword, as shown below.
Example 4: Built-in Scope
The built-in scope includes all defined libraries, classes, functions, and variables that do not require explicit import statements. For example, built-in functions in Python: print, len, range, etc.; and built-in variables: str, int, float, etc.
2. Function Closure
The definition of scope determines the closure property of functions. By default, when a function completes its execution, it does not return a value, meaning that all memory used by the function will be wiped out.
Above, we assigned the value 3 to x, but the function forgets it after execution. What if we don’t want it to forget the value of x?
This is where function closure comes into play. By defining a variable within the closed scope of an inner function, it can be stored in the memory of the inner function even after the function returns.
Below is a simple example function used to count how many times it has been executed.
According to Python rules, we should lose the count variable after the first execution. However, since it is defined in the closure of the inner function, it will remain there until the session closes.
3. Decorators
Besides the count variable, function closures have other important roles, one of which is creating decorators. A decorator is a nested function that can be added to other functions to enhance or even modify their behavior.
As shown below, we created a caching decorator that remembers the state of each positional and keyword argument of the function.
The stateful_function decorator can be added to computationally intensive functions that need to be reused with the same parameters. For example, the Fibonacci recursive function below will return the nth number in the sequence, and if we call the decorator we just mentioned, the code and result are as follows:
The 1000th number takes less than 2 seconds!
What if we don’t use a decorator? Let’s try calculating the 40th number.
Calculating the 40th number took 21 seconds, and without caching, calculating the 1000th number would take days.
4. Generators
Generators are powerful constructs in Python that can handle large amounts of data efficiently. Suppose you have a 10GB log file that records the situation when a certain software crashed. To find out what went wrong, you need to filter it efficiently in Python.
The worst method would be to read the entire file, but since you are reviewing the logs line by line, you don’t need to read all 10GB of data at once; you can read a small portion at a time. This is where you can use generators.
In the example above, we defined a generator that iterates through 1024 lines of the log file at a time, making the final for loop very efficient. In each iteration of the for loop, only 1024 lines of the file are in memory, and previous chunks are discarded once they are used, while the remaining chunks are loaded only when needed.
Another feature of generators is the ability to generate one element at a time using the next function, even outside of the loop. Below, we will define a quick function to generate Fibonacci numbers.
To create a generator, you simply call the function once and call the next function on the generated object.
5. Context Managers
You must have been using context managers for a long time. They allow developers to manage resources such as files, databases, and network connections effectively. They automatically open and close resources, resulting in clear and error-free code.
However, there is a big difference between using context managers and writing your own context managers. If handled properly, they allow you to abstract a lot of boilerplate code on top of the original functionality.
A common example of a custom context manager is a timer, as shown in the code below:
Here, we defined a TimerContextManager class that will serve as a future context manager. Its __enter__ method defines what happens when entering the context using the with keyword. In this example, the __enter__ method starts the timer; in __exit__, we leave the context, stop the timer, and report the elapsed time.
Below is a more complex example that can lock resources, allowing them to be used by only one process at a time.
Original Title:
5 Signs You’ve Become an Advanced Pythonista Without Even Realizing It
Original Link:
https://towardsdatascience.com/5-signs-youve-become-an-advanced-pythonista-without-even-realizing-it-2b1dd7ef57f3
Translator’s Profile
Author’s Profile
Zhao Jiankai, a graduate student in Management Science and Engineering at Zhejiang University, focuses on the application of machine learning in the field of social commerce.
Translation Team Recruitment Information
Job Description: Requires a meticulous heart to translate selected foreign articles into fluent Chinese. If you are an international student in data science/statistics/computer-related fields, or working overseas in related jobs, or confident in your foreign language skills, you are welcome to join the translation team.
What You Can Get: Regular translation training to improve volunteers’ translation skills, enhance awareness of cutting-edge data science, and overseas friends can maintain contact with domestic technical application development. The THU Data Team’s academic and research background provides good development opportunities for volunteers.
Other Benefits: Data scientists from well-known companies and students from prestigious universities such as Peking University and Tsinghua University, as well as overseas students, will become your partners in the translation team.
Click on “Read the Original” at the end to join the Data Team~
Reprint Instructions
If you need to reprint, please indicate the author and source prominently at the beginning (originally from: Data Team ID: DatapiTHU), and place a prominent QR code of the Data Team at the end of the article. For articles with original markings, please send [Article Name – Name and ID of the Authorized Public Account] to the contact email to apply for whitelist authorization and edit according to requirements.
After publishing, please provide feedback on the link to the contact email (see below). Unauthorized reprints and adaptations will be legally pursued.
Click “Read the Original” to embrace the organization