1. Python Basics
- List: A built-in data type in Python is the list: list. A list is an ordered collection that allows adding and removing elements at any time.
>>> classmates = ['Michael', 'Bob', 'Tracy']
- tuple: Another type of ordered list is called a tuple: tuple. A tuple is very similar to a list, but once initialized, a tuple cannot be modified. For example, listing the names of classmates:
>>> classmates = ('Michael', 'Bob', 'Tracy')
- dict: Python has built-in support for dictionaries: dict, which is short for dictionary. In other languages, it is also known as a map, using key-value storage, and has extremely fast lookup speeds.
>>> d = {'Michael': 95, 'Bob': 75, 'Tracy': 85}
Compared to lists, dicts have the following characteristics: lookup and insertion speeds are extremely fast and do not slow down as keys increase; they require a large amount of memory, leading to more memory waste. In contrast, lists: lookup and insertion times increase with the number of elements; they occupy less space and waste little memory. Therefore, dicts are a method of trading space for time. 4. set: A set is similar to a dict, also a collection of keys, but does not store values. Since keys cannot be duplicated, there are no duplicate keys in a set.
>>> s = set([1, 2, 3])
The only difference between a set and a dict is that it does not store corresponding values. However, the principle of a set is the same as that of a dict, so mutable objects cannot be placed in a set because it is impossible to determine if two mutable objects are equal, thus ensuring that there are “no duplicate elements” within the set. Try putting a list into a set and see if it raises an error.
| Name | Mutability | Existence Form | Repetitiveness | Orderliness | Other Characteristics |
|---|---|---|---|---|---|
| List | Elements can be modified | Value form [1, 2] | Values can be repeated | Ordered | Compared to dict, it has the characteristic of occupying less memory, commonly used for stack processing, etc. |
| Tuple | Elements cannot be modified, but the elements themselves can change, for example, when the element is a List | Value form (1, 2) | Values can be repeated | Ordered | Itself immutable, relatively stable |
| Dict | Keys are immutable, values can change | Key-value pair form {1:1,2:2} | Keys cannot be repeated, values can be repeated | Unordered | Follows the idea of trading memory for speed, commonly used for lookups |
| Set | Elements can be modified | Value form set([1,2]) | Values cannot be repeated | Unordered | Commonly used to check if a value exists |
2. Advanced Features
- Slicing: For operations that frequently take a specified index range, using loops is cumbersome. Therefore, Python provides a slicing operator that greatly simplifies this operation. Typically, a slicing operation requires three parameters [start_index: stop_index: step]. start_index is the starting position of the slice; stop_index is the ending position of the slice (not included); step can be omitted, with a default value of 1, and the step value cannot be 0, otherwise a ValueError will be raised.
>>> m = list(range(100))# Create a list from a range function from 0-99 and assign it to m
>>> m
[0, 1, 2, 3, 4, 5, 6, ..., 99]
>>> m[:10]# Take the first ten numbers
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> m[-10:]# Take the last ten numbers
[90, 91, 92, 93, 94, 95, 96, 97, 98, 99]
>>> m[10:20]# Take the numbers from 11-20
[10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
>>> m[:10:2]# From the first ten numbers, take one every 2 numbers
[0, 2, 4, 6, 8]
>>> m[5:15:3]# From the 6th to the 15th numbers, take one every 3 numbers
[5, 8, 11, 14]
>>> m[::10]# From all numbers, take one every 10 numbers
[0, 10, 20, 30, 40, 50, 60, 70, 80, 90]
>>> m[:]# If nothing is written, a list is copied as is
[0, 1, 2, 3, 4, 5, 6, 7, ..., 99]
- Iteration: If a list or tuple is given, it can be looped through using for … in, which we call iteration. In Python, any iterable object can be iterated, regardless of whether it has an index.
>>> d = {'a': 1, 'b': 2, 'c': 3}
>>> for key in d:
print(key)
- List Comprehensions: List comprehensions are a concise way to create lists.
- Basic Syntax: [exp iter_var in iterable]
- Working Process: Iterate through each element in the iterable; each iteration assigns the result to iter_var, then computes a new value through exp; finally, all computed values are returned as a new list.
>>> [x * x for x in range(1, 11)]
[1, 4, 9, 16, 25, 36, 49, 64, 81, 100]
- **With filtering syntax**: [exp iter_var in iterable if_exp]
>>> [x * x for x in range(1, 11) if x % 2 == 0]
[4, 16, 36, 64, 100]
- **Can also use two nested loops to generate permutations**:
>>> [m + n for m in 'ABC' for n in 'XYZ']
['Ax', 'AY', 'AZ', 'BX', 'BY', 'BZ', 'Cx', 'CY', 'CZ']
- Generators: If the elements of a list can be calculated by some algorithm, can we continuously calculate subsequent elements during iteration? This way, we do not need to create a complete list, thus saving a lot of space. In Python, this mechanism of calculating while iterating is called a generator: generator, which saves the algorithm. There are many ways to create a generator.
- First Method: Just change the [] of a list comprehension to (), and a generator is created.
>>> L=[x*x for x in range(10)]
>>> L
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
>>> g=(x*x for x in range(10))
>>> g
<generator object <genexpr> at 0x1022ef630>
The difference between creating L and g is only in the outermost [] and (), where L is a list, and g is a generator. – Another Method: If a function definition contains the yield keyword, then this function is no longer a normal function but a generator.
def fib(max):
n,a,b=0,0,1
while n<max:
yield b
a,b=b,a+b
n=n+1
return 'done'
>>> f=fib(6)
>>> f
<generator object fib at Ox104feaaa0>
The execution flow of a generator function is different from that of a normal function. A function executes sequentially and returns when it encounters a return statement or the last line of the function. In contrast, a generator function executes each time next is called, returning at the yield statement, and continues execution from the last yield statement when called again. Please note the distinction between normal functions and generator functions; normal function calls return results directly.
>>> r=abs(6)
>>> r
6
The “call” of a generator function actually returns a generator object.
>>> g=fib(6)
>>> g
<generator object fib at 0x1022ef948>
- **Difference between list comprehensions and list generators**: List comprehensions compute all results and store them in memory. If the list contains a lot of data, it will occupy too much memory. List generators create a list generator object and do not compute all results at once. If you need to retrieve data by index, you can use the next() function, but be aware that if the next() function cannot retrieve data, it will raise a StopIteration exception. You can use a for loop to iterate over the generator object to retrieve all data as needed. If the data volume is large, it is recommended to use generators; (the elements in the generator are calculated according to a specified algorithm and are generated only when called. This way, there is no need to generate all data at once, thus saving a lot of memory space, making the number of elements generated virtually unlimited, and the return time of operations is also very fast (it is just creating a variable).)
- Iterators
- Use the built-in factory function iter(iterable): it can convert an iterable sequence into an iterator.
- What is an iterator: Iteration, as the name suggests, is to repeat doing something many times (just like what is done in loops). An iterator is an object that implements the __next__() method (this method does not require any parameters when called), and it is a way to access an iterable sequence, usually starting from the first element of the sequence and ending when all elements have been accessed. [Note]: An iterator can only move forward, not backward.
- Advantages of Iterators: Using an iterator does not require preparing all elements in the entire iteration process in advance. An iterator only calculates an element when it iterates to that element, and elements before or after can be nonexistent or destroyed. Therefore, iterators are suitable for traversing huge or even infinite sequences.
- Creating Iterators:
a=[1,2,3,4]
b=(1,2,3)
str='Tomwenxing'
iter(a)
iter(b)
iter(str)
<list_iterator object at 0x00000IB781A898>
<tuple_iterator object at 0x0000017B781A898)
<str_iterator object at 0x000001EB781A898>
-
Custom Iterators:
In Python, an iterator essentially returns the next element or raises StopIteration when the __next__() method is called. Since there is no “iterator” class in Python, any class with the following two characteristics can be called an “iterator” class: has a __next__() method that returns the next element of the container or raises StopIteration; has an __iter__() method that returns the iterator itself.
# Fibonacci sequence
class Fabs():
def __init__(self,max):
self.max=max
self.n,self.a,self.b=0,0,1
def __iter__(self):# Define __iter__ method
return self
def __next__(self):# Define __next__ method
if self.n<self.max:
tmp=self.b
self.a,self.b=self.b,self.a+self.b
# Equivalent to:
# t=(self.a,self.a+self.b)
# self.a=t[0]
# self.b=t[1]
self.n+=1
return tmp
raise StopIteration
print(Fabs(5))
for item in Fabs(10):
print(item,end=' ')
11235813213455 <Fabs object at 0x000001984FF4A748
-
Methods of Iterators:
- iter.next(): Returns the next element of the iterator, but raises StopIteration when there are no more elements.
list=[1,2,3,4]
list=iter(list)
list.__next__()
list.__next__()
list.__next__()
list.__next__()
list.__next__()
1 2 3
Traceback (most recent call last): File "D:/pycharm workspace/duenxing/test.pr', line 7, in (module) print(list.next()) StopIteration
- iter.iter(): Returns the iterator object itself.
list=[1,2,3,4]
list=iter(list)
list.__iter__()
<list_iterator object at 0x0000020BAF1EB1D0)
-
Characteristics of Iterators:
- The user does not need to care about the internal structure of the iterator, only needs to access the next content through the next() method.
- Cannot randomly access a specific value in the collection, can only access sequentially from start to end.
- Cannot go back when halfway through access.
- Facilitates looping through large data sets, saving memory.
- Cannot copy an iterator. If you want to iterate the same object again (or simultaneously), you can only create another iterator object.
In fact, the Iterator object in Python represents a data stream. The Iterator can be called by the next() function to continuously return the next data until there is no data to return, at which point it raises a StopIteration exception. This data stream can be seen as an ordered sequence, but we cannot know the length of this sequence in advance. At the same time, the calculation of the Iterator is lazy; it only calculates and returns the next data when the next() function is called.
-
Relationship between Iterable, Iterator, and generator:
- Generator objects are both iterable and iterators: we already know that generators can be used in for loops and can also be continuously called by the next() function to return the next value until the StopIteration error is raised, indicating that no further values can be returned. In other words, generators satisfy the definitions of both iterable objects and iterators.
- Iterator objects are definitely iterable, but not vice versa: for example, list, dict, str, and other collection data types are iterable but not iterators. However, they can generate an iterator object through the iter() function.
- Iterators, generators, and iterable objects can all be iterated using for loops, and generators and iterators can also be called by the next() function to return the next value.
- Objects that can be directly used in for loops are collectively called iterable objects: Iterable. You can use isinstance() to check if an object is an Iterable object, including: one category is collection data types such as list, tuple, dict, set, str, etc.; another category is generators, including generator functions and those with yield.
- Generators are all Iterator objects, but list, dict, str are Iterable but not Iterator. However, they can obtain an Iterator object through the iter() function.