Follow + Star, learn new Python skills every day
Source: Internet
Lists are the most commonly used data structure in Python, but most programmers only utilize 30% of their functionality. This article will teach you 10 high-frequency, efficient list operation techniques that big tech engineers use daily when handling millions of data points. Mastering them can boost your data processing efficiency by 5-10 times.
1. Understanding the Essence of Lists
Why Learn These 10 Tricks?
Lists in Python are dynamically sized arrays, and each operation has its performance characteristics:
- Access — O(1), very fast
- Append — O(1) (average), fast
- Insert — O(n), slow (requires moving elements)
- Delete — O(n), slow
- Search — O(n), linear scan
The secret of big tech engineers: Choose the right operations and appropriate data structures to avoid high-cost operations.
2. List Creation and Initialization (Tricks 1-2)
Trick 1: List Comprehensions vs Loops — 10 Times Performance Difference
Scenario: Generate a list of squares from 0 to 99
# ❌ Inefficient method (low readability)
squares = []
for i in range(100):
squares.append(i ** 2)
# ✅ Big tech method 1: List comprehension (3 times faster, more Pythonic)
squares = [i ** 2 for i in range(100)]
# ✅ Big tech method 2: map function (functional programming style, for complex operations)
squares = list(map(lambda x: x ** 2, range(100)))
# Performance comparison
import time
def test_performance():
iterations = 10000
# Method 1: Loop
start = time.time()
for _ in range(iterations):
result = []
for i in range(100):
result.append(i ** 2)
time1 = time.time() - start
# Method 2: List comprehension
start = time.time()
for _ in range(iterations):
result = [i ** 2 for i in range(100)]
time2 = time.time() - start
# Method 3: map
start = time.time()
for _ in range(iterations):
result = list(map(lambda x: x ** 2, range(100)))
time3 = time.time() - start
print(f"Loop: {time1:.4f}s")
print(f"List comprehension: {time2:.4f}s (faster by {time1/time2:.1f} times)")
print(f"map: {time3:.4f}s")
test_performance()
# Output:
# Loop: 0.8234s
# List comprehension: 0.2156s (faster by 3.8 times)
# map: 0.3421s
Advanced Usage: Nested comprehensions and conditional filtering
# Create a 2D matrix (3x3)
matrix = [[i*3 + j for j in range(3)] for i in range(3)]
print(matrix)
# Output: [[0, 1, 2], [3, 4, 5], [6, 7, 8]]
# Conditional comprehension: keep only even numbers
numbers = list(range(20))
evens = [n for n in numbers if n % 2 == 0]
print(evens)
# Output: [0, 2, 4, 6, 8, 10, 12, 14, 16, 18]
# Complex conditions: multiple nested
matrix = [
[i*3 + j for j in range(3)]
for i in range(3)
if i % 2 == 0
]
print(matrix)
# Output: [[0, 1, 2], [6, 7, 8]]
# Practical application: Extract data from nested lists
data = [
{"name": "Zhang San", "scores": [85, 90, 92]},
{"name": "Li Si", "scores": [88, 95, 90]},
{"name": "Wang Wu", "scores": [92, 88, 95]},
]
# Extract all scores
all_scores = [score for student in data for score in student["scores"]]
print(all_scores)
# Output: [85, 90, 92, 88, 95, 90, 92, 88, 95]
# Calculate each student's average score
averages = [sum(s["scores"]) / len(s["scores"]) for s in data]
print(averages)
# Output: [89.0, 91.0, 91.66...]
Key Traps: Do not include overly complex logic in comprehensions
# ❌ Bad practice (hard to maintain)
result = [
complex_function(x)
for x in data
if complex_check(x) and another_check(x)
]
# ✅ Good practice (clear and maintainable)
def process_and_filter(x):
if not complex_check(x) or not another_check(x):
return None
return complex_function(x)
result = [process_and_filter(x) for x in data]
result = [r for r in result if r is not None]
Trick 2: *args Unpacking and Initialization — The Art of Laziness
# Quickly create a list of the same elements
# ❌ Bad practice
zeros = []
for i in range(1000):
zeros.append(0)
# ✅ Good practice 1: Multiplication operator
zeros = [0] * 1000
ones = [1] * 1000
# ✅ Good practice 2: More elegant
default_values = [None] * 10
# ⚠️ Trap: Repeated references to mutable objects
# ❌ Wrong practice (all nested lists point to the same object)
matrix = [[0] * 3] * 3
matrix[0][0] = 1
print(matrix)
# Output: [[1, 0, 0], [1, 0, 0], [1, 0, 0]] (all changed!)
# ✅ Correct practice (create independent objects using comprehension)
matrix = [[0] * 3 for _ in range(3)]
matrix[0][0] = 1
print(matrix)
# Output: [[1, 0, 0], [0, 0, 0], [0, 0, 0]] (only the first changed)
# Practical application: Initialize a list of dictionaries
users = [{"name": "", "age": 0, "score": 0} for _ in range(100)]
# Practical application: Create a 2D array (common in matrix operations)
def create_matrix(rows, cols, initial_value=0):
"""Create a rows x cols matrix"""
return [[initial_value for _ in range(cols)] for _ in range(rows)]
matrix = create_matrix(3, 4)
print(matrix)
# Output: [[0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]]
3. List Searching and Indexing (Tricks 3-4)
Trick 3: enumerate() — Get Index and Value
Scenario: When processing data, both value and position are needed
# ❌ Bad practice (inefficient and error-prone)
items = ["apple", "banana", "cherry"]
for i in range(len(items)):
print(f"Index {i}: {items[i]}")
# ✅ Good practice: enumerate (fast and elegant)
for i, item in enumerate(items):
print(f"Index {i}: {item}")
# Specify starting index
for i, item in enumerate(items, start=1):
print(f"Item {i}: {item}")
# Output: Item 1: apple, Item 2: banana, ...
# Practical application 1: Add line numbers to data
def add_line_numbers(lines):
"""Add line numbers to each line"""
return [f"{i+1}: {line}" for i, line in enumerate(lines)]
lines = ["import os", "import sys", "print('hello')"]
numbered_lines = add_line_numbers(lines)
print(numbered_lines)
# Output: ['1: import os', '2: import sys', '3: print('hello')']
# Practical application 2: Find all matching indices
def find_all_indices(lst, target):
"""Find all indices of elements equal to target in the list"""
return [i for i, x in enumerate(lst) if x == target]
numbers = [1, 2, 3, 2, 4, 2, 5]
print(find_all_indices(numbers, 2))
# Output: [1, 3, 5]
# Practical application 3: Process file lines, retaining line number information
def parse_csv_with_line_numbers(csv_text):
"""Parse CSV, return (line_number, data_dict)"""
lines = csv_text.strip().split("\n")
headers = lines[0].split(",")
result = []
for line_num, line in enumerate(lines[1:], start=2):
data = dict(zip(headers, line.split(",")))
result.append((line_num, data))
return result
csv_data = """name,age,city
Zhang San,25,Beijing
Li Si,30,Shanghai"""
parsed = parse_csv_with_line_numbers(csv_data)
for line_num, data in parsed:
print(f"Line {line_num}: {data}")
Trick 4: index() vs find Logic — Choose the Right Search Method
# Find elements in the list
items = ["apple", "banana", "cherry", "banana"]
# Get the first occurrence position
pos = items.index("banana")
print(pos) # Output: 1
# ⚠️ Trap 1: Will raise an error if not found
try:
pos = items.index("grape")
except ValueError as e:
print(f"Error: {e}") # Output: Error: 'grape' is not in list
# ✅ Safe practice 1: Use exception handling
def find_safe(lst, target, default=-1):
try:
return lst.index(target)
except ValueError:
return default
print(find_safe(items, "grape")) # Output: -1
# ✅ Safe practice 2: Use in operator to check first
if "grape" in items:
pos = items.index("grape")
else:
print("Not found")
# Get the last occurrence position
last_pos = len(items) - 1 - items[::-1].index("banana")
print(last_pos) # Output: 3
# Performance comparison: find vs count
import time
large_list = list(range(1000000))
# Method 1: Using index (O(n))
start = time.time()
for _ in range(1000):
try:
pos = large_list.index(999999)
except ValueError:
pass
time1 = time.time() - start
# Method 2: Using in check (O(n))
start = time.time()
for _ in range(1000):
if 999999 in large_list:
pass
time2 = time.time() - start
print(f"index method: {time1:.4f}s")
print(f"in method: {time2:.4f}s")
# Practical application: Remove specific element from the list (need to find position)
def remove_first_occurrence(lst, target):
"""Remove the first occurrence of the target element in the list"""
try:
lst.remove(target) # A more concise method
return True
except ValueError:
return False
numbers = [1, 2, 3, 2, 4]
remove_first_occurrence(numbers, 2)
print(numbers) # Output: [1, 3, 2, 4]
# Practical application: Check for duplicate elements in the list
def find_duplicates(lst):
"""Find all duplicate elements in the list"""
seen = set()
duplicates = set()
for item in lst:
if item in seen:
duplicates.add(item)
else:
seen.add(item)
return list(duplicates)
print(find_duplicates([1, 2, 2, 3, 3, 3, 4]))
# Output: [2, 3]
4. List Modification Operations (Tricks 5-6)
Trick 5: append() vs extend() vs insert() — Choosing Wrongly Can Result in 100 Times Performance Difference
Scenario: Adding elements to a list
# Understand the differences between the three methods
items = [1, 2, 3]
# append(): Add a single element (O(1))
items.append(4)
print(items) # Output: [1, 2, 3, 4]
# extend(): Add all elements from an iterable (O(n), n is the number of new elements)
items.extend([5, 6, 7])
print(items) # Output: [1, 2, 3, 4, 5, 6, 7]
# insert(): Insert a single element at a specified position (O(n), requires moving subsequent elements)
items.insert(0, 0) # Insert at the front
print(items) # Output: [0, 1, 2, 3, 4, 5, 6, 7]
# ⚠️ Performance trap: Frequent insertion at the front of the list
# ❌ Bad practice (O(n²) complexity, very slow)
result = []
for i in range(1000):
result.insert(0, i) # Every time needs to move all elements
# ✅ Good practice 1: Append first, then reverse (O(n))
result = []
for i in range(1000):
result.append(i)
result.reverse()
# ✅ Good practice 2: Use deque (O(1))
from collections import deque
result = deque()
for i in range(1000):
result.appendleft(i)
result = list(result)
# ⚠️ Performance trap: Appending a list (not extend)
# ❌ Wrong practice
combined = [1, 2, 3]
combined.append([4, 5, 6])
print(combined)
# Output: [1, 2, 3, [4, 5, 6]] (nested)
# ✅ Correct practice
combined = [1, 2, 3]
combined.extend([4, 5, 6])
print(combined)
# Output: [1, 2, 3, 4, 5, 6]
# Performance comparison: append vs extend vs +=
import time
data = list(range(100))
iterations = 10000
# Method 1: append single element
start = time.time()
for _ in range(iterations):
result = [1, 2, 3]
result.append(4)
time1 = time.time() - start
# Method 2: extend multiple elements
start = time.time()
for _ in range(iterations):
result = [1, 2, 3]
result.extend([4, 5, 6, 7])
time2 = time.time() - start
# Method 3: Using + operator
start = time.time()
for _ in range(iterations):
result = [1, 2, 3]
result = result + [4, 5, 6, 7]
time3 = time.time() - start
print(f"append: {time1:.4f}s")
print(f"extend: {time2:.4f}s")
print(f"+ operator: {time3:.4f}s (slower by {time3/time2:.1f} times)")
# Practical application: Build an event log
class EventLog:
def __init__(self):
self.events = []
def log_event(self, event):
"""Record a single event"""
self.events.append(event)
def log_batch_events(self, events):
"""Batch record events"""
self.events.extend(events)
def get_latest(self, n):
"""Get the last n events"""
return self.events[-n:]
log = EventLog()
log.log_event({"type": "login", "user": "zhangsan"})
log.log_batch_events([
{"type": "click", "button": "submit"},
{"type": "logout", "user": "zhangsan"}
])
print(log.get_latest(2))
Trick 6: del vs remove vs pop — Subtle Differences in Deletion Operations
# Differences between the three deletion methods
numbers = [1, 2, 3, 4, 5]
# del: Delete by index (O(n), requires moving subsequent elements)
# Advantages: Fast, can delete multiple
# Disadvantages: Needs to know the index
del numbers[2]
print(numbers) # Output: [1, 2, 4, 5]
# remove: Delete the first matching item by value (O(n), requires search)
# Advantages: Delete by value, intuitive
# Disadvantages: Raises an error if the value does not exist
numbers = [1, 2, 3, 4, 5]
numbers.remove(3)
print(numbers) # Output: [1, 2, 4, 5]
# pop: Delete by index and return value (O(n))
# Advantages: Returns the deleted value, can be used for stack operations
# Disadvantages: Default deletes the last one, slow to delete from the front
numbers = [1, 2, 3, 4, 5]
last = numbers.pop()
print(last, numbers) # Output: 5 [1, 2, 3, 4]
first = numbers.pop(0)
print(first, numbers) # Output: 1 [2, 3, 4]
# ⚠️ Performance trap: Deleting elements in a loop
# ❌ Bad practice (will skip elements)
numbers = [1, 2, 3, 4, 5, 2, 2]
for i, num in enumerate(numbers):
if num == 2:
numbers.pop(i) # Wrong! Will skip elements
print(numbers) # Output may not meet expectations
# ✅ Good practice 1: Reverse loop (from back to front)
numbers = [1, 2, 3, 4, 5, 2, 2]
for i in range(len(numbers) - 1, -1, -1):
if numbers[i] == 2:
numbers.pop(i)
print(numbers) # Output: [1, 3, 4, 5]
# ✅ Good practice 2: List comprehension (create a new list)
numbers = [1, 2, 3, 4, 5, 2, 2]
numbers = [x for x in numbers if x != 2]
print(numbers) # Output: [1, 3, 4, 5]
# ✅ Good practice 3: Use filter
numbers = [1, 2, 3, 4, 5, 2, 2]
numbers = list(filter(lambda x: x != 2, numbers))
print(numbers) # Output: [1, 3, 4, 5]
# Performance comparison: Deletion operations
import time
large_list = list(range(10000))
iterations = 1000
# Method 1: pop from the end (O(1))
start = time.time()
for _ in range(iterations):
lst = large_list.copy()
lst.pop()
time1 = time.time() - start
# Method 2: pop from the front (O(n))
start = time.time()
for _ in range(iterations):
lst = large_list.copy()
lst.pop(0)
time2 = time.time() - start
# Method 3: del specified index
start = time.time()
for _ in range(iterations):
lst = large_list.copy()
del lst[5000]
time3 = time.time() - start
print(f"pop(): {time1:.4f}s")
print(f"pop(0): {time2:.4f}s (slower by {time2/time1:.1f} times)")
print(f"del: {time3:.4f}s")
# Practical application: Clean invalid data
def clean_data(records, invalid_marker=None):
"""Clean records containing invalid markers"""
# Method 1: Not recommended (deleting in a loop)
# for i in range(len(records) - 1, -1, -1):
# if records[i] == invalid_marker:
# records.pop(i)
# Method 2: Recommended (create a new list)
return [r for r in records if r != invalid_marker]
records = [1, None, 3, None, 5, 7, None]
clean_records = clean_data(records, None)
print(clean_records) # Output: [1, 3, 5, 7]
5. List Sorting and Rearranging (Tricks 7-8)
Trick 7: sort() vs sorted() — Performance and Usage Scenarios
# sort() in-place sorting (modifies the list itself) vs sorted() returns a new list
numbers = [3, 1, 4, 1, 5, 9, 2, 6]
# Method 1: sort() (O(n log n), in-place sorting)
numbers.sort()
print(numbers) # Output: [1, 1, 2, 3, 4, 5, 6, 9]
# Method 2: sorted() (O(n log n), returns a new list)
numbers = [3, 1, 4, 1, 5, 9, 2, 6]
sorted_numbers = sorted(numbers)
print(sorted_numbers) # Output: [1, 1, 2, 3, 4, 5, 6, 9]
print(numbers) # Original list unchanged: [3, 1, 4, 1, 5, 9, 2, 6]
# Descending order
numbers = [3, 1, 4, 1, 5, 9]
numbers.sort(reverse=True)
print(numbers) # Output: [9, 5, 4, 3, 1, 1]
# ✨ Advanced usage 1: Sort by custom key
students = [
{"name": "Zhang San", "score": 85},
{"name": "Li Si", "score": 92},
{"name": "Wang Wu", "score": 78},
]
# Sort by score
students.sort(key=lambda x: x["score"])
print(students)
# Sort by score in descending order
students.sort(key=lambda x: x["score"], reverse=True)
print(students)
# Sort by name
students.sort(key=lambda x: x["name"])
print(students)
# ✨ Advanced usage 2: Multi-level sorting
students = [
{"name": "Zhang San", "score": 85, "age": 20},
{"name": "Li Si", "score": 85, "age": 22},
{"name": "Wang Wu", "score": 90, "age": 19},
]
# Sort first by score in descending order, then by age in ascending order
from operator import itemgetter
students.sort(key=itemgetter("score", "age"), reverse=True) # ❌ Will reverse all
# ✅ Correct practice: Sort step by step (starting from the secondary condition)
students.sort(key=itemgetter("age")) # Sort by age in ascending order
students.sort(key=itemgetter("score"), reverse=True) # Then sort by score in descending order
# Output will be sorted by score in descending order, same score sorted by age in ascending order
# ✨ Advanced usage 3: Case-insensitive sorting
words = ["Apple", "banana", "Cherry"]
words.sort()
print(words) # Output: ['Apple', 'Cherry', 'banana'] (uppercase first)
words.sort(key=str.lower)
print(words) # Output: ['Apple', 'banana', 'Cherry'] (case-insensitive)
# ✨ Advanced usage 4: Use Pandas for complex sorting
import pandas as pd
# Create a DataFrame
df = pd.DataFrame({
"name": ["Zhang San", "Li Si", "Wang Wu"],
"score": [85, 92, 78],
"age": [20, 22, 19]
})
# Sort by multiple columns
sorted_df = df.sort_values(by=["score", "age"], ascending=[False, True])
print(sorted_df)
# Performance comparison: sort vs sorted
import time
data = list(range(100000, 0, -1))
iterations = 100
# Method 1: sort() (modifies the original list)
start = time.time()
for _ in range(iterations):
lst = data.copy()
lst.sort()
time1 = time.time() - start
# Method 2: sorted() (creates a new list)
start = time.time()
for _ in range(iterations):
result = sorted(data)
time2 = time.time() - start
print(f"sort(): {time1:.4f}s")
print(f"sorted(): {time2:.4f}s")
# Usually the difference is not significant, but sort() saves more memory
# Practical application: Leaderboard system
class Leaderboard:
def __init__(self):
self.players = []
def add_player(self, name, score):
self.players.append({"name": name, "score": score})
def get_top_10(self):
"""Get the top 10 players"""
sorted_players = sorted(self.players,
key=lambda x: x["score"],
reverse=True)
return sorted_players[:10]
def get_rank(self, player_name):
"""Get the player's rank"""
sorted_players = sorted(self.players,
key=lambda x: x["score"],
reverse=True)
for rank, player in enumerate(sorted_players, 1):
if player["name"] == player_name:
return rank
return None
leaderboard = Leaderboard()
leaderboard.add_player("Player A", 1000)
leaderboard.add_player("Player B", 1500)
leaderboard.add_player("Player C", 1200)
print(leaderboard.get_top_10())
print(f"Player B's rank: {leaderboard.get_rank('Player B')}")
Trick 8: reverse() and Slicing Reversal — Which is Faster?
# Two reversal methods
numbers = [1, 2, 3, 4, 5]
# Method 1: reverse() (O(n), in-place reversal)
numbers.reverse()
print(numbers) # Output: [5, 4, 3, 2, 1]
# Method 2: Slicing reversal (O(n), creates a new list)
numbers = [1, 2, 3, 4, 5]
reversed_numbers = numbers[::-1]
print(reversed_numbers) # Output: [5, 4, 3, 2, 1]
# Method 3: reversed() function (returns an iterator)
numbers = [1, 2, 3, 4, 5]
reversed_iter = reversed(numbers)
print(list(reversed_iter)) # Output: [5, 4, 3, 2, 1]
# Performance comparison
import time
large_list = list(range(1000000))
iterations = 1000
# Method 1: reverse()
start = time.time()
for _ in range(iterations):
lst = large_list.copy()
lst.reverse()
time1 = time.time() - start
# Method 2: Slicing reversal
start = time.time()
for _ in range(iterations):
result = large_list[::-1]
time2 = time.time() - start
# Method 3: reversed() function
start = time.time()
for _ in range(iterations):
result = list(reversed(large_list))
time3 = time.time() - start
print(f"reverse(): {time1:.4f}s (fastest, in-place modification)")
print(f"[::-1]: {time2:.4f}s")
print(f"reversed(): {time3:.4f}s")
# ⚠️ Trap: reversed() returns an iterator, not a list
numbers = [1, 2, 3, 4, 5]
for num in reversed(numbers):
print(num) # Can only iterate once
# Practical application 1: Palindrome check
def is_palindrome(lst):
"""Check if the list is a palindrome"""
return lst == lst[::-1]
print(is_palindrome([1, 2, 3, 2, 1])) # True
print(is_palindrome([1, 2, 3, 4, 5])) # False
# Practical application 2: Z-shaped matrix reading
def zigzag_traverse(matrix):
"""Z-shaped traverse of the matrix (alternating row reversal)"""
result = []
for i, row in enumerate(matrix):
if i % 2 == 0:
result.extend(row)
else:
result.extend(row[::-1])
return result
matrix = [
[1, 2, 3],
[4, 5, 6],
[7, 8, 9]
]
print(zigzag_traverse(matrix))
# Output: [1, 2, 3, 6, 5, 4, 7, 8, 9]
6. List Slicing and Copying (Trick 9)
Trick 9: The Art of Slicing — One Line of Code Instead of 10 Lines of Loop
# Slicing basics: list[start:stop:step]
numbers = list(range(10)) # [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
# Get a subset
print(numbers[2:5]) # [2, 3, 4] (includes start, excludes stop)
print(numbers[:5]) # [0, 1, 2, 3, 4] (from start to position 5)
print(numbers[5:]) # [5, 6, 7, 8, 9] (from position 5 to end)
# Step
print(numbers[::2]) # [0, 2, 4, 6, 8] (take one every other)
print(numbers[1::2]) # [1, 3, 5, 7, 9] (start from position 1, take one every other)
# Negative indexing
print(numbers[-3:]) # [7, 8, 9] (last 3)
print(numbers[:-3]) # [0, 1, 2, 3, 4, 5, 6] (all but the last 3)
# Reversal
print(numbers[::-1]) # [9, 8, 7, 6, 5, 4, 3, 2, 1, 0]
# ✨ Advanced trick 1: In-place modification (using slice assignment)
numbers = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
numbers[2:5] = [20, 30, 40] # Replace elements at positions 2-4
print(numbers)
# Output: [0, 1, 20, 30, 40, 5, 6, 7, 8, 9]
# Deleting elements
numbers = list(range(10))
numbers[2:5] = [] # Delete elements at positions 2-4
print(numbers)
# Output: [0, 1, 5, 6, 7, 8, 9]
# Inserting elements
numbers = [1, 2, 5, 6]
numbers[2:2] = [3, 4] # Insert [3, 4] at position 2
print(numbers)
# Output: [1, 2, 3, 4, 5, 6]
# ✨ Advanced trick 2: Rotate array
def rotate(lst, k):
"""Rotate the list to the right by k steps"""
k = k % len(lst)
return lst[-k:] + lst[:-k]
print(rotate([1, 2, 3, 4, 5], 2))
# Output: [4, 5, 1, 2, 3]
# ✨ Advanced trick 3: Take N spaced elements from the list
def sample_every_n(lst, n):
"""Take one every n elements"""
return lst[::n]
data = list(range(20))
print(sample_every_n(data, 5))
# Output: [0, 5, 10, 15]
# ✨ Advanced trick 4: Split the list into N equal parts
def chunk_list(lst, n):
"""Split the list into n equal parts"""
return [lst[i::n] for i in range(n)]
data = list(range(10))
chunks = chunk_list(data, 3)
print(chunks)
# Output: [[0, 3, 6, 9], [1, 4, 7], [2, 5, 8]]
# ✨ Advanced trick 5: Sliding window
def sliding_window(lst, window_size):
"""Return all windows of size window_size"""
return [lst[i:i+window_size] for i in range(len(lst) - window_size + 1)]
print(sliding_window([1, 2, 3, 4, 5], 3))
# Output: [[1, 2, 3], [2, 3, 4], [3, 4, 5]]
# Performance comparison: slicing vs loop
import time
large_list = list(range(1000000))
iterations = 1000
# Method 1: Slicing
start = time.time()
for _ in range(iterations):
result = large_list[10000:20000]
time1 = time.time() - start
# Method 2: Loop
start = time.time()
for _ in range(iterations):
result = []
for i in range(10000, 20000):
result.append(large_list[i])
time2 = time.time() - start
print(f"Slicing: {time1:.4f}s")
print(f"Loop: {time2:.4f}s (slower by {time2/time1:.1f} times)")
# Practical application 1: Batch process data
def batch_process(data, batch_size, process_func):
"""Process data in batches"""
results = []
for i in range(0, len(data), batch_size):
batch = data[i:i+batch_size]
results.append(process_func(batch))
return results
# Example usage
data = list(range(100))
def sum_batch(batch):
return sum(batch)
results = batch_process(data, 10, sum_batch)
print(results[:3]) # [45, 145, 245, ...]
# Practical application 2: Get the first N and last N elements
def get_head_tail(lst, n):
"""Get the first n and last n elements"""
return lst[:n], lst[-n:]
data = list(range(20))
head, tail = get_head_tail(data, 3)
print(f"First 3: {head}, Last 3: {tail}")
# Output: First 3: [0, 1, 2], Last 3: [17, 18, 19]
7. The Traps of List Copying (Trick 10)
Trick 10: Shallow Copy vs Deep Copy — The Birth of a Bug
# Problem: Mutable objects in the list
original = [[1, 2], [3, 4]]
# ❌ Wrong practice 1: Direct assignment (only copies reference)
copy1 = original
copy1[0][0] = 999
print(original) # Output: [[999, 2], [3, 4]] (original list also changed!)
# ✅ Correct practice 1: Shallow copy (copy() method)
original = [[1, 2], [3, 4]]
copy2 = original.copy()
copy2[0][0] = 999
print(original) # Output: [[999, 2], [3, 4]] (⚠️ Inner list still affected)
# ✅ Correct practice 2: Shallow copy (slicing)
original = [[1, 2], [3, 4]]
copy3 = original[:]
copy3[0][0] = 999
print(original) # Output: [[999, 2], [3, 4]] (⚠️ Same issue)
# ✅ Correct practice 3: Deep copy (deepcopy)
import copy
original = [[1, 2], [3, 4]]
deep_copy = copy.deepcopy(original)
deep_copy[0][0] = 999
print(original) # Output: [[1, 2], [3, 4]] (✅ Original list unchanged)
# Understanding shallow copy vs deep copy
original = {
"name": "Zhang San",
"scores": [85, 90, 95],
"info": {"age": 25, "city": "Beijing"}
}
# Shallow copy: only copies the outer layer
import copy
shallow = copy.copy(original)
shallow["scores"][0] = 999
print(original["scores"]) # [999, 90, 95] (changed)
shallow["info"]["city"] = "Shanghai"
print(original["info"]["city"]) # Shanghai (changed)
# Deep copy: recursively copies all layers
deep = copy.deepcopy(original)
deep["scores"][0] = 999
deep["info"]["city"] = "Shanghai"
print(original["scores"]) # [85, 90, 95] (unchanged)
print(original["info"]["city"]) # Beijing (unchanged)
# Performance comparison: copy vs deepcopy
import time
data = [[i for i in range(100)] for _ in range(100)]
# Shallow copy
start = time.time()
for _ in range(1000):
shallow = copy.copy(data)
time1 = time.time() - start
# Deep copy
start = time.time()
for _ in range(1000):
deep = copy.deepcopy(data)
time2 = time.time() - start
print(f"Shallow copy: {time1:.4f}s")
print(f"Deep copy: {time2:.4f}s (slower by {time2/time1:.1f} times)")
# ⚠️ Special case: Immutable objects in the list
original = [1, 2, 3, 4, 5]
copy_list = original.copy()
copy_list[0] = 999
print(original) # Output: [1, 2, 3, 4, 5] (✅ Unchanged, because numbers are immutable)
# Practical application 1: Save original state
class DataProcessor:
def __init__(self, data):
self.original_data = copy.deepcopy(data)
self.processed_data = copy.deepcopy(data)
def reset(self):
"""Restore to original state"""
self.processed_data = copy.deepcopy(self.original_data)
def modify(self, index, value):
"""Modify data"""
self.processed_data[index] = value
processor = DataProcessor([[1, 2], [3, 4]])
processor.modify(0, [999, 999])
print(processor.processed_data) # [[999, 999], [3, 4]]
processor.reset()
print(processor.processed_data) # [[1, 2], [3, 4]]
# Practical application 2: Maintain history records
class DataHistory:
def __init__(self):
self.history = []
def record_state(self, data):
"""Record current state"""
self.history.append(copy.deepcopy(data))
def get_history(self, index):
"""Get historical state"""
return self.history[index] if 0 <= index < len(self.history) else None
history = DataHistory()
data = [1, 2, 3]
history.record_state(data)
data[0] = 999
history.record_state(data)
print(history.get_history(0)) # [1, 2, 3] (first state unchanged)
print(history.get_history(1)) # [999, 2, 3] (second state)
8. Comprehensive Practice: Complete Data Processing System
Practical Case 1: E-commerce Shopping Cart System
class ShoppingCart:
def __init__(self):
self.items = []
def add_item(self, product_id, name, price, quantity=1):
"""Add product"""
# Check if it already exists
for item in self.items:
if item["product_id"] == product_id:
item["quantity"] += quantity
return
# New product
self.items.append({
"product_id": product_id,
"name": name,
"price": price,
"quantity": quantity
})
def remove_item(self, product_id):
"""Remove product"""
self.items = [item for item in self.items
if item["product_id"] != product_id]
def get_total_price(self):
"""Get total price"""
return sum(item["price"] * item["quantity"] for item in self.items)
def get_items_by_price_range(self, min_price, max_price):
"""Filter by price range"""
return [item for item in self.items
if min_price <= item["price"] <= max_price]
def apply_discount(self, discount_rate):
"""Apply discount"""
for item in self.items:
item["price"] *= (1 - discount_rate)
def get_sorted_items(self, sort_by="name"):
"""Sort by specified field"""
if sort_by == "price":
return sorted(self.items, key=lambda x: x["price"])
elif sort_by == "name":
return sorted(self.items, key=lambda x: x["name"])
elif sort_by == "quantity":
return sorted(self.items, key=lambda x: x["quantity"])
return self.items
def get_top_items(self, n=3):
"""Get the top N most expensive items"""
sorted_items = sorted(self.items,
key=lambda x: x["price"],
reverse=True)
return sorted_items[:n]
# Example usage
cart = ShoppingCart()
cart.add_item(1, "Phone", 3999, 1)
cart.add_item(2, "Headphones", 599, 2)
cart.add_item(3, "Power Bank", 199, 1)
print(f"Total price: {cart.get_total_price():.2f}") # Total price: 5396.00
print(f"Top 2 most expensive items: {cart.get_top_items(2)}")
Practical Case 2: Student Grade Management System
class GradeManager:
def __init__(self):
self.students = []
def add_student(self, name, scores):
"""Add student grades"""
self.students.append({
"name": name,
"scores": scores.copy() # Deep copy to avoid external modification
})
def get_average_score(self, name):
"""Get average score of a student"""
for student in self.students:
if student["name"] == name:
return sum(student["scores"]) / len(student["scores"])
return None
def get_class_average(self):
"""Get class average score"""
all_scores = [score for student in self.students
for score in student["scores"]]
return sum(all_scores) / len(all_scores) if all_scores else 0
def get_students_above_average(self):
"""Get all students above class average"""
class_avg = self.get_class_average()
return [student for student in self.students
if self.get_average_score(student["name"]) > class_avg]
def get_ranked_students(self):
"""Get students sorted by average score"""
students_with_avg = [
(student["name"], self.get_average_score(student["name"]))
for student in self.students
]
return sorted(students_with_avg, key=lambda x: x[1], reverse=True)
def get_pass_rate(self, pass_score=60):
"""Get pass rate"""
total_scores = sum(len(student["scores"]) for student in self.students)
pass_scores = sum(
1 for student in self.students
for score in student["scores"]
if score >= pass_score
)
return pass_scores / total_scores if total_scores > 0 else 0
def remove_outliers(self, name, threshold=2):
"""Remove outlier grades for a student (exceeding threshold times standard deviation)"""
for student in self.students:
if student["name"] == name:
scores = student["scores"]
mean = sum(scores) / len(scores)
std_dev = (sum((x - mean) ** 2 for x in scores) / len(scores)) ** 0.5
# Keep grades within mean ± threshold*std_dev
student["scores"] = [
s for s in scores
if abs(s - mean) <= threshold * std_dev
]
break
# Example usage
manager = GradeManager()
manager.add_student("Zhang San", [85, 90, 92])
manager.add_student("Li Si", [88, 95, 90])
manager.add_student("Wang Wu", [70, 75, 80])
print(f"Class average score: {manager.get_class_average():.2f}")
print(f"Rankings: {manager.get_ranked_students()}")
print(f"Pass rate: {manager.get_pass_rate():.2%}")
9. Quick Reference Table
| Index | Trick | Complexity | Commonality | Key Points |
|---|---|---|---|---|
| 1 | List Comprehensions | O(n) | ⭐⭐⭐⭐⭐ | 3 times faster than loops |
| 2 | Initialization (*args) | O(n) | ⭐⭐⭐⭐ | Avoid repeated mutable object references |
| 3 | enumerate() | O(n) | ⭐⭐⭐⭐⭐ | Get both index and value |
| 4 | index() Search | O(n) | ⭐⭐⭐⭐ | Requires exception handling |
| 5 | append/extend | O(1)/O(n) | ⭐⭐⭐⭐⭐ | append for single, extend for multiple |
| 6 | del/remove/pop | O(n) | ⭐⭐⭐⭐ | Fastest to delete from the end |
| 7 | sort() / sorted() | O(n log n) | ⭐⭐⭐⭐⭐ | Supports custom key sorting |
| 8 | reverse() | O(n) | ⭐⭐⭐⭐ | In-place reversal is the fastest |
| 9 | Slicing | O(k) | ⭐⭐⭐⭐⭐ | Flexible and efficient |
| 10 | deepcopy | O(n) | ⭐⭐⭐ | Handles nested structures |
10. Best Practice Recommendations
✅ Do These Things
- Use list comprehensions — Concise, fast, Pythonic
- Use append() for adding — Never concatenate lists with + in loops
- Delete from back to front — Avoid index confusion
- Pre-allocate space — Use * initialization to reduce append overhead
- Use slicing — Replace loops for getting subsets
- Use sort() for in-place sorting — Preferred when sorting is needed
- Deep copy nested structures — Avoid data contamination
- Use enumerate() to get indices — More elegant than range(len())
- Choose appropriate data structures — Use deque for frequent deletions from the front
- Understand complexity — Avoid common mistakes of O(n²)
❌ Do Not Do These Things
- ❌ Concatenate lists with + in loops (O(n²))
- ❌ Delete elements in loops (easily skipped)
- ❌ Assume shallow copy can fully replicate (inner objects still shared)
- ❌ Sort by multiple conditions with sort() (needs step-by-step sorting)
- ❌ Frequently insert at the front of the list (O(n), use deque instead)
- ❌ Ignore mutable object traps in list initialization
- ❌ Not handle exceptions for index()
- ❌ Over-optimize (unless there are performance issues)
- ❌ Confuse append and extend
- ❌ Sort when not needed
11. Common Interview Questions
Question 1: Find the second largest number in a list
def second_largest(lst):
"""Find the second largest number"""
# Method 1: Sort (O(n log n))
return sorted(set(lst), reverse=True)[1]
# Method 2: Two passes (O(n), more efficient)
max1 = max2 = float('-inf')
for num in lst:
if num > max1:
max2 = max1
max1 = num
elif num > max2:
max2 = num
return max2
print(second_largest([3, 1, 4, 1, 5, 9, 2, 6])) # Output: 6
Question 2: Check for duplicates in a list
def has_duplicate(lst):
"""Check for duplicate elements"""
# Method 1: Use set (O(n))
return len(lst) != len(set(lst))
# Method 2: Use seen set (O(n))
seen = set()
for item in lst:
if item in seen:
return True
seen.add(item)
return False
print(has_duplicate([1, 2, 3, 2])) # True
Question 3: Rotate an array
def rotate_array(lst, k):
"""Rotate the array to the right by k steps"""
k = k % len(lst)
return lst[-k:] + lst[:-k]
print(rotate_array([1, 2, 3, 4, 5], 2)) # [4, 5, 1, 2, 3]
Question 4: Intersection of two arrays
def array_intersection(arr1, arr2):
"""Find the intersection of two arrays"""
# Method 1: Use set
return list(set(arr1) & set(arr2))
# Method 2: Use list comprehension
set2 = set(arr2)
return [x for x in arr1 if x in set2]
print(array_intersection([1, 2, 2, 1], [2, 2])) # [2]
12. Performance Optimization Summary
Scenario 1: Large-scale Concatenation
# ❌ Bad (O(n²))
result = []
for item in items:
result = result + [item]
# ✅ Good (O(n))
result = []
for item in items:
result.append(item)
# ✅ Best (O(n))
result = [item for item in items]
Scenario 2: Deleting Multiple Elements
# ❌ Bad (easily error-prone)
for i in range(len(lst)):
if lst[i] == target:
lst.pop(i)
# ✅ Good (clear and efficient)
lst = [x for x in lst if x != target]
Scenario 3: Copying Large Nested Structures
# ❌ Bad (shallow copy, inner objects shared)
copy = original.copy()
# ✅ Good (deep copy, completely independent)
import copy
deep_copy = copy.deepcopy(original)
13. Summary and Next Steps
These 10 tricks cover 95% of Python list operations. Key points:
- Comprehensions vs Loops — Always choose comprehensions
- append vs extend — Choose as needed
- Sorting and Searching — Understand complexity
- The Power of Slicing — One line of code for complex operations
- Shallow Copy vs Deep Copy — Handle nested structures
Long press or scan the QR code below to get free access to Python public courses and hundreds of GB of learning materials organized by experts, including but not limited to Python e-books, tutorials, project orders, source code, etc.
Recommended Reading
How to Write LaTeX Elegantly with Python?
Python: Scrape Any Website in Seconds with One Line of Code!
These Python Efficiency Tools Are Amazing!
10 Python Libraries for Automated Exploratory Data Analysis!
Click to read the original text to learn more