Learning Python (Part 2) – Basic Syntax

As a data analyst, have you encountered these scenarios:

You receive a messy dataset and want to quickly extract key information but get stuck on string processing; while cleaning data with pandas, you cause feature extraction chaos due to list slicing errors; when writing analysis scripts, you mess up category mapping because you’re not familiar with dictionary operations…

In fact, 80% of data analysis efficiency issues stem from a lack of thorough understanding of Python’s basic syntax. Today, we will deconstruct Python’s basic syntax from the practical perspective of a data analyst – not a tedious list of syntax, but with the thought of “what analysis problems can these skills solve,” helping you truly turn the basics into analytical capabilities.

This article starts from practical data analysis, equipping each knowledge point with concept explanation + runnable code + output results + analyst perspective, helping you truly transform basic syntax into analytical capabilities.

1. The First Python Program: Building Analytical Intuition from Data Reading

Concept Principle: The core of a Python program is the “sequence of instructions”. For data analysts, the first meaningful program should not be print("Hello World"), but should implement the complete process of “data reading – initial observation”, starting the first dialogue with the data. For example, reading a CSV file and printing its basic information:

import pandas as pd  # Import data analysis library

# Read data (replace with your data path in actual work)
# Here we use the example dataset provided by pandas
data = pd.read_csv("https://raw.githubusercontent.com/mwaskom/seaborn-data/master/tips.csv")

# Execute the "three axes of data analysis"
print("1. Dataset shape (rows, columns):", data.shape)
print("\n2. Overview of data types:\n", data.dtypes)
print("\n3. Preview of the first 3 rows:\n", data.head(3))

Output Result:

1. Dataset shape (rows, columns): (244, 7)

2. Overview of data types:
 total_bill    float64
 tip           float64
 sex            object
 smoker         object
 day            object
 time           object
 size            int64
dtype: object

3. Preview of the first 3 rows:
    total_bill   tip     sex smoker  day    time  size
0       16.99  1.01  Female     No  Sun  Dinner     2
1       10.34  1.66    Male     No  Sun  Dinner     3
2       21.01  3.50    Male     No  Sun  Dinner     3

Analyst Perspective: This piece of code seems simple, yet it contains the core awareness of data analysis – first confirming the scale (shape) of the data, checking the field types (dtypes, to avoid numeric types being misclassified as strings), and previewing the content (head). If total_bill is recognized as a string type, subsequent summation analysis will be erroneous, which illustrates the practical value of basic operations.

2. Python Comments: Making Your Analytical Logic Traceable

Concept Principle: Comments are explanations of code logic, which are particularly important in data analysis – data processing rules (such as outlier judgment standards) are often closely related to business, and comments ensure that you can understand “why you handled it this way” three months later.

Python comments are divided into two types:

• Single-line comments: start with <span>#</span>
• Multi-line comments: wrapped in triple quotes <span>"""</span>

import pandas as pd

data = pd.read_csv("https://raw.githubusercontent.com/mwaskom/seaborn-data/master/tips.csv")

# -------------- Business Rule Explanation --------------
# 1. According to the business department's requirements, the consumption amount (total_bill) should be greater than 0
# 2. The tip (tip) cannot exceed 50% of the consumption amount (to prevent data entry errors)
# 3. The processed data is used to calculate the "tip ratio" indicator

# Clean outliers
# Filter samples that meet the conditions (logical AND operation requires &, and each condition must be in parentheses)
clean_data = data[(data["total_bill"] > 0) & (data["tip"] <= data["total_bill"] * 0.5)]

"""
Validation of cleaning effect:
- Original data volume: 244 rows
- Cleaned data volume: {} rows
- Outlier proportion: {:.2%}
""".format(len(clean_data), 1 - len(clean_data)/len(data))

Output Result:

'Validation of cleaning effect:
- Original data volume: 244 rows
- Cleaned data volume: 243 rows
- Outlier proportion: 0.41%'

Analyst Perspective: In this piece of code, the comments clearly record “why cleaning is necessary” (business rules) and “how to validate the cleaning effect”. In actual work, when colleagues or leaders question “why this data was deleted”, comprehensive comments provide the best explanation.

3. Python Code Structure: The Logic Chain of Data Analysis in Indentation

Concept Principle: Python uses indentation to distinguish code blocks (usually 4 spaces), which is crucial for data analysis – we often need to use <span>if-else</span> to handle conditional logic and <span>for</span> to loop through data, and indentation errors can directly lead to incorrect analysis results.

import pandas as pd

data = pd.read_csv("https://raw.githubusercontent.com/mwaskom/seaborn-data/master/tips.csv")

# Define the numeric columns to be processed
numeric_columns = ["total_bill", "tip"]

# Batch calculate and add new indicators
for col in numeric_columns:
    # Check if it is a numeric type (to avoid performing mathematical operations on string columns)
    if pd.api.types.is_numeric_dtype(data[col]):
        # Calculate the mean of the column
        col_mean = data[col].mean()
        # Add the "column_name_above_mean" indicator (whether it is above the mean)
        data[f"{col}_above_mean"] = data[col] > col_mean
        print(f"Processed {col}, mean is: {col_mean:.2f}")
    else:
        print(f"{col} is not numeric, skipping processing")

# View the new indicators
data[["total_bill", "total_bill_above_mean", "tip", "tip_above_mean"]].head(3)

Output Result:

Processed total_bill, mean is: 19.79
Processed tip, mean is: 3.29

total_bill	total_bill_above_mean	tip	tip_above_mean
0	16.99	False	1.01	False
1	10.34	False	1.66	False
2	21.01	True	3.5	True

Analyst Perspective: The clear indentation distinguishes the “scope of the loop” and the “execution body of the conditional judgment”. If you forget to indent the line data[f"{col}_above_mean"] = ..., the code will only process the last column, causing the previous columns to fail to process – this illustrates the practical significance of indentation in batch data processing.

4. Operators: The Tools for Filtering and Calculating in Data Analysis

Concept Principle: Operators are the basic tools for processing data, mainly used for two types of operations in data analysis: filtering samples that meet conditions (comparison/logical operators) and generating new indicators (arithmetic operators).

Operator Type	Common Symbols	Data Analysis Scenarios
Arithmetic Operators	+, -, *, /, %	Calculate new indicators (e.g., tip ratio = tip/total_bill)
Comparison Operators	>, <, ==, !=	Single condition filtering (e.g., consumption amount > 30)
Logical Operators	& (and), | (or), ~ (not)	Multi-condition combined filtering
Membership Operators	in, not in	Determine if it belongs to the target group

import pandas as pd

data = pd.read_csv("https://raw.githubusercontent.com/mwaskom/seaborn-data/master/tips.csv")

# 1. Arithmetic Operators: Calculate new indicators
data["tip_rate"] = data["tip"] / data["total_bill"]  # Tip ratio = Tip / Total consumption

# 2. Comparison Operators + Logical Operators: Multi-condition filtering
# Filter high-value customers in the "dinner scenario" and "tip rate > 20%"
high_value = data[(data["time"] == "Dinner") & (data["tip_rate"] > 0.2)]

# 3. Membership Operators: Determine if it belongs to the target group
# Weekends include Saturday and Sunday
weekend_days = ["Sat", "Sun"]
data["is_weekend"] = data["day"].isin(weekend_days)

# View results
print("Number of high-value customers:", len(high_value))
print("\nSample of high-value customers:")
high_value[["total_bill", "tip", "tip_rate", "time", "day"]].head(2)

Output Result:

Number of high-value customers: 29

Sample of high-value customers:
    total_bill   tip  tip_rate    time  day
8         15.04  3.00  0.199468  Dinner  Sun
23        39.42  7.58  0.192288  Dinner  Sat

Analyst Perspective: Operators are at the core of pandas filtering. Note three pitfalls: ① Use <span>&</span> for multiple conditions instead of <span>and</span>; ② Each condition must be wrapped in parentheses; ③ Avoid direct use of <span>==</span> for floating-point comparisons (due to precision issues). For example, to check if the tip ratio equals 20%, use <span>abs(tip_rate - 0.2) < 0.001</span>.

5. String Operations: Cleaning and Extracting Text Data

Concept Principle: 80% of dirty data in data analysis is text (such as messy information entered by users, strings with inconsistent formats). The core of string operations is “standardization” and “information extraction”, converting unstructured text into structured data.

1. String Indexing and Slicing

Strings are ordered sequences of characters, which can be accessed by index (position) for individual characters and sliced to obtain substrings.

# Scenario: Extract company domain and username from user emails
emails = [
    "[email protected]",
    "li.ming_2023@analytics firm.cn",
    "[email protected]"
]

# Batch process emails
for email in emails:
    # Find the position of @
    at_pos = email.index("@")
    # Extract username (the part before @)
    username = email[:at_pos]
    # Extract domain (the part after @, removing possible suffix)
    domain = email[at_pos+1:].split(".")[0]  # Take the first part of the domain
    print(f"Email: {email} → Username: {username}, Company: {domain}")

Output Result:

Email: [email protected] → Username: zhang.san, Company: data-analysis
Email: li.ming_2023@analytics firm.cn → Username: li.ming_2023, Company: analytics firm
Email: [email protected] → Username: wang.hong+report, Company: data

2. Formatting Strings

Used to generate structured text (such as analysis reports, file names), commonly using f-string formatting (Python 3.6+).

# Scenario: Generate a standardized analysis report title
analysis_date = "2023-10-01"
total_users = 15680
conversion_rate = 0.087  # Conversion rate

# Use f-string formatting (can directly embed variables and expressions)
report_title = f"[{analysis_date}] User Conversion Analysis Report"
report_summary = f"""
Core Metrics:
- Total Users: {total_users:,} people  # Thousands separator
- Conversion Rate: {conversion_rate:.1%}  # Percentage format
- Converted Users: {int(total_users * conversion_rate):,} people
"""

print(report_title)
print(report_summary)

Output Result:

[2023-10-01] User Conversion Analysis Report

Core Metrics:
- Total Users: 15,680 people
- Conversion Rate: 8.7%
- Converted Users: 1,364 people

3. Common String Functions

Function	Functionality	Data Analysis Scenarios
strip()	Remove leading and trailing spaces	Clean user-entered names, addresses
split()	Split by delimiter	Split dates (e.g., “2023-10-01” into year/month/day)
replace()	Replace characters	Standardize formats (e.g., “Male→1”, “Female→0”)
lower()/upper()	Convert case	Standardize label formats (e.g., “VIP” and “vip” are considered the same category)
contains()	Check if substring is included	Filter comments containing specific keywords

import pandas as pd

# Scenario: Clean user comment data
comments = pd.DataFrame({
    "comment": [
        "  The service is great, I will come again!   ",
        "The dishes are too salty... average taste",
        "The environment is good, but the price is high",
        "Recommended! Recommended! Strongly recommended",
        "Very poor experience, will not come again"
    ]
})

# 1. Remove leading and trailing spaces
comments["clean_comment"] = comments["comment"].str.strip()

# 2. Standardize to lowercase
comments["clean_comment"] = comments["clean_comment"].str.lower()

# 3. Replace special symbols
comments["clean_comment"] = comments["clean_comment"].str.replace("...", ".", regex=False)

# 4. Check if it contains positive keywords
comments["is_positive"] = comments["clean_comment"].str.contains("good|recommended")

comments

Output Result:

comment	clean_comment	is_positive
0	The service is great, I will come again!	The service is great, I will come again!	True
1	The dishes are too salty… average taste	The dishes are too salty. average taste	False
2	The environment is good, but the price is high	The environment is good, but the price is high	True
3	Recommended! Recommended! Strongly recommended	Recommended! Recommended! Strongly recommended	True
4	Very poor experience, will not come again	Very poor experience, will not come again	False

Analyst Perspective: The core of text cleaning is “making it understandable for computers” – through standardization, originally messy comments become analyzable data, allowing for subsequent statistics on positive comment ratios, keyword extraction, etc. The pandas <span>str</span> accessor makes batch processing of text columns exceptionally simple.

6. Data Types: The “Basic Building Blocks” of Data Analysis

Concept Principle: Data types determine how data is stored and what operations can be performed. Choosing the wrong type (e.g., storing dates as strings) can lead to analysis difficulties, while effectively utilizing the features of each type can significantly enhance efficiency.

1. Basic Data Types

Type	Characteristics	Data Analysis Scenarios
int (integer)	No decimal part	Count data (number of users, order volume)
float (floating point)	With decimals	Measurement data (amount, ratio, score)
str (string)	Text sequence	Categorical data (name, address, label)
bool (boolean)	True/False	Logical judgment results (whether it meets standards, whether it is active)

# Scenario: Data type conversion and validation
# Original data (often from Excel or CSV, may have type errors)
user_data = {
    "user_id": "1001",  # Should actually be an integer
    "age": "28",        # Should actually be an integer
    "total_spend": "356.8",  # Should actually be a float
    "is_vip": "True"    # Should actually be a boolean
}

# Type conversion
cleaned_data = {
    "user_id": int(user_data["user_id"]),
    "age": int(user_data["age"]),
    "total_spend": float(user_data["total_spend"]),
    "is_vip": user_data["is_vip"].lower() == "true"  # Safe conversion to boolean
}

# Validate conversion results
for key, value in cleaned_data.items():
    print(f"{key}: {value} (type: {type(value).__name__})")

Output Result:

user_id: 1001 (type: int)
age: 28 (type: int)
total_spend: 356.8 (type: float)
is_vip: True (type: bool)

2. Lists (list): Ordered and Mutable “Data Containers”

Lists are the most commonly used sequence type in data analysis, used to store ordered data (such as feature column names, parameters for batch processing).

Core Operations:

• Slicing: <span>list[start:end:step]</span> (to get a sublist)
• Common Functions: <span>append()</span> (add element), <span>extend()</span> (merge lists), <span>sort()</span> (sort), <span>len()</span> (length)

# Scenario: Batch process feature columns
import pandas as pd

data = pd.read_csv("https://raw.githubusercontent.com/mwaskom/seaborn-data/master/tips.csv")

# Define the numeric columns to standardize and the categorical columns to encode
numeric_features = ["total_bill", "tip"]
categorical_features = ["sex", "smoker", "day"]

# Merge feature lists (for subsequent loop processing)
all_features = numeric_features.copy()
all_features.extend(categorical_features)  # Merge two lists
print("All features:", all_features)

# Standardize numeric columns ((x-mean)/std)
for col in numeric_features:
    mean = data[col].mean()
    std = data[col].std()
    data[f"{col}_norm"] = (data[col] - mean) / std

# Display standardization results (first 2 rows)
data[numeric_features + [f"{col}_norm" for col in numeric_features]].head(2)

Output Result:

All features: ['total_bill', 'tip', 'sex', 'smoker', 'day']

total_bill	tip	total_bill_norm	tip_norm
0	16.99	1.01	-0.314711	-1.43997
1	10.34	1.66	-1.06323	-0.969209

3. Tuples (tuple): Ordered and Immutable “Configuration Items”

Tuples are similar to lists, but their elements cannot be modified after creation, making them suitable for storing fixed configurations (such as analysis parameters, constants).

# Scenario: Define analysis constants (to avoid accidental modification)
# Use tuples to store immutable business parameters
ANALYSIS_CONFIG = (
    ("min_age", 18),         # Minimum analysis age
    ("max_age", 65),         # Maximum analysis age
    ("target_metrics", ["total_bill", "tip_rate"]),  # Core metrics
    ("is_include_weekend", True)  # Whether to include weekend data
)

# Convert to dictionary for easier use (tuples are often used for data transmission, dictionaries are easier to access)
config_dict = dict(ANALYSIS_CONFIG)

print("Minimum age limit:", config_dict["min_age"])
print("Core analysis metrics:", config_dict["target_metrics"])

# Attempting to modify the tuple will raise an error (protect configuration safety)
try:
    ANALYSIS_CONFIG[0] = ("min_age", 16)
except TypeError as e:
    print("\nError modifying tuple:", e)

Output Result:

Minimum age limit: 18
Core analysis metrics: ['total_bill', 'tip_rate']

Error modifying tuple: 'tuple' object does not support item assignment

4. Dictionaries (dict): The “Translator” of Key-Value Mapping

Dictionaries consist of key-value pairs and are the best tool for “one-to-one mapping”, frequently used in data encoding and parameter configuration.

Core Operations:

• Access: <span>dict[key]</span> or <span>dict.get(key, default)</span> (the latter is recommended to avoid errors when the key does not exist)
• Common Functions: <span>keys()</span> (all keys), <span>values()</span> (all values), <span>items()</span> (all key-value pairs)

# Scenario: Data encoding and mapping
import pandas as pd

data = pd.read_csv("https://raw.githubusercontent.com/mwaskom/seaborn-data/master/tips.csv")

# 1. Define gender encoding mapping (string → number)
gender_map = {
    "Male": 0,
    "Female": 1,
    "Unknown": -1  # Missing value handling
}

# 2. Define day mapping (abbreviation → full name)
day_map = {
    "Thur": "Thursday",
    "Fri": "Friday",
    "Sat": "Saturday",
    "Sun": "Sunday"
}

# 3. Apply mapping (using get method to handle unknown values)
data["gender_code"] = data["sex"].apply(lambda x: gender_map.get(x, -1))
data["day_full"] = data["day"].apply(lambda x: day_map.get(x, "Unknown"))

# View results
data[["sex", "gender_code", "day", "day_full"]].head(3)

Output Result:

sex	gender_code	day	day_full
0	Female	1	Sun	Sunday
1	Male	0	Sun	Sunday
2	Male	0	Sun	Sunday

5. Sets (set): The “Deduplication Expert” with Unordered Uniqueness

Sets have unique and unordered elements, suitable for deduplication and group relationship analysis (intersection, difference).

Core Operations:

• Deduplication: <span>set(list_with_duplicates)</span>
• Set Operations: Intersection (<span>&</span>), Union (<span>|</span>), Difference (<span>-</span>)

# Scenario: User group analysis
# Simulate three groups of user ID data
week1_users = {101, 102, 103, 104, 105}  # Active users in the first week
week2_users = {103, 104, 105, 106, 107}  # Active users in the second week
vip_users = {102, 104, 106}  # VIP users

# 1. Calculate users active for two consecutive weeks (intersection)
repeat_users = week1_users & week2_users
print("Users active for two consecutive weeks:", repeat_users)

# 2. Calculate new users (those in the second week but not in the first)
new_users = week2_users - week1_users
print("New users in the second week:", new_users)

# 3. Calculate the proportion of VIP users
vip_ratio = len(week1_users & vip_users) / len(week1_users)
print("Proportion of VIP users in the first week: {:.0%}".format(vip_ratio))

# 4. Deduplicate and merge all users
all_users = week1_users | week2_users
print("Total unique users:", len(all_users))

Output Result:

Users active for two consecutive weeks: {103, 104, 105}
New users in the second week: {106, 107}
Proportion of VIP users in the first week: 20%
Total unique users: 7

Analyst Perspective: Different data structures have different “best application scenarios” – lists are suitable for batch processing ordered data, tuples are suitable for storing fixed configurations, dictionaries are suitable for value mapping, and sets are suitable for group relationship analysis. Remember: using the right data structure can make your code ten times simpler and improve efficiency a hundredfold.

Final Thoughts: Basic Syntax is the “Inner Skill” of Data Analysis

Many people want to jump straight into learning pandas or machine learning when learning Python, but in actual work, you will find that even when using pandas, some people write concise and efficient code while others write lengthy and error-prone code, and the difference lies in the mastery of basic syntax.

When you can skillfully use list comprehensions to process batch features, quickly encode categorical variables with dictionary mappings, analyze user groups with set operations, and clean messy text with string operations, you will find that the data analysis problems that once troubled you have already been answered by these basic syntaxes.

It is recommended that you copy the code from this article into Jupyter and run it to observe the output results of each line of code – hands-on practice is the key to mastery.

If you find this useful, feel free to share it with friends who are improving their data analysis skills~