In the digital age, HTTP requests are the core bridge connecting local programs with web services—whether querying weather, calling API interfaces, scraping public data, or automating form submissions and testing web services, efficient HTTP communication is essential. Have you ever struggled with the cumbersome syntax of the urllib library? Have you written excessive code to handle cookies, headers, and proxy configurations? Have you felt confused when dealing with GET/POST requests and file uploads/downloads? The Requests library, as the most popular HTTP client library in the Python ecosystem, simplifies all HTTP operations with a “user-friendly” design philosophy, encapsulating complex network communication into an intuitive API. It supports cookie persistence, session management, SSL verification, file uploads, and other full-scenario needs, allowing easy network data interaction without needing to deeply understand the details of the HTTP protocol. It is widely used in daily data queries, API development and testing, automation scripts, and public data scraping, becoming an essential “Swiss Army knife” for network communication for programmers.
1. Installing the Requests Library
Requests is a third-party library with a simple and efficient installation process, supporting all platforms:
1. Basic installation: Open the command line (Windows cmd/macOS/Linux terminal) and enter the following command:
pip install requests2. Accelerated installation (using domestic mirror sources to solve slow download issues):
pip install requests -i https://pypi.tuna.tsinghua.edu.cn/simpleAfter installation, enter<span>import requests</span> in your Python script. If there are no errors, the installation is successful (this is the industry-standard import method, no abbreviation needed).
2. Basic Usage of Requests (Four Steps to Get Started)
1. Sending a GET Request: Fetching Network Data
GET requests are the most commonly used HTTP method for retrieving data from a server (such as querying weather or obtaining public API data):
import requests# 1. Basic GET request (fetching public API data, such as Douban Movie Top 250)url = "https://api.douban.com/v2/movie/top250"response = requests.get(url)# 2. Check the response resultprint(f"Response status code: {response.status_code}") # 200 indicates successprint(f"Response content (text format): {response.text[:200]}") # Preview the first 200 charactersprint(f"Response content (JSON format): {response.json()}") # Automatically parse JSON data (commonly returned by APIs)# 3. GET request with parameters (e.g., specifying query page number and count)params = {"start": 0, "count": 10} # Page 1, return 10 itemsresponse = requests.get(url, params=params)print("Parameter request result:", response.json()["subjects"][0]["title"])2. Sending a POST Request: Submitting Data
POST requests are used to submit data to a server (such as login forms, submitting comments, uploading data):
# 1. Basic POST request (submitting form data)url = "https://httpbin.org/post" # Test interface that returns submitted datadata = {"username": "test_user", "password": "123456"} # Form dataresponse = requests.post(url, data=data)# 2. Submitting JSON format data (commonly used in API interfaces)json_data = {"name": "Zhang San", "age": 25, "city": "Beijing"}response = requests.post(url, json=json_data)print("JSON submission result:", response.json()["json"])# 3. Including request headers (simulating browser access to avoid being rejected by the server)headers = { "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"}response = requests.post(url, json=json_data, headers=headers)3. Handling Responses: Status Codes and Exceptions
Network requests may fail (e.g., broken links, server errors), so it is necessary to ensure program robustness through status codes and exception handling:
# 1. Status code judgment (2xx success, 3xx redirection, 4xx client error, 5xx server error)response = requests.get(url)if response.status_code == 200: print("Request successful!")elif response.status_code == 404: print("Request address does not exist!")elif response.status_code == 500: print("Internal server error!")# 2. Exception handling (network errors, timeouts, etc.)try: # Set timeout (error if no response within 5 seconds) response = requests.get(url, timeout=5) response.raise_for_status() # Directly raise exceptions for 4xx/5xx status codesexcept requests.exceptions.ConnectionError: print("Network connection failed!")except requests.exceptions.Timeout: print("Request timed out!")except requests.exceptions.HTTPError as e: print(f"HTTP error: {e}")4. Cookies and Session Management
Session management is used to maintain login status (e.g., accessing pages that require permissions after logging in) to avoid resubmitting login information:
# 1. Directly obtaining response cookiesresponse = requests.get("https://www.baidu.com")print("Baidu Cookie:", response.cookies.get_dict())# 2. Session management (Session object, automatically saves cookies)session = requests.Session()# Step 1: Simulate login (assuming the login interface requires username and password)login_url = "https://example.com/login"session.post(login_url, data={"username": "test", "password": "123"})# Step 2: Access the page that requires login (automatically carries login cookies)profile_url = "https://example.com/profile"response = session.get(profile_url)print("Personal information after login:", response.text[:300])3. Advanced Usage of Requests
1. File Upload and Download
Supports uploading local files to the server or downloading files from the web to local (e.g., downloading images, documents):
# 1. File upload (simulating image upload)upload_url = "https://httpbin.org/post"files = {"file": open("test.jpg", "rb")} # Open file (rb: binary read mode)response = requests.post(upload_url, files=files)print("File upload result:", response.json()["files"])files["file"].close() # Close file# 2. File download (downloading a network image and saving it)download_url = "https://picsum.photos/200/300" # Random image interfaceresponse = requests.get(download_url, stream=True) # stream=True: stream download (suitable for large files)if response.status_code == 200: with open("downloaded_img.jpg", "wb") as f: for chunk in response.iter_content(chunk_size=1024): # Write in chunks to avoid excessive memory usage if chunk: f.write(chunk) print("File download completed!")2. Proxy Configuration (Bypassing IP Restrictions)
Used in scenarios such as web scraping and cross-regional access to hide the real IP address:
# 1. Ordinary proxy (HTTP/HTTPS)proxies = { "http": "http://127.0.0.1:8080", "https": "https://127.0.0.1:8080"}response = requests.get("https://httpbin.org/ip", proxies=proxies)print("Proxy IP:", response.json()["origin"])# 2. Authenticated proxy (requires username and password)proxies = { "http": "http://username:[email protected]:8080", "https": "https://username:[email protected]:8080"}response = requests.get("https://httpbin.org/ip", proxies=proxies)4. Practical Application Scenarios
1. Daily Data Query Tool
Scenario: Writing scripts to query real-time weather, stock market information, and logistics information, automatically pushing results to email/WeChat. Core operations: calling public APIs (such as weather APIs) → sending GET requests → parsing JSON responses → extracting key information.
2. API Development and Testing
Scenario: Backend developers testing their own API interfaces, verifying GET/POST requests, parameter validation, and response formats meet requirements. Core operations: constructing request parameters → sending requests → asserting response status codes and data → batch automated testing.
3. Public Data Scraping
Scenario: Scraping public industry data, news information, and academic paper abstracts (must comply with website robots protocols) for data analysis. Core operations: setting request headers to simulate browser → paginating data retrieval → parsing response content → saving to local files/databases.
4. Automated Office Scripts
Scenario: Automatically logging into office systems, downloading daily report templates, submitting weekly reports, and querying attendance records to reduce repetitive operations. Core operations: maintaining login status with Session → simulating form submissions → downloading files → automatically filling templates.
5. In-Depth Case Code (Real-Time Weather Query and Push Tool)
import requestsimport smtplibfrom email.mime.text import MIMETextdef get_weather(city: str, api_key: str) -> str: """ Call weather API to get real-time weather :param city: City name (e.g., "Beijing") :param api_key: Weather API key (can be applied from platforms like Juhe Data) :return: Weather information string """ url = "http://v.juhe.cn/tianqi/index" params = { "cityname": city, "key": api_key, "format": 2 # Return JSON format } try: response = requests.get(url, params=params, timeout=10) response.raise_for_status() data = response.json() if data["error_code"] != 0: return f"Query failed: {data['reason']}" # Extract key weather information realtime = data["result"]["realtime"] # Real-time weather future = data["result"]["future"][0] # Today's forecast weather_info = ( f"【{city} Today's Weather】\n" f"Real-time Temperature: {realtime['temperature']}℃\n" f"Weather Condition: {realtime['info']}\n" f"Wind Direction and Force: {realtime['direct']} {realtime['power']} level\n" f"Today's Forecast: {future['weather']}\n" f"Temperature Range: {future['temperature']}\n" f"Friendly Reminder: {realtime['info']}, {future['notice']}" ) return weather_info except Exception as e: return f"Weather query failed: {str(e)}"def send_email(to_email: str, subject: str, content: str, smtp_config: dict): """ Send weather information to email :param to_email: Recipient email :param subject: Email subject :param content: Email content :param smtp_config: SMTP configuration (server, port, username, password) """ msg = MIMEText(content, "plain", "utf-8") msg["Subject"] = subject msg["From"] = smtp_config["username"] msg["To"] = to_email try: # Connect to SMTP server and send email server = smtplib.SMTP_SSL(smtp_config["server"], smtp_config["port"]) server.login(smtp_config["username"], smtp_config["password"]) server.send_message(msg) server.quit() print(f"Weather information has been sent to: {to_email}") except Exception as e: print(f"Email sending failed: {str(e)}")# Main function: Query weather and send emailif __name__ == "__main__": # Configuration parameters (replace with your own information) CITY = "Shanghai" API_KEY = "your_weather_api_key" # Replace with real API key SMTP_CONFIG = { "server": "smtp.qq.com", # QQ email SMTP server "port": 465, # SSL port "username": "[email protected]", # Sender email "password": "your_smtp_password" # Email SMTP authorization code } TO_EMAIL = "[email protected]" # Recipient email # Execute process weather_info = get_weather(CITY, API_KEY) print(weather_info) send_email( to_email=TO_EMAIL, subject=f"{CITY} Today's Weather Forecast", content=weather_info, smtp_config=SMTP_CONFIG )The core value of the Requests library lies in “simplifying complexity”; it shields the underlying details of the HTTP protocol, making network requests simple and understandable through an intuitive API. Whether you are a beginner or an experienced programmer, you can quickly get started to meet various network interaction needs. From daily data queries to complex API testing and automation script development, Requests can enhance development efficiency with concise code, becoming an indispensable foundational tool in Python programming. Its powerful compatibility and rich functional extensions allow it to adapt to most network scenarios while maintaining code readability and maintainability.
Do you often need to interact with web services in your daily programming? For example, calling API interfaces, scraping public data, writing automation scripts, etc., have you encountered issues such as request failures, complex cookie handling, and troublesome proxy configurations? Why not try to refactor the relevant logic using the Requests library and experience the convenience of “one line of code to handle HTTP requests”? Feel free to share your usage scenarios, encountered problems, or practical tips in the comments section, and let us discuss together to make network programming more efficient and simpler!