Requests: A Simple HTTP Library for Python

Hello everyone! Today I want to share with you Requests, a very popular and easy-to-use Python HTTP library. Whether you want to retrieve data from a webpage or send requests to a server, Requests makes everything simple and clear. Next, we will explore the basic usage of Requests, common features, and practical application scenarios together.

What is Requests?

Requests is a Python library for sending HTTP requests. Its design goal is to make HTTP requests more user-friendly, avoiding the cumbersome and complex nature of using Python’s built-in urllib library. With Requests, we can easily send GET, POST, and other requests and handle response data.

Tip

Before we begin, make sure you have installed Requests. You can install it using the following command:

pip install requests

Sending GET Requests

GET requests are the most common type of HTTP request, typically used to retrieve data from a server. Here’s a simple example:

import requests

# Send a GET request
response = requests.get('https://api.github.com')

# Output response status code and content
print(response.status_code)  # Output status code
print(response.json())        # Output response JSON data

In this example, we sent a GET request to GitHub’s API and printed the response’s status code and content. A status code of 200 indicates that the request was successful.

Note

When handling response data, using the response.json() method can directly convert the JSON formatted response content into a Python dictionary.

Sending POST Requests

POST requests are typically used to send data to a server. Here’s an example of sending a POST request:

import requests

# Define the data to send
data = {'username': 'testuser', 'password': 'testpass'}

# Send a POST request
response = requests.post('https://httpbin.org/post', data=data)

# Output response content
print(response.json())  # Output data returned by the server

In this example, we sent a POST request to httpbin.org with a username and password. The server returns the data we sent.

Tip

Using the data parameter sends form data, while using the json parameter sends JSON data directly. For example:

response = requests.post('https://httpbin.org/post', json=data)

Handling Request Headers

Sometimes we need to add custom request headers to our requests. Here’s how to set request headers:

import requests

# Define request headers
headers = {'User-Agent': 'my-app'}

# Send a GET request and add request headers
response = requests.get('https://httpbin.org/headers', headers=headers)

# Output response content
print(response.json())  # Output request header information

In this example, we set a custom User-Agent request header, and the server returns the request header information we sent.

Note

In some cases, the server may return different responses based on the request headers, so it’s important to set appropriate request headers.

Handling Responses

Requests provides various methods to handle response data. In addition to getting the status code and content, we can also check other information about the response. For example:

import requests

response = requests.get('https://api.github.com')

# Check if the response was successful
if response.ok:
    print('Request succeeded!')
else:
    print('Request failed, status code:', response.status_code)

In this example, we use the response.ok property to check if the request was successful.

Tip

The response.raise_for_status() method can raise an exception if the request fails, making it easier for us to handle errors.

Handling Exceptions

When making network requests, various exceptions may occur, such as network connection failures, timeouts, etc. We can use try...except statements to handle these exceptions:

import requests

try:
    response = requests.get('https://api.github.com', timeout=5)  # Set timeout
    response.raise_for_status()  # Check if the request was successful
    print(response.json())
except requests.exceptions.HTTPError as errh:
    print("HTTP error:", errh)
except requests.exceptions.ConnectionError as errc:
    print("Connection error:", errc)
except requests.exceptions.Timeout as errt:
    print("Timeout error:", errt)
except requests.exceptions.RequestException as err:
    print("Request error:", err)

In this example, we catch various possible exceptions and output the corresponding error messages.

Note

Setting a timeout is a good practice to avoid requests hanging for a long time.

Practical Application Scenarios

The Requests library has wide applications in data scraping, API calls, and web crawling. For example, we can use Requests to scrape data from web pages or call third-party APIs to get real-time information.

Exercises

1. Try using Requests to get weather information from a public API and parse the response data.
2. Implement a simple web crawler to scrape the title of a webpage.

Summary

Today, we learned the basic usage of Requests, including sending GET and POST requests, handling request headers and responses, and exception handling. Requests makes HTTP requests simple and understandable, helping us perform network programming more efficiently.

I hope everyone can apply what they have learned in real projects and continue to explore the infinite possibilities of Python! If you have any questions, feel free to reach out. Happy coding!