Requests: The King of HTTP Requests in Python!

Today, let’s talk about the Requests library. When it comes to sending HTTP requests in Python, Requests is absolutely the king! Whether you’re scraping web data or calling API endpoints, Requests can make your work much more efficient.

Installing Requests

Installing Requests is very simple; just type the following in the command line:

pip install requests

Requests will be installed smoothly, even more obediently than pip, haha.

Sending GET Requests

To send a GET request, it only takes one line of code with Requests:

response = requests.get('https://api.github.com')

This line of code is like making a phone call to GitHub’s API endpoint, and the response variable receives GitHub’s reply. Beautiful!

Want to send some parameters along? No problem:

params = {'per_page': 10, 'page': 1}
response = requests.get('https://api.github.com/users', params=params)

Sending POST Requests

Sending POST requests with Requests is just as smooth. For example, to submit some data to an API endpoint:

data = {'key': 'value'}
response = requests.post('https://httpbin.org/post', data=data)

Data is passed in through the data parameter, simple and clear.

If you want to send data in JSON format, it’s just as easy:

import json

data = {'key': 'value'}
response = requests.post('https://httpbin.org/post', data=json.dumps(data))

Friendly reminder: Remember to use json.dumps() to convert the dictionary into a JSON string!

Adding Request Headers

Some API endpoints may have special requirements for request headers, such as needing a User-Agent. Requests can handle that:

headers = {'User-Agent': 'Mozilla/5.0'}
response = requests.get('https://api.github.com', headers=headers)

Response Content

Once you have the response object, Requests provides many attributes and methods to conveniently retrieve various information about the response. For example:

print(response.text)  # Response content (string format)
print(response.content)  # Response content (byte format)
print(response.status_code)  # Status code
print(response.headers)  # Response headers

If the response content is in JSON format, you can use response.json() to directly parse it into a Python object:

data = response.json()

This way, obtaining data from various APIs is incredibly convenient!

Automatic Cookie Handling

Cookies are very common in web requests, and Requests can automatically handle cookies, so you don’t have to worry about it. For example, after logging into GitHub, when you access another page, Requests will automatically carry the logged-in cookies:

session = requests.Session()
session.get('https://github.com/login') 

# Login operation
# ...

# Access other pages, automatically carrying the logged-in cookies 
response = session.get('https://github.com/settings/profile')

Create a session with requests.Session(), and all requests in this session will automatically handle cookies. Super considerate!

Requests has many powerful features, but I won’t list them all today. Once you use Requests, you’ll find that sending HTTP requests in Python can be so elegant, so simple, and so delightful!

That concludes this Python learning sharing session. We learned:

How to install Requests
How to send GET and POST requests with Requests
How to add parameters and headers to requests
How to retrieve various information from responses
How to automatically handle cookies with Requests

If you don’t use Requests in your web scraping scripts, what’s the difference from being a salted fish! Go try Requests, it will make your Python journey much smoother! If you have any questions, feel free to ask me in the comments section, and let’s improve together.