Python Basics: Making Network Requests with the Requests Library

When learning Python, we often need to interact with the web, such as scraping web pages, calling third-party APIs, or retrieving data from backend services. Although Python’s built-in urllib is comprehensive, it can feel a bit cumbersome to use. At this point, let me introduce a “great helper”—requests. It allows you to perform various HTTP request operations with extremely simple code.

1. What is the Requests Library?

Requests is a third-party Python library designed to simplify the handling of HTTP requests and responses. It is very popular in the Python networking programming community, known as “HTTP for Humans“, meaning its API design aligns with human thinking, making it easy to understand how to use.

Easy to Install

Install it using pip install requests, without needing to write a lot of configuration.

Intuitive Syntax

Sending a request can be done in one line with requests.get(url) or requests.post(url, data=...);
The response returns a Response object, and you can directly access the content with response.text or response.json();

Powerful Features

Supports all common HTTP request methods, as well as advanced features like SSL certificate verification, session handling, and proxy settings.

Supports Advanced Usage

Can easily handle advanced scenarios such as custom request headers, cookies, session persistence, proxies, and SSL verification.

In summary, requests makes writing network requests more elegant and readable. If you are writing web scrapers, data collection, or calling Web APIs, this library is almost essential.

2. Preparation: Installing Requests

Requests is not part of the Python standard library, so you need to install it using pip for the first time:

pip install requests

After installation, you can use it in your script by import requests. If you are using Anaconda, it may already have requests built-in, so you can try import requests in your environment to see if it works.

To verify if the installation was successful:

import requests
print(requests.__version__)

# Output
2.31.0

3. Sending GET Requests

GET requests are the most commonly used network requests, used to retrieve resources or data from the server (such as web pages or JSON returned by APIs).

3.1 Basic Usage

import requests

url = "https://api.github.com/users/octocat"
response = requests.get(url)
print("Status Code:", response.status_code)  # 200 indicates success
print("Response Text:", response.text)

# Output
Status Code: 200
Response Text: {"login":"octocat","id":583231,"node_id":"MDQ6VXNlcjU4MzIzMQ==","avatar_url":"https://avatars.githubusercontent.com/u/583231?v=4","gravatar_id":"","url":"https://api.github.com/users/octocat","html_url":"https://github.com/octocat","followers_url":"https://api.github.com/users/octocat/followers","following_url":"https://api.github.com/users/octocat/following{/other_user}","gists_url":"https://api.github.com/users/octocat/gists{/gist_id}","starred_url":"https://api.github.com/users/octocat/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/octocat/subscriptions","organizations_url":"https://api.github.com/users/octocat/orgs","repos_url":"https://api.github.com/users/octocat/repos","events_url":"https://api.github.com/users/octocat/events{/privacy}","received_events_url":"https://api.github.com/users/octocat/received_events","type":"User","user_view_type":"public","site_admin":false,"name":"The Octocat","company":"@github","blog":"https://github.blog","location":"San Francisco","email":null,"hireable":null,"bio":null,"twitter_username":null,"public_repos":8,"public_gists":8,"followers":17445,"following":9,"created_at":"2011-01-25T18:44:36Z","updated_at":"2025-03-22T11:26:21Z"}
Response Content: b'{"login":"octocat","id":583231,"node_id":"MDQ6VXNlcjU4MzIzMQ==","avatar_url":"https://avatars.githubusercontent.com/u/583231?v=4","gravatar_id":"","url":"https://api.github.com/users/octocat","html_url":"https://github.com/octocat","followers_url":"https://api.github.com/users/octocat/followers","following_url":"https://api.github.com/users/octocat/following{/other_user}","gists_url":"https://api.github.com/users/octocat/gists{/gist_id}","starred_url":"https://api.github.com/users/octocat/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/octocat/subscriptions","organizations_url":"https://api.github.com/users/octocat/orgs","repos_url":"https://api.github.com/users/octocat/repos","events_url":"https://api.github.com/users/octocat/events{/privacy}","received_events_url":"https://api.github.com/users/octocat/received_events","type":"User","user_view_type":"public","site_admin":false,"name":"The Octocat","company":"@github","blog":"https://github.blog","location":"San Francisco","email":null,"hireable":null,"bio":null,"twitter_username":null,"public_repos":8,"public_gists":8,"followers":17445,"following":9,"created_at":"2011-01-25T18:44:36Z","updated_at":"2025-03-22T11:26:21Z"}'
Response Headers: {'Date': 'Sat, 29 Mar 2025 17:51:09 GMT', 'Content-Type': 'application/json; charset=utf-8', 'Cache-Control': 'public, max-age=60, s-maxage=60', 'Vary': 'Accept,Accept-Encoding, Accept, X-Requested-With', 'ETag': 'W/"b14760375f79bd0cdc23bdc4704a63213084529a84eb61b229bb75ebc36854ea"', 'Last-Modified': 'Sat, 22 Mar 2025 11:26:21 GMT', 'X-GitHub-Media-Type': 'github.v3; format=json', 'x-github-api-version-selected': '2022-11-28', 'Access-Control-Expose-Headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset', 'Access-Control-Allow-Origin': '*', 'Strict-Transport-Security': 'max-age=31536000; includeSubdomains; preload', 'X-Frame-Options': 'deny', 'X-Content-Type-Options': 'nosniff', 'X-XSS-Protection': '0', 'Referrer-Policy': 'origin-when-cross-origin, strict-origin-when-cross-origin', 'Content-Security-Policy': "default-src 'none'", 'Content-Encoding': 'gzip', 'Server': 'github.com', 'Accept-Ranges': 'bytes', 'X-RateLimit-Limit': '60', 'X-RateLimit-Remaining': '54', 'X-RateLimit-Reset': '1743271352', 'X-RateLimit-Resource': 'core', 'X-RateLimit-Used': '6', 'Content-Length': '495', 'X-GitHub-Request-Id': 'A5A6:3C1181:DD94357:E47D8BB:67E8330D'}

Here we requested a public API from GitHub (to get user information). The result is that response is a Response object that contains rich information:

response.status_code: HTTP status code, such as 200 (OK), 404 (Not Found), 500 (Server Error), etc.;
response.text: The text content returned by the server (usually HTML or JSON);
response.content: The binary content returned by the server (such as downloaded files or images);
response.headers: Response header information;
response.json(): If the returned value is in JSON format, it can be directly parsed into a Python dict/list, which is very convenient.

Tip: The common status code for a successful request is 200; a 404 indicates that the resource does not exist; a 500 indicates a server error; for a more comprehensive understanding, you can refer to the “HTTP Status Codes” table.

3.2 GET Request Parameters

Typically, GET requests include parameters in the URL, such as ?key=value. In requests, you can specify it using the params= parameter, which will automatically append it to the URL:

import requests

url = "https://httpbin.org/get"
params = {"name": "Alice", "age": 20}
response = requests.get(url, params=params)
print("Actual Request URL:", response.url)  # View the concatenated URL
print("Returned JSON:", response.json())

# Output
Actual Request URL: https://httpbin.org/get?name=Alice&amp;age=20
Returned JSON: {'args': {'age': '20', 'name': 'Alice'}, 'headers': {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate', 'Host': 'httpbin.org', 'User-Agent': 'python-requests/2.31.0', 'X-Amzn-Trace-Id': 'Root=1-67e83340-3a6374351388e09c0607a12d'}, 'origin': '143.198.110.169', 'url': 'https://httpbin.org/get?name=Alice&amp;age=20'}

In the above example, params={"name":"Alice","age":20} will make the final URL become https://httpbin.org/get?name=Alice&age=20. On the server side, name and age can be recognized. This is much cleaner than manually concatenating strings.

3.3 Custom Request Headers

Sometimes we may need to customize request headers (HTTP headers), such as:

Disguising as a browser to access a website (to avoid being blocked by scraping).
Submitting data in a special format (such as JSON).
Carrying cookies or authentication information after logging in.

Requests module allows you to easily customize request headers using the headers parameter:

headers = {
    "User-Agent": "MyPythonClient/1.0",
    "Accept": "application/json"
}
response = requests.get("https://httpbin.org/headers", headers=headers)
print(response.json())

# Output
{'headers': {'Accept': 'application/json', 'Accept-Encoding': 'gzip, deflate', 'Host': 'httpbin.org', 'User-Agent': 'MyPythonClient/1.0', 'X-Amzn-Trace-Id': 'Root=1-67e83376-1342537147a5c1ab797d3d67'}}

The response contains the returned headers, proving that the request headers were sent correctly. For beginners, just remember headers= to handle most custom header needs.

Common request headers:

Request Header	Function	Example
`<span>User-Agent</span>`	Disguise client type	`<span>Mozilla/5.0 (Windows NT 10.0; Win64; x64)</span>`
`<span>Referer</span>`	Indicate where the server was redirected from	`<span>https://www.google.com</span>`
`<span>Cookie</span>`	Carry login information	`<span>sessionid=abc123</span>`
`<span>Content-Type</span>`	Specify data format	`<span>application/json</span>`
`<span>Authorization</span>`	Login authentication credentials	`<span>Bearer token123</span>`

4. Sending POST Requests

POST requests are commonly used to submit data to the server, such as uploading forms or submitting JSON information. Unlike GET, POST usually places data in the request body, rather than in the URL.

4.1 Form Data

If you want to submit key-value pair data similar to a form, you can use the data= parameter, such as:

import requests

url = "https://httpbin.org/post"
data = {
    "username": "Bob",
    "password": "1234"
}
response = requests.post(url, data=data)
print("Status Code:", response.status_code)
print("Returned Content:", response.text)

# Output
Status Code: 200
Returned Content: {
"args": {},
"data": "",
"files": {},
"form": {
    "password": "1234",
    "username": "Bob"
  },
"headers": {
    "Accept": "*/*",
    "Accept-Encoding": "gzip, deflate",
    "Content-Length": "26",
    "Content-Type": "application/x-www-form-urlencoded",
    "Host": "httpbin.org",
    "User-Agent": "python-requests/2.31.0",
    "X-Amzn-Trace-Id": "Root=1-67e833bc-2d76db1a4108eec320dca8bf"
  },
"json": null,
"origin": "143.198.110.169",
"url": "https://httpbin.org/post"
}

data= will encode the dictionary into application/x-www-form-urlencoded format, such as username=Bob&password=1234. In most form submission scenarios, this is sufficient.

Related Knowledge Points

application/x-www-form-urlencoded format:

application/x-www-form-urlencoded is an encoding format used to submit data in HTTP requests, typically used in web form submissions.

Its format is very simple:

Connect the key and value with an equal sign (=), forming key=value.
If there are multiple key-value pairs, connect each pair with an ampersand (&).

Part	Meaning
`<span>application</span>`	Indicates that the data is an application data format
`<span>x-www-form</span>`	Indicates that this format is used in web forms (WWW)
`<span>urlencoded</span>`	Indicates that URL encoding is used

URL encoding (urlencoded):

Encodes special characters in the data to ensure that the data does not become ambiguous during transmission.

URL encoding (also known as percent encoding) is a special encoding method used to convert characters that are difficult to transmit (such as spaces and Chinese characters) into a form that is easy to transmit.

For example, a space ( ) → + or %20
For example, the Chinese character ‘Beijing’ → %E5%8C%97%E4%BA%AC

4.2 Submitting JSON Data

More and more APIs expect you to use JSON to pass POST content. In requests, you can use the json= parameter to automatically convert the dictionary to JSON and add the Content-Type: application/json header:

url = "https://httpbin.org/post"
payload = {"title": "Hello", "content": "This is a test"}
response = requests.post(url, json=payload)
print(response.json())

# Output
{'args': {}, 'data': '{"title": "Hello", "content": "This is a test"}', 'files': {}, 'form': {}, 'headers': {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate', 'Content-Length': '47', 'Content-Type': 'application/json', 'Host': 'httpbin.org', 'User-Agent': 'python-requests/2.31.0', 'X-Amzn-Trace-Id': 'Root=1-67e835d2-61d7a6f54169b4006aaa6a07'}, 'json': {'content': 'This is a test', 'title': 'Hello'}, 'origin': '143.198.110.169', 'url': 'https://httpbin.org/post'}

The server will receive the JSON format {"title":"Hello","content":"This is a test"}. For Python beginners, this is more convenient than serializing JSON yourself and writing it to the body.

4.3 File Upload

If you want to upload local files, such as images or Excel files, you can use the files= parameter:

import requests

url = "https://httpbin.org/post"
files = {"myfile": open("test.jpg", "rb")}
response = requests.post(url, files=files)
print(response.json())

# Output
{'args': {}, 'data': '', 'files': {'myfile': ''}, 'form': {}, 'headers': {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate', 'Content-Length': '146', 'Content-Type': 'multipart/form-data; boundary=49415e27bbb5e7195a216220682f7ce8', 'Host': 'httpbin.org', 'User-Agent': 'python-requests/2.31.0', 'X-Amzn-Trace-Id': 'Root=1-67e8369f-299552a2403324db6c8579dd'}, 'json': None, 'origin': '143.198.110.169', 'url': 'https://httpbin.org/post'}

files dictionary’s key is the field name (the backend interface recognizes the file based on this name), and the value is an opened file object (in binary read mode). Requests will send it in multipart/form-data format. The server can then retrieve the file after parsing.

Related Knowledge Points

multipart/form-data format:

multipart/form-data is a format used to submit data in HTTP requests, typically used for:

Uploading files (images, videos, documents)
Submitting forms with files (submitting files and regular data together)

It differs from the ordinary application/x-www-form-urlencoded in that:

It can combine file data and regular form data in the same request.

4.4 File Download

Using requests to download files is actually very simple; you just need to:

Send a request using requests.get(url).
Save the returned content to a file.

import requests

# File URL
url = "https://www.python.org/static/img/python-logo.png"

# Send request
response = requests.get(url)

# Check if the request was successful (status code 200)
if response.status_code == 200:
    # Save content to file (write in binary mode for images)
    with open("python-logo.png", "wb") as f:
        f.write(response.content)
    print("Download successful, filename: python-logo.png")
else:
    print("Download failed, status code:", response.status_code)

# Output
Download successful, filename: python-logo.png

![python-logo](F:\my_github_repo\MyArticle\Python基础知识系列\Python基础：使用 requests 库进行网络请求\python-logo.png)

The above code downloads the Python logo image from the official Python website:

requests.get(url) retrieves the file data.
The file data is stored in response.content (binary data).
Write it using binary mode "wb".

5. Handling Responses

When we use requests.get or requests.post, we get a response object. This response object contains many useful attributes and methods that allow us to view results and perform subsequent processing.

This response data usually includes:

Status code (HTTP status code)
Response headers (headers)
Response body (web content or data)

Common Attributes and Methods in the Response Object:

Attribute/Method	Description
`<span>response.status_code</span>`	HTTP status code (e.g., 200 indicates success)
`<span>response.text</span>`	Text content of the response body (str format)
`<span>response.content</span>`	Binary content of the response body (images, files)
`<span>response.json()</span>`	If the returned content is JSON, automatically parse it into a Python dictionary
`<span>response.headers</span>`	Response header information
`<span>response.encoding</span>`	Encoding format of the returned content
`<span>response.url</span>`	Actual response URL (may have redirects)
`<span>response.cookies</span>`	Cookie information returned in the response

5.1 Getting Response Content:

The response body can be a web page, text, JSON data, image, or file.

Text Content: `response.text`

Generally used for scraping web pages or obtaining plain text content:

response = requests.get("https://httpbin.org/html")

html_content = response.text
print(html_content)

# Output
&lt;!DOCTYPE html&gt;
&lt;html&gt;
&lt;head&gt;
&lt;/head&gt;
&lt;body&gt;
      &lt;h1&gt;Herman Melville - Moby-Dick&lt;/h1&gt;

      &lt;div&gt;
        &lt;p&gt;
          Availing himself of the mild, summer-cool weather that now reigned in these latitudes, and in preparation for the peculiarly active pursuits shortly to be anticipated, Perth, the begrimed, blistered old blacksmith, had not removed his portable forge to the hold again, after concluding his contributory work for Ahab's leg, but still retained it on deck, fast lashed to ringbolts by the foremast; being now almost incessantly invoked by the headsmen, and harpooneers, and bowsmen to do some little job for them; altering, or repairing, or new shaping their various weapons and boat furniture. Often he would be surrounded by an eager circle, all waiting to be served; holding boat-spades, pike-heads, harpoons, and lances, and jealously watching his every sooty movement, as he toiled. Nevertheless, this old man's was a patient hammer wielded by a patient arm. No murmur, no impatience, no petulance did come from him. Silent, slow, and solemn; bowing over still further his chronically broken back, he toiled away, as if toil were life itself, and the heavy beating of his hammer the heavy beating of his heart. And so it was.—Most miserable! A peculiar walk in this old man, a certain slight but painful appearing yawing in his gait, had at an early period of the voyage excited the curiosity of the mariners. And to the importunity of their persisted questionings he had finally given in; and so it came to pass that every one now knew the shameful story of his wretched fate. Belated, and not innocently, one bitter winter's midnight, on the road running between two country towns, the blacksmith half-stupidly felt the deadly numbness stealing over him, and sought refuge in a leaning, dilapidated barn. The issue was, the loss of the extremities of both feet. Out of this revelation, part by part, at last came out the four acts of the gladness, and the one long, and as yet uncatastrophied fifth act of the grief of his life's drama. He was an old man, who, at the age of nearly sixty, had postponedly encountered that thing in sorrow's technicals called ruin. He had been an artisan of famed excellence, and with plenty to do; owned a house and garden; embraced a youthful, daughter-like, loving wife, and three blithe, ruddy children; every Sunday went to a cheerful-looking church, planted in a grove. But one night, under cover of darkness, and further concealed in a most cunning disguisement, a desperate burglar slid into his happy home, and robbed them all of everything. And darker yet to tell, the blacksmith himself did ignorantly conduct this burglar into his family's heart. It was the Bottle Conjuror! Upon the opening of that fatal cork, forth flew the fiend, and shrivelled up his home. Now, for prudent, most wise, and economic reasons, the blacksmith's shop was in the basement of his dwelling, but with a separate entrance to it; so that always had the young and loving healthy wife listened with no unhappy nervousness, but with vigorous pleasure, to the stout ringing of her young-armed old husband's hammer; whose reverberations, muffled by passing through the floors and walls, came up to her, not unsweetly, in her nursery; and so, to stout Labor's iron lullaby, the blacksmith's infants were rocked to slumber. Oh, woe on woe! Oh, Death, why canst thou not sometimes be timely? Hadst thou taken this old blacksmith to thyself ere his full ruin came upon him, then had the young widow had a delicious grief, and her orphans a truly venerable, legendary sire to dream of in their after years; and all of them a care-killing competency.
        &lt;/p&gt;
      &lt;/div&gt;
&lt;/body&gt;
&lt;/html&gt;

Binary Content: `response.content`

Used for downloading files or images:

response = requests.get("https://www.python.org/static/img/python-logo.png")

with open("python-logo.png", "wb") as f:
    f.write(response.content)

JSON Data: `response.json()`

Automatically converts the returned JSON format data into Python objects (dictionaries or lists):

import requests

response = requests.get("https://httpbin.org/json")

data = response.json()
print(data)

# Output
{'slideshow': {'author': 'Yours Truly', 'date': 'date of publication', 'slides': [{'title': 'Wake up to WonderWidgets!', 'type': 'all'}, {'items': ['Why &lt;em&gt;WonderWidgets&lt;/em&gt; are great', 'Who &lt;em&gt;buys&lt;/em&gt; WonderWidgets'], 'title': 'Overview', 'type': 'all'}], 'title': 'Sample Slide Show'}}

5.2 Getting Response Headers

HTTP responses usually come with header information (such as server type, content type, etc.):

import requests

response = requests.get("https://httpbin.org/get")

headers = response.headers

print("Server Type:", headers["Server"])
print("Content Type:", headers["Content-Type"])

# Output
Server Type: gunicorn/19.9.0
Content Type: application/json

5.3 Response Encoding Handling (`response.encoding`)

Sometimes, Chinese web pages may appear garbled; you can set or check the encoding:

import requests

response = requests.get("http://baidu.cn")
# Check original encoding
print("Default Encoding:", response.encoding)
# If garbled, manually set encoding
response.encoding = "utf-8"
print(response.text)

# Output
Default Encoding: ISO-8859-1
&lt;html&gt;
&lt;meta http-equiv="refresh" content="0;url=http://www.baidu.com/"&gt;
&lt;/html&gt;

5.4 Actual Request Redirect URL (`response.url`)

Some requests may be redirected to a new URL by the server; view the actual response URL:

import requests

response = requests.get("http://github.com")  # Automatically redirects to https://github.com

print("Actual Response URL is:", response.url)

# Output
Actual Response URL is: https://github.com/

5.5 Cookie Information in the Response (`response.cookies`)

Get the cookie information returned by the server in the response:

import requests

response = requests.get("https://httpbin.org/cookies/set?name=Bob")

cookies = response.cookies
print("Cookie Information:", cookies.get_dict())

# Output
Cookie Information: {}

Introduction to Cookies:

Cookies are small files temporarily stored on your computer or phone by the website you visit, used to record some of your information.

For example:

After logging into Taobao, if you close the webpage and reopen it, you will still be logged in.
After browsing an e-commerce website, the webpage can remember which products you have viewed.

These are all achieved through cookies.

Cookie Workflow:

The process is roughly as follows:

You visit a website for the first time.
The website server gives you a special data (cookie) in response to your request.
The browser stores this cookie on your computer.
When you visit this website again, the browser automatically sends this cookie back to the server.
The server can identify you and your historical information through this cookie.

5.6 Summary

Common usages:

response.status_code: Returns the status code (e.g., 200, 404, etc.) to determine if the request was successful or failed.
response.text: The response body in string format (suitable for parsing text or HTML).
response.json(): If the server returns JSON, this method can be called to automatically parse it into Python objects (dictionaries or lists), which is very common.
response.headers: View response header information (such as Content-Type, Date).
response.content: The response body in binary format (suitable for downloading files or images).

Example: Get a JSON returned by an API and then parse it:

import requests

url = "https://api.github.com/repos/psf/requests"
resp = requests.get(url)
if resp.status_code == 200:
    data = resp.json()   # Parse into dictionary
    print("Project Name:", data["name"])
    print("Project Description:", data["description"])
    print("Stars:", data["stargazers_count"])
else:
    print("Request failed, status code:", resp.status_code)

# Output
Project Name: requests
Project Description: A simple, yet elegant, HTTP library.
Stars: 52664

6. Session Handling and Persistent Connections

When we send various HTTP requests using Python’s requests module, each request is completely independent:

It does not automatically remember the last login state (cookies).
Each request has to establish a connection again, which is less efficient.

In fact, requests provides a very useful feature called:

❝

Session Handling and Persistent Connections

6.1 What is a Session?

In HTTP communication, a session refers to:

Sharing certain states (such as login state cookies) between multiple requests.
After establishing a connection once, multiple requests can be sent continuously.

6.2 Why Use a Session?

Under normal circumstances:

Each time you use requests.get() or requests.post(), you need to establish a connection again.
Cookies are not automatically carried, and the login state cannot be maintained.

After using a session:

It can automatically remember cookies and maintain the login state.
Each request does not need to establish a connection again, making it faster.

6.3 How to Use Session in Requests?

Use requests.Session() to achieve:

import requests

# Create a session object
session = requests.Session()

# Log in (assuming the website requires login)
login_url = "https://example.com/login"
data = {"username": "alice", "password": "1234"}
session.post(login_url, data=data)

# After logging in, access other pages without logging in again (cookies are automatically carried)
protected_url = "https://example.com/profile"
response = session.get(protected_url)

print(response.text)

6.4 Manually Setting and Getting Cookies in a Session

View Cookies:

import requests

session = requests.Session()
session.get("https://httpbin.org/cookies/set?sessionid=abc123")

# View cookies
print(session.cookies.get_dict())

# Output
{'sessionid': 'abc123'}

Manually Set Cookies:

import requests

session = requests.Session()
session.get("https://httpbin.org/cookies/set?sessionid=abc123")

# View cookies
print(session.cookies.get_dict())
session.cookies.set("my_cookie", "hello")
# Verify
print(session.cookies.get_dict())

# Output
{'sessionid': 'abc123'}
{'sessionid': 'abc123', 'my_cookie': 'hello'}

7. Precautions

Set Timeout: When accessing unstable services, it is best to add the timeout parameter to prevent the script from hanging for a long time. For example, requests.get(url, timeout=5).
```
import requests

try:
 response = requests.get('https://www.example.com', timeout=5)
 print(f"Response Status Code: {response.status_code}")
except requests.Timeout:
 print("Request timed out!")
```
Exception Handling: For network errors (such as connection failures) or timeouts, exceptions from requests.exceptions will be raised, which need to be caught using try-except.
Sessions and Cookies: If you need to maintain login status, you can create a session object using requests.Session() to persist cookies and other information;

Proxies: If you need to access the network through a specific proxy server, you can use the proxies parameter, such as requests.get(url, proxies={"http":"http://127.0.0.1:8888", "https":"https://... "});

import requests

proxies = {
    'http': 'http://10.10.1.10:3128',
    'https': 'http://10.10.1.10:1080',
}

response = requests.get('https://www.example.com', proxies=proxies)
print(f"Response Status Code: {response.status_code}")

SSL Verification: SSL certificate verification is enabled by default; if accessing a self-signed certificate, use verify=False or specify the certificate path.
```
import requests

response = requests.get('https://self-signed.example.com', verify=False)
print(f"Response Status Code: {response.status_code}")
```

Large File Downloads: If you want to download large files or stream read, it is recommended to use stream=True and write the file while downloading to prevent excessive memory usage.

import requests

url = "https://example.com/large_file.zip"

# stream=True means download in chunks, not immediately load into memory
response = requests.get(url, stream=True)

with open("large_file.zip", "wb") as f:
    # Read 1024 bytes of data each time and write to file
    for chunk in response.iter_content(chunk_size=1024):
        if chunk:  # Ensure chunk is not empty
            f.write(chunk)

print("Large file download completed!")

8. Summary

Requests is the most commonly used third-party HTTP library in Python, “HTTP for Humans”, helping you initiate network requests in a concise and friendly manner.
GET requests are generally made using requests.get(url, params=...); POST requests are made using requests.post(url, data=...) or json=...
After obtaining the response object, you can check the status_code (status code), and use response.text/response.json() to get and parse the returned content.
If you need to maintain login or reuse connections, you can use requests.Session()
For ordinary scenarios, requests can meet the vast majority of needs; consider other solutions only for security, large data streams, or asynchronous requirements.

If you find this article helpful, feel free to like or share it with more friends who want to learn Python. See you next time!

Python Basics: Making Network Requests with the Requests Library

1. What is the Requests Library?

2. Preparation: Installing Requests

3. Sending GET Requests

3.1 Basic Usage

3.2 GET Request Parameters

3.3 Custom Request Headers

4. Sending POST Requests

4.1 Form Data

Related Knowledge Points

4.2 Submitting JSON Data

4.3 File Upload

Related Knowledge Points

4.4 File Download

5. Handling Responses

Common Attributes and Methods in the Response Object:

5.1 Getting Response Content:

Text Content: `<span>response.text</span>`

Binary Content: `<span>response.content</span>`

JSON Data: `<span>response.json()</span>`

5.2 Getting Response Headers

5.3 Response Encoding Handling (`<span>response.encoding</span>`)

5.4 Actual Request Redirect URL (`<span>response.url</span>`)

5.5 Cookie Information in the Response (`<span>response.cookies</span>`)

Introduction to Cookies:

Cookie Workflow:

5.6 Summary

6. Session Handling and Persistent Connections

6.1 What is a Session?

6.2 Why Use a Session?

6.3 How to Use Session in Requests?

6.4 Manually Setting and Getting Cookies in a Session

View Cookies:

Manually Set Cookies:

7. Precautions

8. Summary

Leave a Comment Cancel reply

1. What is the Requests Library?

2. Preparation: Installing Requests

3. Sending GET Requests

3.1 Basic Usage

3.2 GET Request Parameters

3.3 Custom Request Headers

4. Sending POST Requests

4.1 Form Data

Related Knowledge Points

4.2 Submitting JSON Data

4.3 File Upload

Related Knowledge Points

4.4 File Download

5. Handling Responses

Common Attributes and Methods in the Response Object:

5.1 Getting Response Content:

Text Content: <span>response.text</span>

Binary Content: <span>response.content</span>

JSON Data: <span>response.json()</span>

5.2 Getting Response Headers

5.3 Response Encoding Handling (<span>response.encoding</span>)

5.4 Actual Request Redirect URL (<span>response.url</span>)

5.5 Cookie Information in the Response (<span>response.cookies</span>)

Introduction to Cookies:

Cookie Workflow:

5.6 Summary

6. Session Handling and Persistent Connections

6.1 What is a Session?

6.2 Why Use a Session?

6.3 How to Use Session in Requests?

6.4 Manually Setting and Getting Cookies in a Session

View Cookies:

Manually Set Cookies:

7. Precautions

8. Summary

Related posts

Leave a Comment Cancel reply

Text Content: `<span>response.text</span>`

Binary Content: `<span>response.content</span>`

JSON Data: `<span>response.json()</span>`

5.3 Response Encoding Handling (`<span>response.encoding</span>`)

5.4 Actual Request Redirect URL (`<span>response.url</span>`)

5.5 Cookie Information in the Response (`<span>response.cookies</span>`)