Understanding HTTP Request Methods

In network communication, HTTP request methods are the core of interaction between the client and the server. They define the type of operation the client wishes to perform on the server resources. For Python web scraping developers, understanding these methods is crucial as they determine how to send requests and retrieve data from the target website. Below are the common HTTP request methods along with their characteristics and application scenarios.

1. Common HTTP Request Methods

1.1 GET Method

• Function: Retrieve resources or data from the server.
• Characteristics:
• Parameters are appended to the request path via the URL query string.
• Idempotency: Sending the same request multiple times will not change the server state.
• Application Scenarios: Scraping web content, search queries, retrieving images or JSON data.
• Example:

  import requests
  
  url = "https://example.com/search"
  params = {"q": "Python"}
  response = requests.get(url, params=params)
  
  print("Status Code:", response.status_code)
  print("URL:", response.url)
  print("Content:", response.text)
  

1.2 POST Method

• Function: Submit data (such as forms, files) to the server or create resources.
• Characteristics:
• Data is included in the request body rather than the URL.
• Non-idempotent: Repeated submissions may lead to the creation of multiple identical resources on the server.
• Application Scenarios: User login, submitting comments, uploading files.
• Example:

  url = "https://example.com/login"
  data = {"username": "user", "password": "pass"}
  
  response = requests.post(url, data=data)
  
  print("Status Code:", response.status_code)
  print("Response Content:", response.text)
  

1.3 PUT Method

• Function: Update or replace resources on the server.
• Characteristics:
• Idempotency: Sending the same request multiple times does not change the server state.
• Suitable for modifying entire resources.
• Application Scenarios: Updating user information, replacing files.
• Example:

  url = "https://example.com/resource/123"
  data = {"name": "Alice", "age": 30}
  
  response = requests.put(url, json=data)
  print("Status Code:", response.status_code)
  

1.4 DELETE Method

• Function: Request the server to delete the specified resource.
• Characteristics:
• Idempotency: Sending the delete request multiple times yields the same result.
• Application Scenarios: Deleting files, removing data records.
• Example:

  url = "https://example.com/resource/123"
  
  response = requests.delete(url)
  print("Status Code:", response.status_code)
  

1.5 HEAD Method

• Function: Similar to GET, but only returns the response headers without the response body.
• Characteristics:
• Used to check if a resource exists, reducing data transfer.
• Application Scenarios: Validating link validity, checking resource status.
• Example:

  url = "https://example.com/resource/123"
  
  response = requests.head(url)
  print("Status Code:", response.status_code)
  print("Response Headers:", response.headers)
  

1.6 PATCH Method

• Function: Partially update a resource.
• Characteristics:
• Idempotency: Only modifies specific fields of the resource.
• Unlike PUT, it retains unmodified fields.
• Application Scenarios: Modifying a single field, such as updating a username or password.
• Example:

  url = "https://example.com/resource/123"
  data = {"age": 35}
  
  response = requests.patch(url, json=data)
  print("Status Code:", response.status_code)
  

1.7 OPTIONS Method

• Function: Retrieve the list of HTTP methods supported by the server.
• Characteristics:
• Typically used for debugging, helping to understand server capabilities.
• Example:

  url = "https://example.com"
  
  response = requests.options(url)
  print("Supported Methods:", response.headers.get("Allow"))
  

2. Comparison of GET and POST

3. Practice: Combining GET and POST Applications

The following code simulates sending a search request via GET and submitting a login form via POST:

import requests

# Simulate search (GET request)
search_url = "https://example.com/search"
search_params = {"query": "Python"}
search_response = requests.get(search_url, params=search_params)
print("Search Results:", search_response.text)

# Simulate login (POST request)
login_url = "https://example.com/login"
login_data = {"username": "user", "password": "pass"}
login_response = requests.post(login_url, data=login_data)
print("Login Response:", login_response.text)