Recently, while tinkering with Python web programming, I discovered a particularly useful HTTP request library – HTTPx.

To be honest, I was a bit resistant at first since I was used to requests, but after trying it out, I found it to be fantastic!

Not only does it support synchronous requests, but it can also handle asynchronous requests, making it an incredible tool.

Getting Started

Installing HTTPx is super simple:

pip install httpx

Want to make a basic GET request? Check this out:

import httpx

Sending a GET Request

r = httpx.get(‘https://www.python.org’) print(r.status_code) # 200 print(r.text) # Outputs the webpage content

Tip: The interface design of HTTPx is very similar to requests, so if you have used requests before, the transition will be seamless, and you won’t need to change much of your code.

Asynchronous Requests

The most impressive feature of HTTPx is its asynchronous request capability. The syntax is also very straightforward:

import asyncio import httpx

async def get_url(url): async with httpx.AsyncClient() as client: r = await client.get(url) return r.status_code

async def main(): urls = [‘https://www.python.org’, ‘https://www.github.com’, ‘https://www.google.com’] tasks = [get_url(url) for url in urls] results = await asyncio.gather(*tasks) print(results)

asyncio.run(main())

This code can request multiple websites simultaneously, and the speed is impressive! Although the code looks a bit more complex, the actual running efficiency can improve several times.

Advanced Features

HTTPx also supports some fancy features, such as HTTP/2:

import httpx

Enabling HTTP/2 Support

client = httpx.Client(http2=True) r = client.get(‘https://www.python.org’)

Setting timeouts and retries is also a piece of cake:

Setting a 10-Second Timeout

r = httpx.get(‘https://www.python.org’, timeout=10.0)

Disabling Timeout

r = httpx.get(‘https://www.python.org’, timeout=None)

Tip: Don’t think that setting a timeout is a hassle; it’s a great way to prevent your program from hanging!

Some Useful Tips

1. Use Session to Maintain a Connection:

with httpx.Client() as client: r1 = client.get(‘https://api.example.com/login’) r2 = client.get(‘https://api.example.com/dashboard’)

2. Custom Request Headers:

headers = {‘User-Agent’: ‘Mozilla/5.0’} r = httpx.get(‘https://www.python.org’, headers=headers)

Encountering SSL certificate verification failures? Try this:

r = httpx.get(‘https://example.com’, verify=False)

Tip: Disabling SSL verification is not a good habit; it’s just for temporary use during debugging.

Asynchronous HTTP requests are like giving your program a rocket booster, especially when you need to handle multiple requests simultaneously. HTTPx not only perfectly inherits the advantages of requests but also comes with an asynchronous aura, making it an essential tool for Python web requests!

No matter how beautifully the code is written, exceptions still need to be handled:

try: r = httpx.get(‘https://www.python.org’) r.raise_for_status() except httpx.RequestError as e: print(f”Request error: {e}”)

Learning these basics is enough!

HTTPx is indeed a great tool, especially since it’s quick to get started and powerful.

Whether you’re writing web scrapers or testing APIs, using it is definitely a good choice!