python中httpx庫的詳細使用方法及案例詳解

更新時間：2025年02月27日 09:38:54 作者：數據知道

httpx 是一個現(xiàn)代化的 Python HTTP 客戶端庫,支持同步和異步請求,功能強大且易于使用,它比 requests 更高效,支持 HTTP/2 和異步操作,以下是 httpx 的詳細使用方法,感興趣的小伙伴跟著小編一起來看看吧

1. 安裝 httpx

首先，確保已經安裝了 httpx?？梢酝ㄟ^以下命令安裝：pip install httpx

如果需要支持 HTTP/2，可以安裝額外依賴：pip install httpx[http2]

2. 同步請求

發(fā)送 GET 請求

import httpx

# 發(fā)送 GET 請求
response = httpx.get('https://httpbin.org/get')
print(response.status_code)  # 狀態(tài)碼
print(response.text)         # 響應內容

發(fā)送 POST 請求

# 發(fā)送 POST 請求
data = {'key': 'value'}
response = httpx.post('https://httpbin.org/post', json=data)
print(response.json())  # 解析 JSON 響應

設置請求頭

headers = {'User-Agent': 'my-app/1.0.0'}
response = httpx.get('https://httpbin.org/headers', headers=headers)
print(response.json())

設置查詢參數

params = {'key1': 'value1', 'key2': 'value2'}
response = httpx.get('https://httpbin.org/get', params=params)
print(response.json())

處理超時

try:
    response = httpx.get('https://httpbin.org/delay/5', timeout=2.0)
except httpx.TimeoutException:
    print("請求超時")

3. 異步請求

httpx 支持異步操作，適合高性能場景。

發(fā)送異步 GET 請求

import httpx
import asyncio

async def fetch(url):
    async with httpx.AsyncClient() as client:
        response = await client.get(url)
        print(response.text)

asyncio.run(fetch('https://httpbin.org/get'))

發(fā)送異步 POST 請求

async def post_data(url, data):
    async with httpx.AsyncClient() as client:
        response = await client.post(url, json=data)
        print(response.json())

asyncio.run(post_data('https://httpbin.org/post', {'key': 'value'}))

并發(fā)請求

async def fetch_multiple(urls):
    async with httpx.AsyncClient() as client:
        tasks = [client.get(url) for url in urls]
        responses = await asyncio.gather(*tasks)
        for response in responses:
            print(response.text)

urls = ['https://httpbin.org/get', 'https://httpbin.org/ip']
asyncio.run(fetch_multiple(urls))

4. 高級功能

使用 HTTP/2

# 啟用 HTTP/2
client = httpx.Client(http2=True)
response = client.get('https://httpbin.org/get')
print(response.http_version)  # 輸出協(xié)議版本

文件上傳

files = {'file': open('example.txt', 'rb')}
response = httpx.post('https://httpbin.org/post', files=files)
print(response.json())

流式請求

# 流式上傳
def generate_data():
    yield b"part1"
    yield b"part2"

response = httpx.post('https://httpbin.org/post', data=generate_data())
print(response.json())

流式響應

# 流式下載
with httpx.stream('GET', 'https://httpbin.org/stream/10') as response:
    for chunk in response.iter_bytes():
        print(chunk)

5. 錯誤處理

httpx 提供了多種異常類，方便處理錯誤。

處理網絡錯誤

try:
    response = httpx.get('https://nonexistent-domain.com')
except httpx.NetworkError:
    print("網絡錯誤")

處理 HTTP 錯誤狀態(tài)碼

response = httpx.get('https://httpbin.org/status/404')
if response.status_code == 404:
    print("頁面未找到")

6. 配置客戶端

可以通過 httpx.Client 或 httpx.AsyncClient 配置全局設置。

設置超時

client = httpx.Client(timeout=10.0)
response = client.get('https://httpbin.org/get')
print(response.text)

設置代理

proxies = {
    "http://": "http://proxy.example.com:8080",
    "https://": "http://proxy.example.com:8080",
}
client = httpx.Client(proxies=proxies)
response = client.get('https://httpbin.org/get')
print(response.text)

設置基礎 URL

client = httpx.Client(base_url='https://httpbin.org')
response = client.get('/get')
print(response.text)

7. 結合 Beautiful Soup 使用

httpx 可以與 Beautiful Soup 結合使用，抓取并解析網頁。

import httpx
from bs4 import BeautifulSoup

# 抓取網頁
response = httpx.get('https://example.com')
html = response.text

# 解析網頁
soup = BeautifulSoup(html, 'lxml')
title = soup.find('title').text
print("網頁標題:", title)

8. 示例：抓取并解析網頁

以下是一個完整的示例，展示如何使用 httpx 抓取并解析網頁數據：

import httpx
from bs4 import BeautifulSoup

# 抓取網頁
url = 'https://example.com'
response = httpx.get(url)
html = response.text

# 解析網頁
soup = BeautifulSoup(html, 'lxml')

# 提取標題
title = soup.find('title').text
print("網頁標題:", title)

# 提取所有鏈接
links = soup.find_all('a', href=True)
for link in links:
    href = link['href']
    text = link.text
    print(f"鏈接文本: {text}, 鏈接地址: {href}")