标准化爬虫,接口验证
目标网站
不确定请求头,不确定请求体格式,进行接口验证,标准化爬虫
- 找到数据接口(XHR抓包搜索)
- 用接口的路径断点调试,找到请求对象的发送位置
- 补充请求头,更改data…
1 2 3 4 5 6 7 8 9 10 11 12
| import requests
headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36", "Accept": "application/json, text/plain, */*", "Content-Type": "application/json;charset=utf-8", "X-Requested-With": "XMLHttpRequest"} url = "http://www.whggzy.com/portal/category" data = "{\"pageNo\":1,\"pageSize\":15,\"categoryCode\":\"GovernmentProcurement\",\"_t\":1719906086000}"
resp = requests.post(url=url, headers=headers, data=data)
print(resp.json())
|