Requests 小方法

返回页面乱码问题

方案一

import requests
headers = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36'
}
url = 'https://search.jd.com/Search?keyword=%E7%AC%94%E8%AE%B0%E6%9C%AC&enc=utf-8&wq=%E7%AC%94%E8%AE%B0%E6%9C%AC'
res = requests.get(url,headers=headers)

res.encoding = res.apparent_encoding

print(res.text)

方案二

import chardet
import requests
headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36'
}
url = 'https://search.jd.com/Search?keyword=%E7%AC%94%E8%AE%B0%E6%9C%AC&enc=utf-8&wq=%E7%AC%94%E8%AE%B0%E6%9C%AC'
res = requests.get(url,headers=headers)
res.encoding =chardet.detect(res.content)['encoding']
print(res.text)
print(res.encoding)

忽略警告

使用 verify=False 会提示不安全警告

InsecureRequestWarning: Unverified HTTPS request is being made to host 'api.growingio.com'. Adding certificate verification is strongly advised. See:

import warnings
warnings.filterwarnings("ignore")

处理 Cookie

response = request(url="url", headers=headers)
cookies = requests.utils.dict_from_cookiejar(response.cookies)

s = requests.Session()
# 清理Cookie
s.cookies.clear()

# 添加 Cookie
cookie_splicing = {
      "SESSION": "4297f44b13955235245b2497399d7a93",
      "SETTINGS_REGION": "CN",
      "SETTINGS_LANGUAGE": "zh-CN",
  }
s.cookies.set(name="SESSION", value=cookie_splicing.get('SESSION'))
s.cookies.set(name="SETTINGS_REGION", value=cookie_splicing.get('SETTINGS_REGION'))