Requests库基本使用

Requests基于urllib3比urllib更简单方便。

基本的GET请求

1 2 3 4

import requests   response = requests.get('http://httpbin.org/get') print(response.text)

 

带参数的GET请求

方式1

1 2 3 4

import requests   response = requests.get('http://httpbin.org/get?name=0bug&age=25') print(response.text)

 

方式二

1 2 3 4 5 6 7 8

import requests   data = {     'name': '0bug',     'age': 25 } response = requests.get('http://httpbin.org/get', params=data) print(response.text)

 

解析Json

1 2 3 4 5 6

import requests   response = requests.get('http://httpbin.org/get') print(type(response.text)) print(response.json()) print(type(response.json()))

 

获取二进制数据

1 2 3 4 5 6 7

import requests   response = requests.get('https://github.com/favicon.ico') print(type(response.text)) print(type(response.content)) print(response.text) print(response.content)

 

写入图片

1 2 3 4 5

import requests   response = requests.get('https://github.com/favicon.ico') with open('img.ico','wb') as f:     f.write(response.content)

添加headers

1 2 3 4 5 6 7 8

import requests   headers = {     'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.117 Safari/537.36'   } response = requests.get('https://www.baidu.com/', headers=headers) print(response.status_code)

基本的POST请求

1 2 3 4 5

import requests   data = {'name':'0bug'} response = requests.post('http://httpbin.org/post',data=data) print(response.text)

 

1 2 3 4 5 6 7 8 9

import requests   data = {'name': '0bug'} headers = {     'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.117 Safari/537.36'   } response = requests.post('http://httpbin.org/post', data=data, headers=headers) print(response.json())

 

response属性

1 2 3 4 5 6 7 8

import requests   response = requests.get('http://www.jianshu.com') print(type(response.status_code), response.status_code) print(type(response.headers), response.headers) print(type(response.cookies), response.cookies) print(type(response.url), response.url) print(type(response.history), response.history)

 

文件上传

1 2 3 4 5

import requests   files = {'file': open('img.ico', 'rb')} response = requests.post('http://httpbin.org/post', files=files) print(response.text)

 

获取cookie

1 2 3 4 5 6

import requests   response = requests.get('https://www.baidu.com') print(response.cookies) for key,value in response.cookies.items():     print(key+'='+value)

 

会话维持

1 2 3 4 5 6

import requests   s = requests.Session() s.get('http://httpbin.org/cookies/set/number/123456') response = s.get('http://httpbin.org/cookies') print(response.text)

 

证书验证

1.无证书报错

1 2 3 4

import requests   response = requests.get('https://www.12306.cn') print(response.status_code)

2.设置不使用证书，会返回200，也会有警告信息

1 2 3 4

import requests   response = requests.get('https://www.12306.cn',verify=False) print(response.status_code)

3.消除警告信息

1 2 3 4 5

import requests import urllib3 urllib3.disable_warnings() response = requests.get('https://www.12306.cn',verify=False) print(response.status_code)

4.使用本地证书

1 2 3 4

import requests   response = requests.get('https://www.12306.cn',cert=('/path/server.crt','/path/key')) print(response.status_code)

代理设置

1 2 3 4 5 6 7 8 9

import requests   proxies = {     'http': 'http://127.0.0.1:9743',     'https': 'https://127.0.0.1:9743' }   response = requests.get('https://www.taobao.com', proxies=proxies) print(response.status_code)

有用户名和密码的代理

1 2 3 4 5 6 7 8

import requests   proxies = {     'http': 'http://user:password@127.0.0.1:9743', }   response = requests.get('https://www.taobao.com', proxies=proxies) print(response.status_code)

使用socks代理，需要安装一个模块

1	pip install requests[socks]

再使用代理

1 2 3 4 5 6 7 8 9

import requests   proxies = {     'http': 'socks5://127.0.0.1:9742',     'https': 'socks5://127.0.0.1:9742', }   response = requests.get('https://www.taobao.com', proxies=proxies) print(response.status_code)

超时设置

1 2 3 4

import requests   response = requests.get('https://www.taobao.com',timeout=0.01) print(response.status_code)

错误处理

1 2 3 4 5 6 7 8

import requests from requests.exceptions import ReadTimeout   try:     response = requests.get('https://www.taobao.com', timeout=0.01)     print(response.status_code) except ReadTimeout:     print('time out')

认证设置

方式1

1 2 3 4 5

import requests from requests.auth import HTTPBasicAuth   r = requests.get('http://127.0.0.1:8080', auth=HTTPBasicAuth('user', '123')) print(r.status_code)

方式2

1 2 3 4

import requests   r = requests.get('http://127.0.0.1:8080', auth=('user', '123')) print(r.status_code)

异常处理

1 2 3 4 5 6 7 8 9 10 11 12

import requests from requests.exceptions import ReadTimeout, HTTPError, RequestException   try:     response = requests.get('http://httpbin.org/get', timeout=0.01)     print(response.status_code) except ReadTimeout:     print('Timeout') except HTTPError:     print('HTTP REEOR') except RequestException:     print('Error')