python 爬虫 使用requests设置代理

免费代理的网站: http://www.xicidaili.com/nn/

代码部分：

import requests

proxy='124.243.226.18:8888'

#如果代理需要验证，只需要在前面加上用户名密码，如下所示

# proxy='username:password@124.243.226.18:8888'
proxies={
    'http':'http://'+proxy,
    'https':'https://'+proxy,
}
try:
    response=requests.get('http://httpbin.org/get',proxies=proxies)
    print(response.text)
except requests.exceptions.ConnectionError as e:
    print("Error",e.args)
 

输出：

{
  "args": {},
  "headers": {
    "Accept": "*/*",
    "Accept-Encoding": "gzip, deflate",
    "Connection": "close",
    "Host": "httpbin.org",
    "User-Agent": "python-requests/2.20.0"
  },
  "origin": "124.243.226.18",
  "url": "http://httpbin.org/get"
}

基于 selenium的代理设置：

from selenium import webdriver

proxy='124.243.226.18:8888'

option=webdriver.ChromeOptions()

option.add_argument('--proxy-server=http://'+proxy)

driver = webdriver.Chrome(options=option)

driver.get('http://httpbin.org/get')