zoukankan      html  css  js  c++  java
  • python之https爬虫出现 SSL: CERTIFICATE_VERIFY_FAILED (同时打开fiddler就会出现)

    1.参考

    Py 坑之 CERTIFICATE_VERIFY_FAILED

    Python 升级到 2.7.9 之后引入了一个新特性,当你urllib.urlopen一个 https 的时候,会验证一次 SSL 证书,当目标网站使用的是自签名的证书时就会爆出一个 urllib2.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)> 的错误消息

    Python Requests throwing up SSLError

    The problem you are having is caused by an untrusted SSL certificate.

    Like @dirk mentioned in a previous comment, the quickest fix is setting verify=False.

    Please note that this will cause the certificate not to be verified. This will expose your application to security risks, such as man-in-the-middle attacks.

    Of course, apply judgment. As mentioned in the comments, this may be acceptable for quick/throwaway applications/scripts, but really should not go to production software.

    If just skipping the certificate check is not acceptable in your particular context, consider the following options, your best option is to set the verifyparameter to a string that is the path of the .pem file of the certificate (which you should obtain by some sort of secure means).

    So, as of version 2.0, the verify parameter accepts the following values, with their respective semantics:

    • True: causes the certificate to validated against the library's own trusted certificate authorities (Note: you can see which Root Certificates Requests uses via the Certifi library, a trust database of RCs extracted from Requests: Certifi - Trust Database for Humans).
    • False: bypasses certificate validation completely.
    • Path to a CA_BUNDLE file for Requests to use to validate the certificates.

    Source: Requests - SSL Cert Verification

    Also take a look at the cert parameter on the same link.

     

     

     

    2.解决办法

    (1)全局设置,对requests不生效 

     ssl._create_default_https_context = ssl._create_unverified_context

     

    (2)urllib2.urlopen 传参

     
    context = ssl._create_unverified_context()
    
    urllib2.urlopen(context=context)

     

    (3)参考 urllib2.urlopen 为opener叠加handler

    复制代码

    # C:Program FilesAnaconda2Liburllib2.py
    # def urlopen
    # elif context:
    # https_handler = HTTPSHandler(context=context)
    # opener = build_opener(https_handler)

    # urllib2.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)>
    try:
    opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cookiejar))
    resp = opener.open(req)
    except urllib2.URLError as err:
    print err
    opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cookiejar), urllib2.HTTPSHandler(context=context)) #叠加多个 handler
    resp = opener.open(req)

    复制代码

     

    (4)request.get 传参

     
    # requests.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)
    try:
        r = requests.get(url, headers=headers, cookies=cookie)
    except requests.exceptions.SSLError as err:
        print err
        r = requests.get(url, headers=headers, cookies=cookie, verify=False)
  • 相关阅读:
    HDU 1004 Let the Balloon Rise【STL<map>】
    UVA 1030
    UVA 10881
    POJ 3154 Graveyard【多解,数论,贪心】
    浅谈Notepad++选中行操作+快捷键+使用技巧【超详解】
    COGS 68. [NOIP2005] 采药【01背包复习】
    [phomeflashpic]怎样调用帝国CMS图片幻灯效果
    微信认证新增公对公账户银行卡转账支付审核费用 缩减认证审核时长
    微信公众平台回复过了怎么不能再次回复?亲们要注意查看"公众平台回复用户消息时限变更通知"的公告啊
    新版微信终于支持消息撤回了 微信零钱也能转账了[微信5.3.1.16更新]
  • 原文地址:https://www.cnblogs.com/piwefei/p/11188071.html
Copyright © 2011-2022 走看看