zoukankan      html  css  js  c++  java
  • 更新换代之requests库

          好久不更新博客了。。。

    之前的博文都是通过urllib2进行http访问,接下来我要说一个利器啊!requests模块,无法用语言对他进行赞扬了,需要的,有兴趣的,可以去了解下,移步官方中文文档:

    Requests: 让 HTTP 服务人类

    简直是不要太刁。。。

          这篇博文呢,主要是将之前博文中用urllib2写的HttpClient类换成request。代码如下:

    # coding=utf-8
    from __future__ import unicode_literals
    import requests
    from io import StringIO
    
    
    class HttpClient:
        def __init__(self):
            pass
        __headers = {
            'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0',
            'Accept':'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
            # 'Host':'www.xiami.com'
            }
        __proxies = {
              # "http": "http://10.10.1.10:3128",
              # "https": "http://10.10.1.10:1080",
            }
    
        def get(self, url, params=None, retries=3):
            try:
                req = requests.get(url, headers=self.__headers, timeout=30, params=params,
                                   proxies=self.__proxies)
                req.raise_for_status()
                return req.text
            except Exception,e:
                print e
                if retries > 0:
                    return self.get(url, params, retries - 1)
                else:
                    print "Get Failed", url
                    return ''
    
        def post(self, url, data=None, retires=3):
            try:
                req = requests.post(url, headers=self.__headers, timeout=30, data=data,
                                    proxies=self.__proxies)
                req.raise_for_status()
                return req.text
            except Exception,e:
                print e
                if retires > 0:
                    return self.post(url,data,retires - 1)
                else:
                    print "Post Failed", url
                    return ''
    
        def download(self, url, file_name, params=None, cookies=None):
            try:
                req = requests.get(url, headers=self.__headers, params=params,
                                   proxies=self.__proxies)
                output = open(file_name, 'wb')
                output.write(req.content)
                output.close()
            except Exception,e:
                print 'error',e
    
        def get_cookies(self, url, key, params=None):
            try:
                req = requests.get(url, headers=self.__headers, timeout=30, params=params,
                                   proxies=self.__proxies)
                req.raise_for_status()
                return req.cookies.get(key,'')
            except Exception,e:
                return ''
    
        def get_headers(self, url, key, params=None):
            try:
                req = requests.get(url, headers=self.__headers, timeout=30, params=params,
                                   proxies=self.__proxies)
                req.raise_for_status()
                return req.headers.get(key)
            except Exception,e:
                return ''
    

      记录一下,后面会时常更新博文的。

  • 相关阅读:
    about springmvc intergation jquery with ajax &noajax version
    spring mvc book very good
    use spring config editor to configure xsi:schemaLocation
    install firefox9 on ubuntu lucid
    TCMalloc : ThreadCaching Malloc
    ubuntu amd64bit lts lucid lynx image download address
    一个词评价python 和rails
    virtualbox ubuntu下使用设置
    how to install chromium on ubuntu lucid
    Extjs4 MVC 示例
  • 原文地址:https://www.cnblogs.com/jixin/p/5726733.html
Copyright © 2011-2022 走看看