zoukankan      html  css  js  c++  java
  • Python下HttpHTTPClient和AsyncHTTPClient

    HTTPClient 使用例子:
        from tornado.httpclient import HTTPClient 

    def synchronous_fetch(url):
         http_client = HTTPClient()
         response = http_client.fetch(url)
         return response.body



    AsyncHTTPClient使用例子:

    方法1:
    from tornado.httpclient import AsyncHTTPClient

    def asynchronous_fetch(url, callback):
      http_client = AsyncHTTPClient()
    def handle_response(response): # 创建一个函数内的函数,来处理返回的结果
    callback(response.body)
    http_client.fetch(url, callback=handle_response)
    # 异步处理结束后会调用指定的callback的函数


    方法2:
    from tornado.httpclient import AsyncHTTPClient
    from tornado.concurrent import Future

    def async_fetch_future(url):
    http_client = AsyncHTTPClient()
    my_future = Future()
    fetch_future = http_client.fetch(url)
    fetch_future.add_done_callback(lambda f: my_future.set_result(f.result()))
    return my_future

    方法3:
    from tornado.httpclient import AsyncHTTPClient
    from tornado import gen

    @gen.coroutine #添加异步访问的装饰器
    def fetch_coroutine(url):
    http_client = AsyncHTTPClient()
    response = yield http_client.fetch(url) # 获取异步结果时要使用yield
    raise gen.Return(response.body) # 使用抛出异常的方式来返回结果,不能使用return来返回



    以下知识是额外的可以了解下,但不保证知识是完整的:
      async and await 在python3.5,tornado4.3中可以了解下

      例子:
    async deffetch_coroutine(url):
       http_client = AsyncHTTPClient()
    response = await http_client.fetch(url)
    return response.body



    使用yield来遍历异步的结果是,以下方法是在项目中没有试验过的
    --------------------------------- start --------------------------------------
    @gen.coroutine
    def parallel_fetch(url1, url2):
        resp1, resp2 = yield [http_client.fetch(url1),
                              http_client.fetch(url2)]
    
    @gen.coroutine
    def parallel_fetch_many(urls):
        responses = yield [http_client.fetch(url) for url in urls]
        # responses is a list of HTTPResponses in the same order
    
    @gen.coroutine
    def parallel_fetch_dict(urls):
        responses = yield {url: http_client.fetch(url)
                            for url in urls}
        # responses is a dict {url: HTTPResponse}

    --------------------------------- end ----------------------------------------


  • 相关阅读:
    JAVA日报
    JAVA日报
    JAVA日报
    论文爬取(四)
    论文爬取(三)
    论文爬取(二)
    剑指 Offer 59
    剑指 Offer 58
    剑指 Offer 58
    剑指 Offer 57
  • 原文地址:https://www.cnblogs.com/wind-wang/p/5649239.html
Copyright © 2011-2022 走看看