zoukankan      html  css  js  c++  java
  • 进程池与线程池例子(异步调用与回调)

    多线程下载网站数据

    from concurrent.futures import ThreadPoolExecutor
    import time
    import requests
    def get(url):
        print("get %s" % url)
        response=requests.get(url)
        time.sleep(3)
        return {"url":url,"content":response.text}
    def parse(res):
        res=res.result()
        print("%s parse res is %s" % (res['url'],len(res['content'])))
    
    if __name__=="__main__":
        urls=[
            "http://fanyi.youdao.com/",
            "http://news.baidu.com/",
            "http://news.cctv.com/2018/05/16/ARTIjk8z8DQrpR7ciSzijCgl180516.shtml",
            "http://news.cctv.com/2018/05/16/ARTI43pH6clUqOHJfYaRgHri180516.shtml",
            "http://news.china.com/domestic/945/20180516/32417253.html?newsbaidu",
            "http://money.china.com/"
        ]
        pool=ThreadPoolExecutor(2)
        for url in urls:
            pool.submit(get,url).add_done_callback(parse)

    get http://fanyi.youdao.com/
    get http://news.baidu.com/
    http://fanyi.youdao.com/ parse res is 18720
    get http://news.cctv.com/2018/05/16/ARTIjk8z8DQrpR7ciSzijCgl180516.shtml
    http://news.baidu.com/ parse res is 74087
    get http://news.cctv.com/2018/05/16/ARTI43pH6clUqOHJfYaRgHri180516.shtml
    http://news.cctv.com/2018/05/16/ARTIjk8z8DQrpR7ciSzijCgl180516.shtml parse res is 88508
    get http://news.china.com/domestic/945/20180516/32417253.html?newsbaidu
    http://news.cctv.com/2018/05/16/ARTI43pH6clUqOHJfYaRgHri180516.shtml parse res is 32127
    get http://money.china.com/
    http://news.china.com/domestic/945/20180516/32417253.html?newsbaidu parse res is 65824
    http://money.china.com/ parse res is 63167

    Process finished with exit code 0

    线程池控制服务端线程数量

    from socket import *
    from concurrent.futures import ThreadPoolExecutor
    def communicate(conn):
        while True:
            try:
                data = conn.recv(1024)
                if not data: break
                conn.send(data.upper())
            except ConnectionResetError:
                break
        conn.close()
    def servers(ip,port):
        server = socket(AF_INET, SOCK_STREAM)
        server.bind((ip, port))
        server.listen(5)
        while True:
            conn,addr=server.accept()
            print(addr)
            pool.submit(communicate,conn)
        server.close()
    if __name__=="__main__":
        pool=ThreadPoolExecutor(2)
        servers("127.0.0.1",8081)
  • 相关阅读:
    未来十年Python的前景会怎样?
    mysql配置mha高可用防火墙未关闭报错
    Cenos7 切换单用户模式
    奇葩问题: lsattr -d /data 显示:----------I--e- /data/
    在Vue 的main.js 文件使用Element-UI的this.$message('msg')
    IIS上部署项目,网页控制台报错,.svg 等文件类型找不到
    git commit 格式
    Entity Framework 插入带有外键的数据,重新添加了外建表里的数据
    element-ui Select 组件传递对象
    Entity Framework Code First实体关联数据加载
  • 原文地址:https://www.cnblogs.com/yaya625202/p/9048497.html
Copyright © 2011-2022 走看看