zoukankan      html  css  js  c++  java
  • 网络编程

    一、协议遇到IO自动切换

    python网络编程,遇到IO自动切换,通过模块gevent来实现;

    import gevent,time

    def g1():
    print ("g1 is start!")
    gevent.sleep(3)
    print ("g1 is end!")
    def g2():
    print ("g2 is start!")
    gevent.sleep(2)
    print ("g2 is end!")
    def g3():
    print ("g3 is start!")
    gevent.sleep(1)
    print ("g3 is end!")

    gevent.joinall([gevent.spawn(g1),
    gevent.spawn(g2),
    gevent.spawn(g3)])
    二、协议gevent并发爬网页
    import gevent,time
    from urllib import request
    from gevent import monkey
    monkey.patch_all()#把当前io的所有操作做上标记(如果未导入该语句,gevent无法确认urllib是io操作);

    def f(url):
    print ("GET %s" % url)
    res =request.urlopen(url)
    data = res.read()
    # f= open("a.html","wb")
    # f.write(data)
    # f.close()
    print ("%d bytes received from %s" %(len(data),url))
    urls = ["https://www.python.org/",
    "https://www.yahoo.com/",
    "https://www.github.com/"]
    time_start = time.time()
    for url in urls:
    f(url)
    total_time = time.time()-time_start
    print (total_time)
    time_start1 = time.time()
    gevent.joinall([
    gevent.spawn(f,"https://www.python.org/"),
    gevent.spawn(f,"https://www.yahoo.com/"),
    gevent.spawn(f,"https://www.github.com/")
    ])
    async_time = time.time() - time_start1
    print (async_time)
    运行结果:

    GET https://www.python.org/
    48990 bytes received from https://www.python.org/
    GET https://www.yahoo.com/
    508828 bytes received from https://www.yahoo.com/
    GET https://www.github.com/
    79979 bytes received from https://www.github.com/
    5.865999937057495
    GET https://www.python.org/
    GET https://www.yahoo.com/
    GET https://www.github.com/
    508974 bytes received from https://www.yahoo.com/
    48990 bytes received from https://www.python.org/
    79979 bytes received from https://www.github.com/
    1.371999979019165

    
    

    Process finished with exit code 0

    从结果中,可以看出,for循环,默认为串行操作;gevent.joinall操作,默认遇到IO操作时,自动切换成并行;
  • 相关阅读:
    安装mongodb,开启mongo服务,创建mongodb用户,登陆mongodb
    【Flask】bootstrap table基础
    【网络安全】加解密算法最详解
    UAC table
    Docker的部署安装(CentOS)-by paymob
    Docker中运行MySQL5.7并挂载宿主机目录到镜像
    搭建harbor企业级私有registry
    Rancher的部署安装(编排选用K8S)
    使用Rancher pipeline搭建基于容器的CICD
    Docker的部署安装(CentOS)
  • 原文地址:https://www.cnblogs.com/wulafuer/p/10281910.html
Copyright © 2011-2022 走看看