zoukankan      html  css  js  c++  java
  • python使用gevent实现并发下载器

    并发下载原理

    import gevent
    from gevent import monkey
    import urllib.request
    
    
    monkey.patch_all()
    
    
    def my_download(url):
            print('GET: %s' % url)
            resp = urllib.request.urlopen(url)
            data = resp.read()
            print('%d bytes received from %s.' % (len(data), url))
        
    
    gevent.joinall([
            gevent.spawn(my_download, "https://www.baidu.com"),
            gevent.spawn(my_download, "https://www.cnblogs.com"),
            gevent.spawn(my_download, "https://www.huya.com")
        
    ])
    

    运行结果:

    GET: https://www.baidu.com
    GET: https://www.cnblogs.com
    GET: https://www.huya.com
    227 bytes received from https://www.baidu.com.
    46411 bytes received from https://www.cnblogs.com.
    353563 bytes received from https://www.huya.com.
    

    实现多张图片同时下载

    import gevent
    from gevent import monkey
    import urllib.request
    
    
    monkey.patch_all()
    
    
    def my_download(url, image_path):
    	print('GET: %s' % url)
    	resp = urllib.request.urlopen(url)
    	data = resp.read()
    	print('%d bytes received from %s.' % (len(data), url))
    	
    	with open(image_path, "wb") as f:
    		f.write(data)
    	
    
    gevent.joinall([
    	gevent.spawn(my_download, "https://huyaimg.msstatic.com/cdnimage/anchorpost/1099/4c/73ff3e6ce165fb658b2082d4d126c6_2168_1544414129.jpg", "1.jpg"),
    	gevent.spawn(my_download, "https://huyaimg.msstatic.com/cdnimage/anchorpost/1020/03/7949816c55f45b9adc9f03d9330af3_2168_1542941499.jpg", "2.jpg"),	
    	gevent.spawn(my_download, "https://huyaimg.msstatic.com/cdnimage/anchorpost/1061/55/8071537fa10120c43c5a1359a88530_2168_1545302709.jpg", "3.jpg"),
    ])
    
  • 相关阅读:
    [cdq分治][树状数组] Bzoj P3262 陌上花开
    [Prufer序列] Bzoj P4766 文艺计算姬
    [欧拉回路][并查集] Bzoj P3706 反色刷
    [欧拉回路][dfs] Uoj #117 欧拉回路
    [并查集][Tarjan] Bzoj P5017 炸弹
    day18
    day17
    树形DP学习笔记
    [分治]JZOJ 6308 中间值
    [贪心][完全背包]JZOJ 6309 完全背包
  • 原文地址:https://www.cnblogs.com/zhangfengxian/p/10163655.html
Copyright © 2011-2022 走看看