zoukankan      html  css  js  c++  java
  • 携程,

    》》》》》携程greenlet模块

     1 from greenlet import greenlet
     2 def test1():
     3     print(15)
     4     gr1.switch()
     5     print(16)
     6     gr1.switch()
     7 def test2():
     8     print(25)
     9     gr2.switch()
    10     print(26)
    11     gr2.switch()
    12 gr2 = greenlet(test1) #启动一个携程
    13 gr1 = greenlet(test2)
    14 # gr2.switch()
    15 gr2.switch()
    View Code

     1,注意开关的顺序所造成的不同结果

    》》》》》yield

     1 import time
     2 def consumer(name):#消费者
     3     print('start eat baozi')
     4     while True:
     5         new_baozi = yield#阻塞有包子就吃没有就等待
     6         print("[%s] is eatting %s baozi"%(name,new_baozi))
     7         
     8 def producer():#生产者
     9     r = con1.__next__()
    10     r = con2.__next__()
    11     n = 0
    12     while n < 5:
    13         n += 1
    14         con1.send(n)
    15         con2.send(n)
    16         time.sleep(1)
    17         print("生产%s包子 "%n)
    18         
    19 if __name__ =='__main__':
    20     con1 = consumer("c1")
    21     con2 = consumer("c2")
    22     p = producer()
    View Code

    1,生产者消费者模型,yield阻塞等待

    》》》》》爬虫

     1 from gevent import monkey;
     2 import gevent
     3 from urllib.request import urlopen
     4 import time
     5 
     6 def f(url):
     7     print("get : %s" % url)
     8     resp = urlopen(url)
     9     data = resp.read()
    10     print("%s bytes recv from %s"%(len(data) , url))
    11 
    12 time_start = time.time()
    13 gevent.joinall([
    14     gevent.spawn(f,'https://www.python.org/'),
    15     gevent.spawn(f, 'https://www.yahoo.com/'),
    16     gevent.spawn(f, 'https://github.com/'),
    17 ])
    18 
    19 print("同步cost",time.time() - time_start)
    同步执行的爬虫
     1 __author__ = "Alex Li"
     2 
     3 from gevent import monkey;
     4 
     5 # monkey.patch_all()
     6 import gevent
     7 from  urllib.request import urlopen
     8 import time
     9 
    10 def f(url):
    11     print('GET: %s' % url)
    12     resp = urlopen(url)
    13     data = resp.read()
    14     print('%d bytes received from %s.' % (len(data), url))
    15 
    16 urls = [ 'https://www.python.org/',
    17          'https://www.yahoo.com/',
    18          'https://github.com/'
    19          ]
    20 
    21 
    22 async_time_start = time.time()
    23 gevent.joinall([
    24     gevent.spawn(f, 'https://www.python.org/'),
    25     gevent.spawn(f, 'https://www.yahoo.com/'),
    26     gevent.spawn(f, 'https://github.com/'),
    27 ])
    28 print("异步cost",time.time()-async_time_start )
    async爬虫
  • 相关阅读:
    Flsak中的socket是基于werkzeug实现的。
    Flask中的CBV
    beego——静态文件
    beego——模板函数
    beego——模板处理
    beego——模板语法
    beego——模型定义
    beego——事务处理和命令模式
    beego——构造查询
    beego——原生SQL查询
  • 原文地址:https://www.cnblogs.com/cerofang/p/8025513.html
Copyright © 2011-2022 走看看