一 gevent模块
适用于单线程下多个任务的I/O行为,实现遇到I/O就自动切到另外一个任务。
安装
pip3 install gevent
Gevent 是一个第三方库,可以轻松通过gevent实现并发同步或异步编程,在gevent中用到的主要模式是Greenlet, 它是以C扩展模块形式接入Python的轻量级协程。 Greenlet全部运行在主程序操作系统进程的内部,但它们被协作式地调度。
#用法
g1=gevent.spawn(func,1,,2,3,x=4,y=5)创建一个协程对象g1,spawn括号内第一个参数是函数名,如eat,后面可以有多个参数,可以是位置实参或关键字实参,都是传给函数eat的
g2=gevent.spawn(func2)
g1.join() #等待g1结束
g2.join() #等待g2结束
#或者上述两步合作一步:gevent.joinall([g1,g2])
g1.value#拿到func1的返回值
遇到IO阻塞时会自动切换任务
# pip3 install greenlet # 安装 import gevent import time def eat(name): print('%s eat 1' % name) gevent.sleep(3) print('%s eat 2' % name) def play(name): print('%s play 1' % name) gevent.sleep(4) print('%s play 2' % name) start_time = time.time() g1 = gevent.spawn(eat, "alex") # 提交任务eat,传参“alex”(异步提交) g2 = gevent.spawn(play, "jack") g1.join() g2.join() stop_time = time.time() print(stop_time-start_time)
上例gevent.sleep(2)模拟的是gevent可以识别的io阻塞,
而time.sleep(2)或其他的阻塞,gevent是不能直接识别的需要用下面一行代码,打补丁,就可以识别了
from gevent import monkey;monkey.patch_all()必须放到被打补丁者的前面,如time,socket模块之前
或者我们干脆记忆成:要用gevent,需要将from gevent import monkey;monkey.patch_all()放到文件的开头
# pip3 install greenlet # 安装 from gevent import monkey monkey.patch_all() # 下面的代码中涉及I/O操作的,相当于做了标记,能够被gevent识别。 import gevent import time def eat(name): print('%s eat 1' % name) time.sleep(3) print('%s eat 2' % name) def play(name): print('%s play 1' % name) time.sleep(4) print('%s play 2' % name) start_time = time.time() g1 = gevent.spawn(eat, "alex") # 提交任务eat,传参“alex”(异步提交) g2 = gevent.spawn(play, "jack") g1.join() g2.join() stop_time = time.time() print(stop_time-start_time)
我们可以用threading.current_thread().getName()来查看每个g1和g2,查看的结果为DummyThread-n,即假线程
gevent是异步提交任务
# 异步提交任务 # pip3 install greenlet # 安装 from gevent import monkey monkey.patch_all() # 下面的代码中涉及I/O操作的,相当于做了标记,能够被gevent识别。 import gevent import time def eat(name): print('%s eat 1' % name) time.sleep(3) print('%s eat 2' % name) def play(name): print('%s play 1' % name) time.sleep(4) print('%s play 2' % name) g1 = gevent.spawn(eat, "alex") # 提交任务eat,传参“alex”(异步提交) g2 = gevent.spawn(play, "jack") # time.sleep(5) # 实际上现在有3个线程(time.sleep,g1,g2) # g1.join() # g2.join() # 主线程在g1、g2运行完后在死 gevent.joinall([g1, g2])
二 练习
通过gevent实现单线程下的socket并发(from gevent import monkey;monkey.patch_all()一定要放到导入socket模块之前,否则gevent无法识别socket的阻塞)
服务端
from gevent import monkey monkey.patch_all() import gevent import socket def communicate(conn): while True: try: data = conn.recv(1024) if not data: break conn.send(data.upper()) except ConnectionRefusedError: break conn.close() def server(ip, port): server = socket.socket(socket.AF_INET, socket.SOCK_STREAM) server.bind((ip, port)) server.listen(5) while True: conn, addr = server.accept() gevent.spawn(communicate, conn) # 保证服务端不会因为客户端而无限制的开启 if __name__ == "__main__": g = gevent.spawn(server, "127.0.0.1", 8080) g.join()
多线程并发多个客户端
import socket from threading import Thread, currentThread def client(): client = socket.socket(socket.AF_INET, socket.SOCK_STREAM) client.connect(("127.0.0.1", 8080)) while True: # msg = input(">>:").strip() # if not msg: # continue client.send(("%s hello" % currentThread().getName()).encode("utf-8")) data = client.recv(1024) print(data.decode("utf-8")) client.close() if __name__ == "__main__": for i in range(500): t = Thread(target=client) t.start()