Python学习-day10

 一、多进程multiprocessing

       multiprocessing包是Python中的多进程管理包。与threading.Thread类似，它可以利用multiprocessing.Process对象来创建一个进程。该Process对象与Thread对象的用法相同，也有start(), run(), join()的方法。

import multiprocessing,threading
import time

def thread_run():
    print(threading.get_ident())
def run(name):
    time.sleep(2)
    print('Hello ',name)
    t = threading.Thread(target=thread_run)
    t.start()


if __name__== '__main__':
    for i in range(10):
        p = multiprocessing.Process(target=run, args=('bob_%s'%i,))
        p.start()

运行结果：

Hello  bob_0
1144
Hello  bob_8
1268
Hello  bob_4
4360
Hello  bob_2
768
Hello  bob_6
5308
Hello  bob_5
Hello  bob_1
6076
5088
Hello  bob_9
6104
Hello  bob_3
5196
Hello  bob_7
748

二、进程池

      如果要创建多个进程，可以使用进程池，启用进程池需要使用Pool库，使用指令pool=Pool()可自动调用所有CPU，

map()函数相当于一个循环，将参数2中的列表元素逐次灌入参数1的函数中。

from multiprocessing import Pool

def squre(num):
    return num ** 2


if __name__ == '__main__':
    numbers = [0,1,2,3,4,5]
    pool = Pool(processes=5)#进程池中最多能放入5个进程
    print(pool.map(squre,numbers))

 运行结果：

[0, 1, 4, 9, 16, 25]

 Pool还有以下常用的方法:

• apply_async(func, args)从进程池中取出一个进程执行func，args为func的参数。它将返回一个AsyncResult的对象，我们可以调用get()方法以获得结果。

• close() 关闭进程池，不再创建新的进程

• join() wait()进程池中的全部进程。但是必须对Pool先调用close()方法才能join.

from  multiprocessing import Process, Pool,freeze_support
import time
import os

def Foo(i):
    time.sleep(2)
    print("in process",os.getpid())
    return i + 100

def Bar(arg):
    print('-->exec done:', arg,os.getpid())

if __name__ == '__main__':
    #freeze_support()
    pool = Pool(processes=3)
    print("主进程",os.getpid())
    for i in range(10):
        pool.apply_async(func=Foo, args=(i,), callback=Bar) #callback=回调
        
    print('end')
    pool.close()
    pool.join() 

 运行结果：

主进程 5660
end
in process 7048
-->exec done: 100 5660
in process 3396
-->exec done: 101 5660
in process 6728
-->exec done: 102 5660
in process 7048
-->exec done: 103 5660
in process 3396
-->exec done: 104 5660
in process 6728
-->exec done: 105 5660
in process 7048
-->exec done: 106 5660
in process 3396
-->exec done: 107 5660
in process 6728
-->exec done: 108 5660
in process 7048
-->exec done: 109 5660

除了主进程，其它结果是三个一组执行的，因为进程池中每次最多有三个进程。

三、进程通信

      进程间通信常用两种方法：Queue和pipe,Queue可以用在多个进程间实现通信，pipe用在两个进程间通信。

import os
import multiprocessing
import time
#==================
# input worker
def inputQ(queue):
    info = str(os.getpid()) + '(put):' + str(time.time())
    queue.put(info)

# output worker
def outputQ(queue,lock):
    info = queue.get()
    lock.acquire()
    print (str(os.getpid()) + '(get):' + info)
    lock.release()

if __name__ == '__main__':
    record1 = []  # store input processes
    record2 = []  # store output processes
    lock = multiprocessing.Lock()  # To prevent messy print
    queue = multiprocessing.Queue(3)

    
    for i in range(10):
        process = multiprocessing.Process(target=inputQ, args=(queue,))
        process.start()
        record1.append(process)

   
    for i in range(10):
        process = multiprocessing.Process(target=outputQ, args=(queue, lock))
        process.start()
        record2.append(process)

    for p in record1:
        p.join()

    queue.close()  

    for p in record2:
        p.join()

运行结果：

4004(get):4556(put):1476337412.4875286
512(get):5088(put):1476337412.6345284
8828(get):7828(put):1476337412.7965286
8372(get):1032(put):1476337412.8185284
7740(get):1496(put):1476337412.9205284
4176(get):632(put):1476337412.9855285
5828(get):8508(put):1476337412.9595284
4236(get):9204(put):1476337412.9925284
7632(get):8956(put):1476337413.2055285
6376(get):4160(put):1476337413.0705285

      Pipe可以是单向(half-duplex)，也可以是双向(duplex)。我们通过mutiprocessing.Pipe(duplex=False)创建单向管道 (默认为双向)。一个进程从PIPE一端输入对象，然后被PIPE另一端的进程接收，单向管道只允许管道一端的进程输入，而双向管道则允许从两端输入。

from multiprocessing import Process, Pipe


def f(conn):
    conn.send([42, None, 'hello from child'])
    conn.send([42, None, 'hello from child2'])
    print("from parent:",conn.recv())
    conn.close()

if __name__ == '__main__':
    parent_conn, child_conn = Pipe()
    p = Process(target=f, args=(child_conn,))
    p.start()
    print(parent_conn.recv())  # prints "[42, None, 'hello']"
    print(parent_conn.recv())  # prints "[42, None, 'hello']"
    parent_conn.send("chupi可好") # prints "[42, None, 'hello']"
    p.join()

运行结果：

[42, None, 'hello from child']
[42, None, 'hello from child2']
from parent: chupi可好