multiprocessing -- python进程协同
https://docs.python.org/3.7/library/multiprocessing.html
multiprocessing
is a package that supports spawning processes using an API similar to thethreading
module. Themultiprocessing
package offers both local and remote concurrency, effectively side-stepping the Global Interpreter Lock by using subprocesses instead of threads. Due to this, themultiprocessing
module allows the programmer to fully leverage multiple processors on a given machine. It runs on both Unix and Windows.
可以同一机器和不同机器上,进行进程协同。
相同机器:
from multiprocessing import Process import os def info(title): print(title) print('module name:', __name__) print('parent process:', os.getppid()) print('process id:', os.getpid()) def f(name): info('function f') print('hello', name) if __name__ == '__main__': info('main line') p = Process(target=f, args=('bob',)) p.start() p.join()
不同机器:
from multiprocessing.connection import Listener from array import array address = ('localhost', 6000) # family is deduced to be 'AF_INET' with Listener(address, authkey=b'secret password') as listener: with listener.accept() as conn: print('connection accepted from', listener.last_accepted) conn.send([2.25, None, 'junk', float]) conn.send_bytes(b'hello') conn.send_bytes(array('i', [42, 1729]))
from multiprocessing.connection import Client from array import array address = ('localhost', 6000) with Client(address, authkey=b'secret password') as conn: print(conn.recv()) # => [2.25, None, 'junk', float] print(conn.recv_bytes()) # => 'hello' arr = array('i', [0, 0, 0, 0, 0]) print(conn.recv_bytes_into(arr)) # => 8 print(arr) # => array('i', [42, 1729, 0, 0, 0])
subprocess -- call non python app
https://docs.python.org/3.7/library/subprocess.html
生成外部程序作为子进程,并获取执行其执行结果。
The
subprocess
module allows you to spawn new processes, connect to their input/output/error pipes, and obtain their return codes. This module intends to replace several older modules and functions:The recommended approach to invoking subprocesses is to use the
run()
function for all use cases it can handle. For more advanced use cases, the underlyingPopen
interface can be used directly.
https://pymotw.com/3/subprocess/index.html
import subprocess completed = subprocess.run( ['ls', '-1'], stdout=subprocess.PIPE, ) print('returncode:', completed.returncode) print('Have {} bytes in stdout: {}'.format( len(completed.stdout), completed.stdout.decode('utf-8')) )