concurrent.futures 学习笔记

concurrent.futures

先看下官方介绍

The asynchronous execution can be performed with threads, using ThreadPoolExecutor, or separate processes, using ProcessPoolExecutor. Both implement the same interface, which is defined by the abstract Executor class.

重点是 asynchronous

concurrent.futures._base.Executor

submit(fn, *args, **kwargs) -> Future

with ThreadPoolExecutor(max_workers=1) as executor:
    future = executor.submit(pow, 323, 1235)
    print(future.result())

map(func, *iterables, timeout=None, chunksize=1) -> typing.Generator

类似于 map(内置函数)

In [26]: def test(item):
    ...:     print("echo", item)
    ...:     return item
    ...:
In [27]: list(map(test, range(10)))
echo 0
echo 1
echo 2
echo 3
echo 4
echo 5
echo 6
echo 7
echo 8
echo 9
Out[27]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

• demo

def test(item):
    print(f"echo {item}")
    return item
with ThreadPoolExecutor(max_workers=1) as executor:
    future = executor.map(test, range(3))

shutdown

# todo

ThreadPoolExecutor

• 注意避免在一个 Future 里面获取另外一个 Future 的 result

ProcessPoolExecutor

在 IO 密集型下使用 ThreadPoolExecutor

Future

• cancel()
  • 无法取消当前正在执行的
• result() -> bool
• running() -> bool
• done() -> bool
• result(timeout=None) -> Any
• exception(timeout=None) -> Exception

exector = ThreadPoolExecutor(max_workers=10)
f: Future = exector.submit(test, 1)
exception: Exception = f.exception(timeout=None)

• add_done_callback(fn) -> None

def callback(*args):
    print(args) # (<Future at 0x102bcb898 state=finished raised KeyError>,) 即 f


exector = ThreadPoolExecutor(max_workers=10)
f: Future = exector.submit(test, 1)
f.add_done_callback(callback)