poolifier nodejs 线程池工具

nodejs 也有一个线程池的实现worker_threads，但是属于静态配置的，实际很多使用我们需要的是动态的，poolifier 是一个很不错的实现
使用简单，灵活支持固定线程是以及动态线程池，以下是一个简单的学习使用（注意node 版本需要12.x 以及以上版本）

项目准备

• 项目结构

 

├── README.md

├── package.json

├── src

│ ├── index.js

│ └── myworker.js

└── yarn.lock

• 代码说明
  package.json node 依赖以及npm script 定义

 

{

  "name": "node-thread-pool",

  "version": "1.0.0",

  "main": "src/index.js",

  "license": "MIT",

  "dependencies": {

    "poolifier": "^1.0.0"

  },

  "scripts": {

    "start": "node src/index.js",

    "test:standard":"standard",

    "test:standard-fix":"standard --fix"

  },

  "devDependencies": {

    "standard": "^14.3.1"

  }

}

src/index.js node 应用入口

'use strict'

​

// fix (node:2381) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 26 error listeners added. Use emitter.setMaxListeners() to increase limit

require('events').EventEmitter.defaultMaxListeners = 30

​

const { FixedThreadPool, DynamicThreadPool } = require('poolifier')

​

// a fixed thread pool

const pool = new FixedThreadPool(15,

  './src/myWorker.js',

  { errorHandler: (e) => console.error(e), onlineHandler: () => console.log('worker1 is online') })

​

// or a dynamic thread pool

const pool2 = new DynamicThreadPool(10, 100,

  './src/myWorker.js',

  { errorHandler: (e) => console.error(e), onlineHandler: () => console.log('worker2 is online') })

​

pool2.emitter.on('FullPool', () => console.log('Pool is full'))

​

// the execute method signature is the same for both implementations,

// so you can easy switch from one to another

pool.execute({ username: 'dalong', userage: 333 }).then(res => {

  console.log(res)

}).catch(err => {

  console.log(err)

})

pool2.execute({ username: 'dalong2', userage: 555 }).then(res => {

  console.log(res)

}).catch(err => {

  console.log(err)

})

src/myworker.js worker 代码定义

'use strict'

const { ThreadWorker } = require('poolifier')

function myFunction (data) {

  return { ok: 1, data }

}

module.exports = new ThreadWorker(myFunction, { maxInactiveTime: 60000 })

启动&& 效果

• 启动

yarn start

• 效果

yarn start 

yarn run v1.21.1

$ node src/index.js

worker1 is online

worker1 is online

worker1 is online

worker1 is online

worker1 is online

worker1 is online

worker1 is online

worker1 is online

worker1 is online

worker1 is online

worker1 is online

worker1 is online

worker1 is online

worker1 is online

worker1 is online

worker2 is online

worker2 is online

worker2 is online

worker2 is online

worker2 is online

worker2 is online

worker2 is online

worker2 is online

worker2 is online

worker2 is online

{ ok: 1, data: { username: 'dalong', userage: 333 } }

{ ok: 1, data: { username: 'dalong2', userage: 555 } }

 

 

说明

poolifier 简化了基于nodejs worker_threads 开发多线程应用的成本，使用简单，灵活

参考资料

https://nodejs.org/api/worker_threads.html#worker_threads_worker_threads
https://github.com/pioardi/poolifier
https://github.com/rongfengliang/node-thread-pool-learning