cube.js 调度&&查询队参考参数

参考参数

packages/cubejs-query-orchestrator/src/orchestrator/QueryQueue.js

• 构造函数

 constructor(redisQueuePrefix, options) {

    this.redisQueuePrefix = redisQueuePrefix;

    this.concurrency = options.concurrency || 2;

    this.continueWaitTimeout = options.continueWaitTimeout || 5;

    this.executionTimeout = options.executionTimeout || 600;

    this.orphanedTimeout = options.orphanedTimeout || 120;

    this.heartBeatInterval = options.heartBeatInterval || 30;

    this.sendProcessMessageFn = options.sendProcessMessageFn || ((queryKey) => { this.processQuery(queryKey); });

    this.sendCancelMessageFn = options.sendCancelMessageFn || ((query) => { this.processCancel(query); });

    this.queryHandlers = options.queryHandlers;

    this.cancelHandlers = options.cancelHandlers;

    this.logger = options.logger || ((message, event) => console.log(`${message} ${JSON.stringify(event)}`));

    const queueDriverOptions = {

      redisQueuePrefix: this.redisQueuePrefix,

      concurrency: this.concurrency,

      continueWaitTimeout: this.continueWaitTimeout,

      orphanedTimeout: this.orphanedTimeout,

      heartBeatTimeout: this.heartBeatInterval * 4,

      redisPool: options.redisPool

    };

    this.queueDriver = options.cacheAndQueueDriver === 'redis' ?

      new RedisQueueDriver(queueDriverOptions) :

      new LocalQueueDriver(queueDriverOptions);

  }

packages/cubejs-query-orchestrator/src/orchestrator/QueryCache.ts
参考构造函数

 

 public constructor(

    protected readonly redisPrefix: string,

    protected readonly driverFactory: DriverFactoryByDataSource,

    protected readonly logger: any,

    protected readonly options: {

      refreshKeyRenewalThreshold?: number;

      externalQueueOptions?: any;

      externalDriverFactory?: DriverFactory;

      backgroundRenew?: Boolean;

      queueOptions?: object | ((dataSource: String) => object);

      redisPool?: any;

      continueWaitTimeout?: number;

      cacheAndQueueDriver?: 'redis' | 'memory';

      maxInMemoryCacheEntries?: number;

    } = {}

  ) {

    this.cacheDriver = options.cacheAndQueueDriver === 'redis' ?

      new RedisCacheDriver({ pool: options.redisPool }) :

      new LocalCacheDriver();

    this.memoryCache = new LRUCache<string, CacheEntry>({

      max: options.maxInMemoryCacheEntries || 10000

    });

  }

参考配置

• 调度

edisPrefix  Prefix to be set an all Redis keys  STANDALONE

rollupOnlyMode  When enabled, an error will be thrown if a query can't be served from a pre-aggregation (rollup)  false

queryCacheOptions Query cache options for DB queries  {}

queryCacheOptions.refreshKeyRenewalThreshold  Time in seconds to cache the result of refreshKey check defined by DB dialect

queryCacheOptions.backgroundRenew Controls whether to wait in foreground for refreshed query data if refreshKey value has been changed. Refresh key queries or pre-aggregations are never awaited in foreground and always processed in background unless cache is empty. If true it immediately returns values from cache if available without refreshKey check to renew in foreground. Default value before 0.15.0 was true false

queryCacheOptions.queueOptions  Query queue options for DB queries  {}

preAggregationsOptions  Query cache options for pre-aggregations  {}

preAggregationsOptions.queueOptions Query queue options for pre-aggregations  {}

preAggregationsOptions.externalRefresh  When running a separate instance of Cube.js to refresh pre-aggregations in the background, this option can be set on the API instance to prevent it from trying to check for rollup data being current - it won't try to create or refresh them when this option is true

• 查询

concurrency Maximum number of queries to be processed simultaneosly. For drivers with connection pool CUBEJS_DB_MAX_POOL should be adjusted accordingly.  2

continueWaitTimeout Long polling interval 5

executionTimeout  Total timeout of single query 600

orphanedTimeout Query will be marked for cancellation if not requested during this period.  120

heartBeatInterval Worker heartbeat interval. If 4*heartBeatInterval time passes without reporting, the query gets cancelled.  30

说明

以上配置来自官方文档以及代码，实际使用还是比较有用的，对于一些部署中的优化很重要，有些参数可能官方文档没有提供，但是通过代码可以查看了解

参考资料

https://cube.dev/docs/config#options-reference-orchestrator-options