zoukankan      html  css  js  c++  java
  • celery使用的时候的坑

    一、delay函数或者apply_async函数的传参问题

    1、通过delay或者apply_async传参数给异步任务的时候不能传实例,否则会报错raised unexpected: EncodeError(TypeError('<ExtractCashRecord: ExtractCashRecord object> is not JSON serializable,因为我传的参数有一个是ExtractCashRecord这个类的实例。

    二、任务重复执行

    celery使用redis作为broker的话,apply_async的异步任务如果执行之间是延迟超过1小时的话,celery会重复发布任务,会导致任务重复执行。这个在官方文档中有说明celery文档

    if a task isn’t acknowledged within the Visibility Timeout the task will be redelivered to another worker and executed.This causes problems with ETA/countdown/retry tasks where the time to execute exceeds the visibility timeout; in fact if that happens it will be executed again, and again in a loop.So you have to increase the visibility timeout to match the time of the longest ETA you’re planning to use.Note that Celery will redeliver messages at worker shutdown, so having a long visibility timeout will only delay the redelivery of ‘lost’ tasks in the event of a power failure or forcefully terminated workers.Periodic tasks won’t be affected by the visibility timeout, as this is a concept separate from ETA/countdown.You can increase this timeout by configuring a transport option with the same name:
    app.conf.broker_transport_options = {'visibility_timeout': 43200}*
    The value must be an int describing the number of seconds.

    这个时候,我们就需要在配置文件里面增加一项,来增大visibility_timeout的时间
    BROKER_TRANSPORT_OPTIONS = {'visibility_timeout': 43200}
    或者
    app.conf.broker_transport_options = {'visibility_timeout': 43200}

  • 相关阅读:
    机器学习十四--深度学习-卷积
    机器学习十三——垃圾邮件分类2
    机器学习十二----朴素贝叶斯-垃圾邮件分类
    机器学习十一 ——分类与监督学习,朴素贝叶斯分类算法
    机器学习九----主成分分析
    机器学习八——特征选择
    基于D3.js 绘制一个折柱混合图
    一个轮播图
    贪吃蛇
    数组中哪些方法是响应式的
  • 原文地址:https://www.cnblogs.com/lanlingshao/p/10087280.html
Copyright © 2011-2022 走看看