zoukankan      html  css  js  c++  java
  • celery的安装和使用

     celery是python开发的分布式任务调度模块,接口简单,开发容易,五分钟就写出一个异步发送邮件的服务,celery本身不含消息服务,它使用第三方消息服务来传递任务,目前,celery支持的消息服务有RabbitMQ,redis甚至是数据库,redis是最佳选择

    Windows使用celery只能安装 3.1.25版
    
    pip install celery==3.1.25  

    编写tasks.py

    # tasks.py
    import time
    from celery import Celery
    
    celery = Celery('tasks', broker='redis://localhost:6379/0')
    
    @celery.task
    def sendmail(mail):
        print('sending mail to %s...' % mail['to'])
        time.sleep(2.0)
        print('mail sent.')

    启动celery处理任务

    celery -A tasks worker --loglevel=info

    tasks是任务文件名,worker任务角色,--loglevel=info 任务日志级别

    上面的命令行实际上启动的是worker,如果要放到后台运行,可以扔给supervisor

    要在目录下启动

    D:py3codejintong_day1ce>celery -A tasks worker --loglevel=info
    [2018-05-30 19:37:15,815: WARNING/MainProcess] d:py3.6libsite-packagescelery
    appsworker.py:161: CDeprecationWarning:
    Starting from version 3.2 Celery will refuse to accept pickle by default.
    
    The pickle serializer is a security concern as it may give attackers
    the ability to execute any command.  It's important to secure
    your broker from unauthorized access when using pickle, so we think
    that enabling pickle should require a deliberate action and not be
    the default choice.
    
    If you depend on pickle then you should set a setting to disable this
    warning and to be sure that everything will continue working
    when you upgrade to Celery 3.2::
    
        CELERY_ACCEPT_CONTENT = ['pickle', 'json', 'msgpack', 'yaml']
    
    You must only enable the serializers that you will actually use.
    
    
      warnings.warn(CDeprecationWarning(W_PICKLE_DEPRECATED))
    
     -------------- celery@E73-PC v3.1.25 (Cipater)
    ---- **** -----
    --- * ***  * -- Windows-7-6.1.7601-SP1
    -- * - **** ---
    - ** ---------- [config]
    - ** ---------- .> app:         tasks:0x395deb8
    - ** ---------- .> transport:   redis://localhost:6379/0
    - ** ---------- .> results:     disabled://
    - *** --- * --- .> concurrency: 4 (prefork)
    -- ******* ----
    --- ***** ----- [queues]
     -------------- .> celery           exchange=celery(direct) key=celery
    
    
    [tasks]
      . tasks.sendmail
    
    [2018-05-30 19:37:15,926: INFO/MainProcess] Connected to redis://localhost:6379/
    0
    [2018-05-30 19:37:15,995: INFO/MainProcess] mingle: searching for neighbors
    [2018-05-30 19:37:17,160: INFO/MainProcess] mingle: all alone
    [2018-05-30 19:37:17,183: WARNING/MainProcess] celery@E73-PC ready.

    怎么发送任务

    在当前目录下打开命令行,进入python

    >>> from tasks import sendmail
    >>> sendmail.delay(dict(to='celery@python.org'))
    <AsyncResult: 1a0a9262-7858-4192-9981-b7bf0ea7483b>

    在worker里就可以看到任务处理的消息

    [2018-05-30 19:36:13,517: INFO/MainProcess] Received task: tasks.sendmail[815178
    90-2406-4756-a4b5-d650ea8cd2e2]
    [2018-05-30 19:36:13,517: WARNING/Worker-1] sending mail to celery@hhh
    [2018-05-30 19:36:15,524: WARNING/Worker-1] mail sent
    [2018-05-30 19:36:15,524: INFO/MainProcess] Task tasks.sendmail[81517890-2406-47
    56-a4b5-d650ea8cd2e2] succeeded in 1.9970000000030268s: None

     celery默认设置就能满足基本要求,worker以pool模式启动,默认大小为CPU核心数量,缺省化机制为pickle,但可以指定为json,由于python调用UNIX/Linux太容易,所以,用celery作为异步任务框架非常合适

    celery还有一些高级用法,比如把多个任务组合成一个原子任务,还有一个完善的监控接口

  • 相关阅读:
    网络流之转换为对偶图
    BZOJ 1051: [HAOI2006]受欢迎的牛(SCC)
    BZOJ[HNOI2005]狡猾的商人(差分约束)
    BZOJ [ZJOI2007]矩阵游戏(二分图匹配)
    BZOJ 1191: [HNOI2006]超级英雄Hero(二分图匹配)
    BZOJ 1270: [BeijingWc2008]雷涛的小猫(DP)
    BZOJ 1303: [CQOI2009]中位数图
    BZOJ [HNOI2006]鬼谷子的钱袋
    BZOJ1002 [FJOI2007]轮状病毒(最小生成树计数)
    A* 算法讲解
  • 原文地址:https://www.cnblogs.com/z-x-y/p/9112926.html
Copyright © 2011-2022 走看看