python3 异步任务之----celery

celery是一个“自带电池”的任务队列。

运行环境：

• Django==1.11.4
• PyMySQL==0.8.1
• configparser==3.5.0
• django-crontab==0.7.1
• celery==3.1.25
• redis==3.2.8

工程列表：

1. 在工程下的settings.py文件中添加如下内容：

BROKER_URL = 'redis://127.0.0.1:6379'
CELERY_RESULT_BACKEND = 'redis://127.0.0.1:6379'
CELERY_ACCEPT_CONTENT = ['application/json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_TIMEZONE = 'Asia/Shanghai

　　2.在工程目录下添加celery.py文件，增加如下内容：

# coding=utf8
# Autor : Vglede
# Time  : 2018/6/26 16:53
# File  : celery.py

from __future__ import absolute_import
import os
from celery import Celery
from django.conf import settings

# set the default Django settings module for the 'Ops' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'Ops.settings')
app = Celery('release')

# Using a string here means the worker will not have to
# pickle the object when using Windows.
app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)


@app.task(bind=True)
def debug_task(self):
    print('Request: {0!r}'.format(self.request))

　　3.在工程下面__init__.py添加如下代码

from .celery import app as celery_app

　　4.在注册的应用（release）下添加任务：

# coding=utf8
# Autor : Vglede
# Time  : 2018/6/27 10:46
# File  : release_api.py
from celery import task

@task
def release_async_bash(src_info):
   print("job[release_async_bash] include %s is running"%(str(src_info)))
    result = True
    return result

　　5.通过任务函数名，使用delay()方法去启动任务。

#somecode
exec_order="哈哈"
release_async_bash.delay(exec_order)

　　6.创建pid以及log存放目录：

mkdir  -p /var/run/celery/
mkdir  -p /var/log/celery/

　　7.启动celery（附上编者写的脚本）：

# coding=utf8
# Autor : Vglede
# Time  : 2018/6/28 14:53
# File  : contorl_celery.sh

#!/bin/sh
#

case "$1" in
    start)
        celery multi start w1 -A Ops -l info --pidfile=/var/run/celery/%n.pid 
                                        --logfile=/var/log/celery/%n.log
        ;;
    stop)
        if [ ! -f $PIDFILE ]
        then
                echo "$PIDFILE does not exist, process is not running"
        else
                PIDFILE=`ls /var/run/celery/*.pid`
                PID=$(cat $PIDFILE)
                echo "Stopping ..."
                celery multi stop w1 -A Ops -l info --pidfile=/var/run/celery/%n.pid 
                                        --logfile=/var/log/celery/%n.log
                while [ -x /proc/${PID} ]
                do
                    echo "Waiting for Celery to shutdown ..."
                    sleep 1
                done
                echo "Celery stopped"
        fi
        ;;
    restart)
        if [ ! -f $PIDFILE ]
        then
                echo "$PIDFILE does not exist, process is not running"
        else
                PIDFILE=`ls /var/run/celery/*.pid`
                PID=$(cat $PIDFILE)
                echo "Stopping ..."
                celery multi stop w1 -A Ops -l info --pidfile=/var/run/celery/%n.pid 
                                        --logfile=/var/log/celery/%n.log
                while [ -x /proc/${PID} ]
                do
                    echo "Waiting for Celery to shutdown ..."
                    sleep 1
                done
                echo "Celery stopped"
        fi
        celery multi start w1 -A Ops -l info --pidfile=/var/run/celery/%n.pid 
                                        --logfile=/var/log/celery/%n.log
        ;;

    *)
        echo "Please use start or stop as first argument"
        ;;
esac 

 了解更多，参考celery3.1.7文档（http://docs.jinkan.org/docs/celery/getting-started/first-steps-with-celery.html）