Django——django-json-rpc、supervisor配置celery、celery运行

开放项目的啥时候，遇到的一些新技术，记录一下

OSS接口

短信接口

JSON-RPC

pip install django-json-rpc

基本的API：

myproj / myapp / views.py

from jsonrpc import jsonrpc_method

@jsonrpc_method('myapp.sayHello')
def whats_the_time(request, name='Lester'):
  return "Hello %s" % name

@jsonrpc_method('myapp.gimmeThat', authenticated=True)
def something_special(request, secret_data):
  return {'sauce': ['authenticated', 'sauce']}

myproj / urls.py

from django.conf.urls.defaults import *
from jsonrpc import jsonrpc_site
import myproj.myapp.views # you must import the views that need connected

urlpatterns = patterns('',
  url(r'^json/browse/', 'jsonrpc.views.browse', name="jsonrpc_browser"), # for the graphical browser/web console only, omissible
  url(r'^json/', jsonrpc_site.dispatch, name="jsonrpc_mountpoint"),
  (r'^json/(?P<method>[a-zA-Z0-9.]+)$', jsonrpc_site.dispatch) # for HTTP GET only, also omissible
)

测试您的服务： 您可以使用提供的图形浏览器和控制台来测试您的服务，该浏览器可以从http：// YOUR_URL / json / browse /获得（如果使用上面的url模式），或者与随附的ServiceProxy一起使用：

>>> from jsonrpc.proxy import ServiceProxy

>>> s = ServiceProxy('http://localhost:8080/json/')

>>> s.myapp.sayHello('Sam')
{u'error': None, u'id': u'jsonrpc', u'result': u'Hello Sam'}

>>> s.myapp.gimmeThat('username', 'password', 'test data')
{u'error': None, u'id': u'jsonrpc', u'result': {u'sauce': [u'authenticated', u'sauce']}}

我们将jsonrpc_version变量添加到请求对象。它可以是“ 1.0”，“ 1.1”或“ 2.0”

How to keep Celery running with supervisor

Installation

First, you need to install supervisor in your virtualenv and generate a configuration file.

I store a supervisord.conf config file at the root of each project, and also, be careful to use the absolute path to the Python interpreter of the virtualenv.

$ pip install supervisor
$ cd /path/to/your/project
$ echo_supervisord_conf > supervisord.conf

Next, just add this section after the [supervisord] section:

[program:celeryd]
command=/home/thomas/virtualenvs/yourvenv/bin/celery -A=myapp.main worker -l info 
stdout_logfile=/path/to/your/logs/celeryd.log
stderr_logfile=/path/to/your/logs/celeryd.log
autostart=true
autorestart=true
startsecs=10
stopwaitsecs=600

It's a simplified version of the Celery supervisor example configuration file, adapted to work with virtualenvs.

Usage

Just run supervisord in your project directory.

$ supervisord

Then, you can use the supervisorctl command to enter the interactive shell. Type help to get started. You can also execute supervisor command directly:

$ supervisorctl tail celeryd
$ supervisorctl restart celeryd

常用配置

supervisorctl status ： 查看所管理的服务状态； 
supervisorctl start <program_name>：启动一个服务； 
supervisorctl restart <program_name>：重启一个服务（注意：重启服务不会重新加载配置文件）； 
supervisorctl stop <program_name>：关闭一个服务； 
supervisorctl update：重新加载配置文件，并重启配置有变动的服务； 
supervisorctl reread：重新加载配置文件，但不会重启配置有变动的服务； 
supervisorctl reload：重启 Supervisor 服务端； 
supervisorctl clear <program_name>：清理一个服务的 stdout log；

修改配置以后这样重新启动

首先进入 supervisor 控制台：

supervisorctl

然后重新读取配置：

reread

更新配置：

update

开始所有配置：

start all

查看所有状态：

status

至此，解决了新的守护进程的生效问题。

celery-redis不同系统的运行方法

正常在linux系统下，配置supervisor就可以使用celery-redis了，但是windows系统，celery4及以上的版本已经不支持了，所以需要安装

通过以下方法可使windows上支持celery 4.x以上版本。
方法一：
安装eventlet模块

pip install eventlet

重新启动celery worker

celery_testScriptscelery.exe worker -A celery_task -l info -P eventlet

调用任务
再打开一个终端， 进行命令行模式，调用任务

# -*- coding: utf-8 -*-
from celery_task.task1 import test_celery
# 立即告知celery去执行test_celery任务，并传入一个参数
result = test_celery.delay('第一个的执行')

运行正常

 -------------- celery@PC-20151019PZHE v4.3.0 (rhubarb)
---- **** -----
--- * ***  * -- Windows-7-6.1.7601-SP1 2019-08-04 14:35:05
-- * - **** ---
- ** ---------- [config]
- ** ---------- .> app:         tasks:0x3854790
- ** ---------- .> transport:   redis://:**@192.168.190.128:6379//
- ** ---------- .> results:     redis://:**@192.168.190.128/
- *** --- * --- .> concurrency: 4 (eventlet)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
 -------------- [queues]
                .> celery           exchange=celery(direct) key=celery


[tasks]
  . celery_test.add

[2019-08-04 14:35:05,356: INFO/MainProcess] Connected to redis://:**@192.168.190.128:6379//
[2019-08-04 14:35:05,381: INFO/MainProcess] mingle: searching for neighbors
[2019-08-04 14:35:07,969: INFO/MainProcess] mingle: all alone
[2019-08-04 14:35:08,011: INFO/MainProcess] pidbox: Connected to redis://:**@192.168.190.128:6379//.
[2019-08-04 14:35:08,074: INFO/MainProcess] celery@PC-20151019PZHE ready.
[2019-08-04 14:35:32,146: INFO/MainProcess] Received task: celery_test.add[3861c769-21b0-4d40-bffb-093461745d41]
[2019-08-04 14:35:32,148: WARNING/MainProcess] running...
[2019-08-04 14:35:32,151: WARNING/MainProcess] 4
[2019-08-04 14:35:32,151: WARNING/MainProcess] 0
[2019-08-04 14:35:32,163: INFO/MainProcess] Task celery_test.add[3861c769-21b0-4d40-bffb-093461745d41] succeeded in 0.01599999999962165s
: 4

方法二：

如果涉及到多进程，则安装以下模块。

安装gevent模块

pip install gevent

重新启动celery worker

celery -A celery_test worker --loglevel=info -P gevent