zoukankan      html  css  js  c++  java
  • linux 后台进程管理利器supervisor

    Linux的后台进程运行有好几种方法,例如nohup,screen等,但是,如果是一个服务程序,要可靠地在后台运行,我们就需要把它做成daemon,最好还能监控进程状态,在意外结束时能自动重启
     
    supervisor就是用Python开发的一套通用的进程管理程序,能将一个普通的命令行进程变为后台daemon,并监控进程状态,异常退出时能自动重启。
    ubuntu安装:
    apt-get install supervisor
    在/etc/supervisor 目录下有supervisord.conf 文件,内容如下:
    ; supervisor config file
    
    [unix_http_server]
    file=/var/run/supervisor.sock   ; (the path to the socket file)
    chmod=0700                       ; sockef file mode (default 0700)
    
    [supervisord]
    logfile=/var/log/supervisor/supervisord.log ; (main log file;default $CWD/supervisord.log)
    pidfile=/var/run/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
    childlogdir=/var/log/supervisor            ; ('AUTO' child log dir, default $TEMP)
    
    ; the below section must remain in the config file for RPC
    ; (supervisorctl/web interface) to work, additional interfaces may be
    ; added by defining them in separate rpcinterface: sections
    [rpcinterface:supervisor]
    supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
    
    [supervisorctl]
    serverurl=unix:///var/run/supervisor.sock ; use a unix:// URL  for a unix socket
    
    ; The [include] section can just contain the "files" setting.  This
    ; setting can list multiple files (separated by whitespace or
    ; newlines).  It can also contain wildcards.  The filenames are
    ; interpreted as relative to this file.  Included files *cannot*
    ; include files themselves.
    
    [include]
    files = /etc/supervisor/conf.d/*.conf
    然后,给我们自己开发的应用程序编写一个配置文件,让supervisor来管理它。每个进程的配置文件都可以单独分拆,放在/etc/supervisor/conf.d/目录下,以.conf作为扩展名,如果修改了 /etc/supervisord.conf ,,需要执行 supervisorctl reload (重启)来重新加载配置文件,否则会感觉没有生效,折腾到抓狂。。。
    如,app.conf定义了一个gunicorn的进程:
     
    [html] 
    [program:app]  
    command=/usr/bin/gunicorn -w 1 wsgiapp:application  
    directory=/srv/www  
    user=www-data  
     
    其中,进程app定义在[program:app]中,command是命令,directory是进程的当前目录,user是进程运行的用户身份。
     
    重启supervisor,让配置文件生效,
     
    # supervisorctl start app  
     
    然后运行命令supervisorctl启动进程:
    停止进程:
     
     
    # supervisorctl stop app  
     
    如果要在命令行中使用变量,就需要自己先编写一个shell脚本:
     
    [html] 
    #!/bin/sh  
    /usr/bin/gunicorn -w `grep -c ^processor /proc/cpuinfo` wsgiapp:application  
     
    然后,加上x权限,再把command指向该shell脚本即可。
     

    先弄懂两个命令:

    supervisord : supervisor的服务器端部分,启动supervisor就是运行这个命令

    supervisorctl:启动supervisor的命令行窗口。

    参考:http://www.2cto.com/os/201308/238166.html

    一篇文章:使用supervisor监控进程:

    在linux下监控进程,可以使用inittab,最近找到了supervisor,也很好用,记录一下:
    1、系统要安装python,并安装与之对应的setuptools,下载地址在此
    2、安装:
    # sh setuptoolsxxxx.egg
    3、安装supervisor,下载地址在此,解压缩后
    # python setup.py install
    这就ok了,然后执行
    # echo_supervisord_conf > /etc/supervisord.conf
    修改/etc/supervisord.conf文件,加入你要监控的进程,里面的注释很详细,举个简单的例子:
    这是一段要监控的进程的描述信息,添加到这个文件的末尾就好了:
    [program:meta.txn.recover.on.error]
    command=/cas/bin/meta.txn.recover.on.error ; 被监控的进程路径
    numprocs=1                    ; 启动几个进程
    directory=/cas/bin                ; 执行前要不要先cd到目录去,一般不用
    autostart=true                ; 随着supervisord的启动而启动
    autorestart=true              ; 自动重启。。当然要选上了
    startretries=10               ; 启动失败时的最多重试次数
    exitcodes=0                 ; 正常退出代码(是说退出代码是这个时就不再重启了吗?待确定)
    stopsignal=KILL               ; 用来杀死进程的信号
    stopwaitsecs=10               ; 发送SIGKILL前的等待时间
    redirect_stderr=true          ; 重定向stderr到stdout
    为了节省空间,注释的内容就不贴出来了。
    执行
    # supervisord -n
    能在控制台看到监控进程的输出:
    2010-08-17 10:26:07,467 INFO supervisord started with pid 943
    2010-08-17 10:26:08,469 INFO spawned: 'meta.txn.recover.on.error' with pid 1009
    2010-08-17 10:26:09,876 INFO success: meta.txn.recover.on.error entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
    2010-08-17 10:26:48,442 INFO exited: meta.txn.recover.on.error (terminated by SIGKILL; not expected)
    2010-08-17 10:26:49,444 INFO spawned: 'meta.txn.recover.on.error' with pid 2427
    2010-08-17 10:26:50,487 INFO success: meta.txn.recover.on.error entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
    黑体的地方是我用kill -9杀掉进程后出来的,看到supervisor检测到进程退出后又再次启动了进程。
    不带参数运行supervisord是以daemon方式运行。
    把supervisord加入到开机启动项里就可以完成监控进程的功能了。

    【注意】:当supervisord以非daemon方式运行时,杀掉supervisord后,被监控的进程也退出了。
    而以daemon方式运行,杀掉supervisord对被监控进程无影响。

    python安装:

    easy_install supervisor

    测试是否安装成功

    echo_supervisord_conf

    会显示以下内容,其实是一个配置模版:

    ; Sample supervisor config file.
    ;
    ; For more information on the config file, please see:
    ; http://supervisord.org/configuration.html
    ;
    ; Note: shell expansion ("~" or "$HOME") is not supported. Environment
    ; variables can be expanded using this syntax: "%(ENV_HOME)s".

    [unix_http_server]
    file=/tmp/supervisor.sock ; (the path to the socket file)
    ;chmod=0700 ; socket file mode (default 0700)
    ;chown=nobody:nogroup ; socket file uid:gid owner
    ;username=user ; (default is no username (open server))
    ;password=123 ; (default is no password (open server))

    ;[inet_http_server] ; inet (TCP) server disabled by default
    ;port=127.0.0.1:9001 ; (ip_address:port specifier, *:port for all iface)
    ;username=user ; (default is no username (open server))
    ;password=123 ; (default is no password (open server))

    [supervisord]
    logfile=/tmp/supervisord.log ; (main log file;default $CWD/supervisord.log)
    logfile_maxbytes=50MB ; (max main logfile bytes b4 rotation;default 50MB)
    logfile_backups=10 ; (num of main logfile rotation backups;default 10)
    loglevel=info ; (log level;default info; others: debug,warn,trace)
    pidfile=/tmp/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
    ……

    ; the below section must remain in the config file for RPC
    ; (supervisorctl/web interface) to work, additional interfaces may be
    ; added by defining them in separate rpcinterface: sections
    [rpcinterface:supervisor]
    supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

    [supervisorctl]
    serverurl=unix:///tmp/supervisor.sock ; use a unix:// URL for a unix socket
    ;serverurl=http://127.0.0.1:9001 ; use an http:// url to specify an inet socket
    ;username=chris ; should be same as http_username if set
    ;password=123 ; should be same as http_password if set
    ;prompt=mysupervisor ; cmd line prompt (default "supervisor")
    ;history_file=~/.sc_history ; use readline history if available

    ; The below sample program section shows all possible program subsection values,
    ; create one or more ‘real’ program: sections to be able to control them under
    ; supervisor.

    ;[program:theprogramname]
    ;command=/bin/cat ; the program (relative uses PATH, can take args)
    ;process_name=%(program_name)s ; process_name expr (default %(program_name)s)
    ;numprocs=1 ; number of processes copies to start (def 1)
    ;directory=/tmp ; directory to cwd to before exec (def no cwd)
    ;umask=022 ; umask for process (default None)
    ;priority=999 ; the relative start priority (default 999)
    ;autostart=true ; start at supervisord start (default: true)
    ;autorestart=unexpected ; whether/when to restart (default: unexpected)
    ;startsecs=1 ; number of secs prog must stay running (def. 1)
    ;startretries=3 ; max # of serial start failures (default 3)
    ;exitcodes=0,2 ; ‘expected’ exit codes for process (default 0,2)
    ;stopsignal=QUIT ; signal used to kill process (default TERM)
    ;stopwaitsecs=10 ; max num secs to wait b4 SIGKILL (default 10)
    ……


    ; The [include] section can just contain the "files" setting. This
    ; setting can list multiple files (separated by whitespace or
    ; newlines). It can also contain wildcards. The filenames are
    ; interpreted as relative to this file. Included files *cannot*
    ; include files themselves.

    ;[include]
    ;files = relative/directory/*.ini

    创建配置文件

    echo_supervisord_conf > /etc/supervisord.conf

    (easy_install安装的方式好像没有/etc/supervisord.conf,所以这里我们手动创建)

    修改配置文件

    在supervisord.conf最后增加(分号后边的表示注释,可以不写):

    [program:open_search]
    command=/home/s/www/fukun/gopath/src/open_search/open_search ;需要执行的命令
    directory=/home/s/www/fukun/gopath/src/open_search/ ; directory to cwd to before exec (def no cwd)
    autostart=true ; start at supervisord start (default: true)
    autorestart=true ; whether/when to restart (default: unexpected)
    startsecs=3 ; number of secs prog must stay running (def. 1)
    redirect_stderr=true ; redirect proc stderr to stdout (default false) 错误输出重定向
    stdout_logfile=/tmp/open_search_gorun.log ; stdout log path, NONE for none; default AUTO, log输出

    (更多配置说明请参考:http://supervisord.org/configuration.html)

     supervisor管理gearman

    参考官方文档:

    http://supervisord.org/running.html

    Running supervisord automatically on startup

    If you are using a distribution-packaged version of Supervisor, it should already be integrated into the service management infrastructure of your distribution.

    There are user-contributed scripts for various operating systems at:https://github.com/Supervisor/initscripts

    There are some answers at Serverfault in case you get stuck: How to automatically start supervisord on Linux (Ubuntu)

    github的脚本下载下来:

    #!/bin/bash
    #
    # supervisord   This scripts turns supervisord on
    #
    # Author:       Mike McGrath <mmcgrath@redhat.com> (based off yumupdatesd)
    #               Jason Koppe <jkoppe@indeed.com> adjusted to read sysconfig,
    #                   use supervisord tools to start/stop, conditionally wait
    #                   for child processes to shutdown, and startup later
    #               Mikhail Mingalev <mingalevme@gmail.com> Merged
    #                   redhat-init-jkoppe and redhat-sysconfig-jkoppe, and
    #                   made the script "simple customizable".
    #               Brendan Maguire <maguire.brendan@gmail.com> Added OPTIONS to
    #                   SUPERVISORCTL status call
    #
    # chkconfig:    345 83 04
    #
    # description:  supervisor is a process control utility.  It has a web based
    #               xmlrpc interface as well as a few other nifty features.
    #               Script was originally written by Jason Koppe <jkoppe@indeed.com>.
    #
    
    # source function library
    . /etc/rc.d/init.d/functions
    
    set -a
    
    PREFIX=/usr
    
    SUPERVISORD=$PREFIX/bin/supervisord
    SUPERVISORCTL=$PREFIX/bin/supervisorctl
    
    PIDFILE=/var/run/supervisord.pid
    LOCKFILE=/var/lock/subsys/supervisord
    
    OPTIONS="-c /etc/supervisord.conf"
    
    # unset this variable if you don't care to wait for child processes to shutdown before removing the $LOCKFILE-lock
    WAIT_FOR_SUBPROCESSES=yes
    
    # remove this if you manage number of open files in some other fashion
    ulimit -n 96000
    
    RETVAL=0
    
    
    running_pid()
    {
        # Check if a given process pid's cmdline matches a given name
        pid=$1
        name=$2
        [ -z "$pid" ] && return 1
        [ ! -d /proc/$pid ] && return 1
        (cat /proc/$pid/cmdline | tr "00" "
    "|grep -q $name) || return 1
        return 0
    }
    
    running()
    {
    # Check if the process is running looking at /proc
    # (works for all users)
    
        # No pidfile, probably no daemon present
        [ ! -f "$PIDFILE" ] && return 1
        # Obtain the pid and check it against the binary name
        pid=`cat $PIDFILE`
        running_pid $pid $SUPERVISORD || return 1
        return 0
    }
    
    start() {
            echo "Starting supervisord: "
        
            if [ -e $PIDFILE ]; then 
            echo "ALREADY STARTED"
            return 1
        fi
    
        # start supervisord with options from sysconfig (stuff like -c)
            $SUPERVISORD $OPTIONS
        
        # show initial startup status
        $SUPERVISORCTL $OPTIONS status
        
        # only create the subsyslock if we created the PIDFILE
            [ -e $PIDFILE ] && touch $LOCKFILE
    }
    
    stop() {
            echo -n "Stopping supervisord: "
            $SUPERVISORCTL $OPTIONS shutdown
        if [ -n "$WAIT_FOR_SUBPROCESSES" ]; then 
                echo "Waiting roughly 60 seconds for $PIDFILE to be removed after child processes exit"
                for sleep in  2 2 2 2 4 4 4 4 8 8 8 8 last; do
                    if [ ! -e $PIDFILE ] ; then
                        echo "Supervisord exited as expected in under $total_sleep seconds"
                        break
                    else
                        if [[ $sleep -eq "last" ]] ; then
                            echo "Supervisord still working on shutting down. We've waited roughly 60 seconds, we'll let it do its thing from here"
                            return 1
                        else
                            sleep $sleep
                            total_sleep=$(( $total_sleep + $sleep ))
                        fi
    
                    fi
                done
            fi
    
            # always remove the subsys. We might have waited a while, but just remove it at this point.
            rm -f $LOCKFILE
    }
    
    restart() {
            stop
            start
    }
    
    case "$1" in
        start)
            start
            RETVAL=$?
            ;;
        stop)
            stop
            RETVAL=$?
            ;;
        restart|force-reload)
            restart
            RETVAL=$?
            ;;
        reload)
            $SUPERVISORCTL $OPTIONS reload
            RETVAL=$?
            ;;
        condrestart)
            [ -f $LOCKFILE ] && restart
            RETVAL=$?
            ;;
        status)
            $SUPERVISORCTL $OPTIONS status
            if running ; then
                RETVAL=0
            else
                RETVAL=1
            fi
            ;;
        *)
            echo $"Usage: $0 {start|stop|status|restart|reload|force-reload|condrestart}"
            exit 1
    esac
    
    exit $RETVAL

    vim /etc/init.d/supervisord , 修改文件属性为可执行chmod 777 /etc/init.d/supervisordmy

    加入gearman的任务,下面是百度云推送的任务

    vim /etc/supervisord.conf

    [program:APNS_PUSH]
    command=/usr/bin/php /data2/www/web/apns/worker_origin.php
    process_name=APNS_PUSH_%(process_num)s
    numprocs=2
    autostart=true
    autorestart=true
    user=nginx
    stdout_logfile=/data2/log/push/push_%(process_num)s.log
    stderr_logfile=/data2/log/push/push.error.log

    下面先启动gearman,启动supervisord,ps -ef |grep xxx 查看一下任务有没有执行

    参考:http://www.cnblogs.com/trying/p/4038285.html

     Supervisord是用Python实现的一款非常实用的进程管理工具,在批量服务化管理时特别有效。

    1、安装
    安装环境 centos python2.4
    安装 wget http://pypi.python.org/packages/source/s/supervisor/supervisor-3.0a10.tar.gz#md5=99c6fbd45bade87301296b7a597fb68e 
    tar zxvf supervisor-3.0a10.tar.gz
    cd supervisor-3.0a10
    python setup.py install
    如果上述遇到问题,请按如下操作。
    依赖setuptools安装 wget http://pypi.python.org/packages/2.4/s/setuptools/setuptools-0.6c11-py2.4.egg#md5=bd639f9b0eac4c42497034dec2ec0c2b 
    sh setuptools-0.6c11-py2.4.egg

    2、配置
    产生基础配置文件 echo_supervisord_conf > /etc/supervisord.conf
    配置 [inet_http_server]前面的分号去掉; 定义port=192.168.1.1:8888 启动 supervisord -c /etc/supervisord.conf 查看 浏览器打开 http://192.168.10.129:9001 
    深入配置 [program:service] 
    command=sh /opt/soft/service/run.sh
    目前我们用supervisord 监控 memcached 进程
    [program:memcached] ###定义服务 
    command=memcached -m 2000 -u nobody -l 0.0.0.0 -c 10240 -p 11211  ###服务运行时的显示的command
    numprocs=1                                                                                       ###控制memcached启动的进程数
    user=root                                                                                            ###程序运行的用户
    autostart=true                                                                                     ###supervisord启动memcached启动
    autorestart=true                                                                                  ###supervisord重启memcache自动重启
    stdout_logfile=/var/log/memcached.stdout.log                                       ###监控memcached日志文件
    redirect_stderr=true                                                                            ###将stderr重定向到stdout(或者创建个err.log)
    stopsignal=QUIT

    这个程序最好不能是daemon后台守护程序。supervisord会作为这个程序的父来启动它

    3、启动
    supervisord (以daemon方式启动)
    或 supervisord -c /etc/supervisord.conf (非daemon)

    supervisord,初始启动Supervisord,启动、管理配置中设置的进程。
    supervisorctl 简单的后台进程控制工具

    supervisorctl stop programxxx,停止某一个进程(programxxx),programxxx为[program:chatdemon]里配置的值
    supervisorctl start programxxx,启动某个进程
    supervisorctl restart programxxx,重启某个进程
    supervisorctl stop all,停止全部进程,注:start、restart、stop都不会载入最新的配置文件。
    supervisorctl reload,载入最新的配置文件,并按新的配置启动、管理所有进程
    supervisorctl reread,当一个服务由自动启动修改为手动启动时执行一下就ok

    不带参数运行supervisord是以daemon方式运行
    当supervisord以非daemon方式运行时,杀掉supervisord后,被监控的进程也退出了。
    而以daemon方式运行,杀掉supervisord对被监控进程无影响

    supervosrd配置文件:

    command 字段设置的是后台守护应用的启动命令, 注意: 该命令必须是在前台执行的, 即会独占控制台, 否则会导致 supervisor 无法获得标准输出, 并失去进程的控制权.

    更多:

    http://digdeeply.org/archives/07102224.html

  • 相关阅读:
    ioctlsocket()
    阻塞式socket例子学习
    accept()
    listen()
    WSAStartup
    C#动态操作DataTable(新增行、列、查询行、列等)
    C# 连接SQL Server数据库的几种方式--server+data source等方式
    ExcelHelper类
    c#使用椭圆签名算法制作软件序列号
    LINQ查询操作符之First、FirstOrDefault、Last、LastOrDefault、ElementAt、ElementAtOrDefault、Contains、Any、All、Count 等
  • 原文地址:https://www.cnblogs.com/youxin/p/4147384.html
Copyright © 2011-2022 走看看