在这篇文章中,提到了Master进程对信号的处理函数,其中有两个信号比较有意思。
SIGHUP:用来热更新(Reload)应用
SIGUSR2:用来在线升级(upgrade on the fly)gunicorn
下面来详细看一下:
SIGHUP:
Reload the configuration, start the new worker processes with a new configuration and gracefully shutdown older workers.
SIGUP对应的信号处理函数是Arbiter.reload。简化后的核心代码如下:
1 def reload(self): 2 old_address = self.cfg.address 3 4 # reload conf,重新加载配置 5 self.app.reload() 6 self.setup(self.app) 7 8 # reopen log files 9 self.log.reopen_files() 10 11 # do we need to change listener ?,处理监听端口变化的情况 12 if old_address != self.cfg.address: 13 # close all listeners 14 [l.close() for l in self.LISTENERS] 15 # init new listeners 16 self.LISTENERS = create_sockets(self.cfg, self.log) 17 listeners_str = ",".join([str(l) for l in self.LISTENERS]) 18 self.log.info("Listening at: %s", listeners_str) 19 20 # spawn new workers,启动新的Worker(数量和类型来自新的配置) 21 for i in range(self.cfg.workers): 22 self.spawn_worker() 23 24 # manage workers,这里会kill掉原来的Worker 25 self.manage_workers()
在上面的引用中提到,会“优雅”(graceful)地kill掉老的worker进程。实现也很简单,Arbiter为每一个fork出的worker进程设置一个自增的“worker_age”,worker中这个属性越小,表明这个worker越老。在manage_workers中,如果已经启动的worker进程数量大于配置中的数量,那么会kill掉比较老的worker进程。代码如下
1 def manage_workers(self): 2 """ 3 Maintain the number of workers by spawning or killing 4 as required. 5 """ 6 if len(self.WORKERS.keys()) < self.num_workers: 7 self.spawn_workers() 8 9 workers = self.WORKERS.items() 10 workers = sorted(workers, key=lambda w: w[1].age) # 按worker的age顺序排序 11 while len(workers) > self.num_workers: # num_workers是配置的worker数量 12 (pid, _) = workers.pop(0) 13 self.kill_worker(pid, signal.SIGTERM)
SIGUSR2
Upgrade the Gunicorn on the fly. A separate TERM signal should be used to kill the old process.
对应的信号处理函数是Arbiter.reexec,该函数会重新fork出新的master-workers进程,但不会影响到原来的master,worker进程,所以上面提到需要将原来的进程kill掉。
1 def reexec(self): 2 """ 3 Relaunch the master and workers. 4 """ 5 if self.reexec_pid != 0: 6 self.log.warning("USR2 signal ignored. Child exists.") 7 return 8 9 if self.master_pid != 0: 10 self.log.warning("USR2 signal ignored. Parent exists") 11 return 12 13 master_pid = os.getpid() 14 self.reexec_pid = os.fork() 15 if self.reexec_pid != 0: 16 return 17 18 self.cfg.pre_exec(self) 19 20 environ = self.cfg.env_orig.copy() 21 fds = [l.fileno() for l in self.LISTENERS] 22 environ['GUNICORN_FD'] = ",".join([str(fd) for fd in fds]) # 设置了一些环境变量,用来区分是正常启动gunicorn还是通过fork重新启动。 23 environ['GUNICORN_PID'] = str(master_pid) 24 25 os.chdir(self.START_CTX['cwd']) 26 27 # exec the process using the original environnement 28 os.execvpe(self.START_CTX[0], self.START_CTX['args'], environ)
下面做一下实验,两个终端,1号终端运行代码,2号终端发送信号。
step1:
1号终端启动gunirorn: gunicorn -w 2 gunicorn_app:app ,
2号终端查看python进程: ps -ef | grep python,
1号终端输出
[2017-01-19 15:21:23 +0000] [6166] [INFO] Starting gunicorn 19.6.0
[2017-01-19 15:21:23 +0000] [6166] [INFO] Listening at: http://127.0.0.1:8000 (6166)
[2017-01-19 15:21:23 +0000] [6166] [INFO] Using worker: sync
[2017-01-19 15:21:23 +0000] [6171] [INFO] Booting worker with pid: 6171
[2017-01-19 15:21:23 +0000] [6172] [INFO] Booting worker with pid: 6172
2号终端输出
hzliumi+ 6166 5721 0 15:21 pts/0 00:00:00 /usr/bin/python /usr/local/bin/gunicorn -w 2 gunicorn_app:app
hzliumi+ 6171 6166 0 15:21 pts/0 00:00:00 /usr/bin/python /usr/local/bin/gunicorn -w 2 gunicorn_app:app
hzliumi+ 6172 6166 0 15:21 pts/0 00:00:00 /usr/bin/python /usr/local/bin/gunicorn -w 2 gunicorn_app:app
可以看到,master进程的PID是6166,然后两个worker进程的pid分别是6171、6172. 下面需要用到master进程的PID
step2:
2号终端发送信号:kill -SIGUSR2 6166 , 然后查看python进程
1号终端追加输出
[2017-01-19 15:27:08 +0000] [6166] [INFO] Handling signal: usr2
[2017-01-19 15:27:08 +0000] [6629] [INFO] Starting gunicorn 19.6.0
[2017-01-19 15:27:08 +0000] [6629] [INFO] Listening at: http://127.0.0.1:8000 (6629)
[2017-01-19 15:27:08 +0000] [6629] [INFO] Using worker: sync
[2017-01-19 15:27:08 +0000] [6634] [INFO] Booting worker with pid: 6634
[2017-01-19 15:27:08 +0000] [6635] [INFO] Booting worker with pid: 6635
2号终端输出
hzliumi+ 6166 5721 0 15:21 pts/0 00:00:00 /usr/bin/python /usr/local/bin/gunicorn -w 2 gunicorn_app:app
hzliumi+ 6171 6166 0 15:21 pts/0 00:00:00 /usr/bin/python /usr/local/bin/gunicorn -w 2 gunicorn_app:app
hzliumi+ 6172 6166 0 15:21 pts/0 00:00:00 /usr/bin/python /usr/local/bin/gunicorn -w 2 gunicorn_app:app
hzliumi+ 6629 6166 3 15:27 pts/0 00:00:00 /usr/bin/python /usr/local/bin/gunicorn -w 2 gunicorn_app:app
hzliumi+ 6634 6629 0 15:27 pts/0 00:00:00 /usr/bin/python /usr/local/bin/gunicorn -w 2 gunicorn_app:app
hzliumi+ 6635 6629 0 15:27 pts/0 00:00:00 /usr/bin/python /usr/local/bin/gunicorn -w 2 gunicorn_app:app
从1号终端可以看出,基本是重新启动了gunicorn,不过从2号终端输出可以看到,新的master进程(pid 6629)是老的master进程(pid 6166)的子进程
step3:
2号终端发送信号:kill -SIGTERM 6166 , 稍等几秒后查看python进程:
1号终端追加输出
[2017-01-19 15:29:42 +0000] [6166] [INFO] Handling signal: term
[2017-01-19 15:29:42 +0000] [6171] [INFO] Worker exiting (pid: 6171)
[2017-01-19 15:29:42 +0000] [6172] [INFO] Worker exiting (pid: 6172)
[2017-01-19 15:29:42 +0000] [6166] [INFO] Shutting down: Master
[2017-01-19 15:29:42 +0000] [6629] [INFO] Master has been promoted.
2号终端输出
hzliumi+ 6629 1 0 15:27 pts/0 00:00:00 /usr/bin/python /usr/local/bin/gunicorn -w 2 gunicorn_app:app
hzliumi+ 6634 6629 0 15:27 pts/0 00:00:00 /usr/bin/python /usr/local/bin/gunicorn -w 2 gunicorn_app:app
hzliumi+ 6635 6629 0 15:27 pts/0 00:00:00 /usr/bin/python /usr/local/bin/gunicorn -w 2 gunicorn_app:app
可以看到,老的master进程(6166)及其fork出的worker子进程(pid分别是6171、6172)已经被kill掉了
references: