背景:
有一次晚上下班,发完版,刚把电脑合上走到楼下,就接到报警,说是线上有一个tomcat进程不存在了,想着以为是误报,但是还是回去看看了,发现线上确实是刚才发版的项目,进程不存在了,想了想,刚才测试已经通过了的,赶紧查看了下日志,看到如下报错:
18-Jul-2018 19:51:14.064 信息 [Thread-9] org.apache.coyote.AbstractProtocol.pause Pausing ProtocolHandler ["http-nio-8007"] 18-Jul-2018 19:51:14.115 信息 [Thread-9] org.apache.catalina.core.StandardService.stopInternal Stopping service Catalina 18-Jul-2018 19:51:14.192 警告 [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc The web application [erpQposRest] registered the JDBC driver [com.alibaba.druid.proxy.DruidDriver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered. 18-Jul-2018 19:51:14.193 警告 [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc The web application [erpQposRest] registered the JDBC driver [com.mysql.jdbc.Driver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered. 18-Jul-2018 19:51:14.193 警告 [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [erpQposRest] appears to have started a thread named [Abandoned connection cleanup thread] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread: java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135) com.mysql.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:64) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) 18-Jul-2018 19:51:14.199 信息 [Thread-9] org.apache.coyote.AbstractProtocol.stop Stopping ProtocolHandler ["http-nio-8007"] 18-Jul-2018 19:51:14.201 信息 [Thread-9] org.apache.coyote.AbstractProtocol.destroy Destroying ProtocolHandler ["http-nio-8007"]
赶紧根据关键字网上翻资料,找到问题所在,因为项目发版用的是shell脚本操作的,所有的项目启动停止都在脚本完成。问题的最终原因是因为,由于有tailf catalina.out,是为了观察项目的tomcat是否正常启动,当时没有对命令进行ctrl +c ,就直接关闭xshell所以控制台一直停留在tailf catalina.out界面。这样直接关闭xshell窗口,会导致该tomcat进程被kill掉。
具体的为什么关闭xshell会kill掉tomcat进程,后面有时间再来研究。
总结:作为运维,凡事线上操作都要小心,不要忽略任何小问题,这样才会保障线上稳定