1、通过top 查看具体是哪个进程占用内存较多
Tasks: 65 total, 1 running, 64 sleeping, 0 stopped, 0 zombie %Cpu(s): 2.0 us, 1.0 sy, 0.0 ni, 96.3 id, 0.3 wa, 0.0 hi, 0.3 si, 0.0 st KiB Mem : 1016168 total, 63544 free, 824060 used, 128564 buff/cache KiB Swap: 2047996 total, 1612356 free, 435640 used. 49916 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 25894 www 20 0 3080304 757036 3252 S 3.3 74.5 25:55.03 java 1 root 20 0 43300 2180 1340 S 0.0 0.2 3:20.35 systemd 2 root 20 0 0 0 0 S 0.0 0.0 0:00.43 kthreadd 3 root 20 0 0 0 0 S 0.0 0.0 12:44.14 ksoftirqd/0 5 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/0:0H 7 root rt 0 0 0 0 S 0.0 0.0 0:00.00 migration/0
2、定位线程问题(通过命令查看PID 为25894 进程的线程情况),命令:# ps p 25894 -L -o pcpu,pmem,pid,tid,time,tname,cmd
# ps p 25894 -L -o pcpu,pmem,pid,tid,time,tname,cmd %CPU %MEM PID TID TIME TTY CMD 0.0 74.4 25894 25894 00:00:00 ? /App/java/jdk1.8.0_102/bin/java -Djava.util.logging.config.file=/App/tomcat_jtws-opefront/conf/logging.properties -Djava.util.logging.manager=org.ap 0.0 74.4 25894 25896 00:00:00 ? /App/java/jdk1.8.0_102/bin/java -Djava.util.logging.config.file=/App/tomcat_jtws-opefront/conf/logging.properties -Djava.util.logging.manager=org.ap 0.1 74.4 25894 25897 00:01:09 ? /App/java/jdk1.8.0_102/bin/java -Djava.util.logging.config.file=/App/tomcat_jtws-opefront/conf/logging.properties -Djava.util.logging.manager=org.ap 0.0 74.4 25894 25898 00:00:00 ? /App/java/jdk1.8.0_102/bin/java -Djava.util.logging.config.file=/App/tomcat_jtws-opefront/conf/logging.properties -Djava.util.logging.manager=org.ap 0.0 74.4 25894 25899 00:00:00 ? /App/java/jdk1.8.0_102/bin/java -Djava.util.logging.config.file=/App/tomcat_jtws-opefront/conf/logging.properties -Djava.util.logging.manager=org.ap 0.0 74.4 25894 25900 00:00:00 ? /App/java/jdk1.8.0_102/bin/java -Djava.util.logging.config.file=/App/tomcat_jtws-opefront/conf/logging.properties -Djava.util.logging.manager=org.ap 0.0 74.4 25894 25901 00:00:00 ? /App/java/jdk1.8.0_102/bin/java -Djava.util.logging.config.file=/App/tomcat_jtws-opefront/conf/logging.properties -Djava.util.logging.manager=org.ap 0.0 74.4 25894 25902 00:00:00 ? /App/java/jdk1.8.0_102/bin/java -Djava.util.logging.config.file=/App/tomcat_jtws-opefront/conf/logging.properties -Djava.util.logging.manager=org.ap 0.1 74.4 25894 25903 00:01:18 ? /App/java/jdk1.8.0_102/bin/java -Djava.util.logging.config.file=/App/tomcat_jtws-opefront/conf/logging.properties -Djava.util.logging.manager=org.ap 0.0 74.4 25894 25904 00:00:14 ? /App/java/jdk1.8.0_102/bin/java -Djava.util.logging.config.file=/App/tomcat_jtws-opefront/conf/logging.properties -Djava.util.logging.manager=org.ap .....................................以下部分省略..................................... # ps p 25894 -L -o pcpu,pmem,pid,tid,time,tname,cmd |wc -l 101
由此可以看到这个PID:25894的进程产生了很多线程。接下来就可以通过jstack查看内存使用的堆栈,具体可以执行 ps p 25894 -L -o pcpu,pmem,pid,tid,time,tname,cmd |wc -l 来查具体有多少线程.
3、将PID为25894的堆栈信息打印到jstack.log中,命令: #jstack -l 25894 > jstack.log
4、查看日志,有很多WAITING & TIMED_WAITING 状态的日志信息,具体需要研发查看一下代码的情况;
#cat jstack.log |grep -E 'WAITING|TIMED_WAITING' java.lang.Thread.State: TIMED_WAITING (parking) java.lang.Thread.State: TIMED_WAITING (parking) java.lang.Thread.State: TIMED_WAITING (parking) java.lang.Thread.State: TIMED_WAITING (on object monitor) java.lang.Thread.State: TIMED_WAITING (on object monitor) java.lang.Thread.State: TIMED_WAITING (parking) java.lang.Thread.State: TIMED_WAITING (parking) java.lang.Thread.State: TIMED_WAITING (parking) java.lang.Thread.State: TIMED_WAITING (parking) java.lang.Thread.State: TIMED_WAITING (parking) java.lang.Thread.State: TIMED_WAITING (parking) java.lang.Thread.State: TIMED_WAITING (parking) java.lang.Thread.State: WAITING (parking) java.lang.Thread.State: WAITING (parking) java.lang.Thread.State: TIMED_WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: WAITING (on object monitor) java.lang.Thread.State: TIMED_WAITING (parking) java.lang.Thread.State: TIMED_WAITING (parking) java.lang.Thread.State: TIMED_WAITING (parking) java.lang.Thread.State: TIMED_WAITING (parking) java.lang.Thread.State: TIMED_WAITING (parking) java.lang.Thread.State: TIMED_WAITING (parking) java.lang.Thread.State: TIMED_WAITING (parking) java.lang.Thread.State: TIMED_WAITING (parking) java.lang.Thread.State: TIMED_WAITING (parking) java.lang.Thread.State: TIMED_WAITING (parking) java.lang.Thread.State: TIMED_WAITING (parking) java.lang.Thread.State: TIMED_WAITING (parking) java.lang.Thread.State: TIMED_WAITING (parking) java.lang.Thread.State: TIMED_WAITING (parking) java.lang.Thread.State: TIMED_WAITING (parking) java.lang.Thread.State: TIMED_WAITING (parking) java.lang.Thread.State: TIMED_WAITING (parking) java.lang.Thread.State: TIMED_WAITING (parking) java.lang.Thread.State: TIMED_WAITING (parking) java.lang.Thread.State: TIMED_WAITING (parking) java.lang.Thread.State: TIMED_WAITING (parking)
后经研发检查代码并升级程序至新版本,问题解决。
参考链接:https://www.javatang.com/archives/2017/10/25/36441958.html#TIMED_WAITING