zoukankan      html  css  js  c++  java
  • 关于springmvc的helloworld的压测报告

      都说hello world 很简单,应该能承受很大的请求压力,那么到底有多大?你知道吗?如果知道,那咱们就不继续了。如果不知道,我们来看一下!

    1. 准备工作,快速建立一个基于springmvc的helloworld

    1.1. 在pom.xml引入spring必须的包级日志组件

    <?xml version="1.0" encoding="UTF-8"?>
    <project xmlns="http://maven.apache.org/POM/4.0.0"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
        <modelVersion>4.0.0</modelVersion>
    
        <groupId>com.yougewe</groupId>
        <artifactId>mvn-local-test</artifactId>
        <version>1.0-SNAPSHOT</version>
        <packaging>war</packaging>
    
        <properties>
            <org.springframework.version>4.3.20.RELEASE</org.springframework.version>
            <freemarker.version>2.3.23</freemarker.version>
            <slf4j.version>1.7.12</slf4j.version>
            <mybatis.version>3.4.5</mybatis.version>
            <aspectj.version>1.8.13</aspectj.version>
        </properties>
    
        <dependencies>
                <!-- https://mvnrepository.com/artifact/com.alibaba/fastjson -->
                <dependency>
                    <groupId>com.alibaba</groupId>
                    <artifactId>fastjson</artifactId>
                    <version>1.2.44</version>
                </dependency>
    
            <dependency>
                <groupId>javax.servlet</groupId>
                <artifactId>javax.servlet-api</artifactId>
                <version>3.1.0</version>
            </dependency>
            <!-- https://mvnrepository.com/artifact/org.springframework/spring-core -->
            <dependency>
                <groupId>org.springframework</groupId>
                <artifactId>spring-core</artifactId>
                <version>${org.springframework.version}</version>
            </dependency>
    
            <dependency>
                <groupId>org.springframework</groupId>
                <artifactId>spring-context</artifactId>
                <version>${org.springframework.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework</groupId>
                <artifactId>spring-orm</artifactId>
                <version>${org.springframework.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework</groupId>
                <artifactId>spring-web</artifactId>
                <version>${org.springframework.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework</groupId>
                <artifactId>spring-webmvc</artifactId>
                <version>${org.springframework.version}</version>
            </dependency>
            <dependency>
                <groupId>org.springframework</groupId>
                <artifactId>spring-aop</artifactId>
                <version>${org.springframework.version}</version>
            </dependency>
            <dependency>
                <groupId>org.aspectj</groupId>
                <artifactId>aspectjweaver</artifactId>
                <version>1.6.8</version>
            </dependency>
    
                <!-- https://mvnrepository.com/artifact/junit/junit -->
                <dependency>
                    <groupId>junit</groupId>
                    <artifactId>junit</artifactId>
                    <version>4.12</version>
                    <!-- 不关注位置先 -->
                    <!--<scope>test</scope>-->
                </dependency>
                <!-- https://mvnrepository.com/artifact/org.projectlombok/lombok -->
                <dependency>
                    <groupId>org.projectlombok</groupId>
                    <artifactId>lombok</artifactId>
                    <version>1.16.18</version>
                    <scope>provided</scope>
                </dependency>
                <!-- https://mvnrepository.com/artifact/com.rabbitmq/amqp-client -->
                <dependency>
                    <groupId>com.rabbitmq</groupId>
                    <artifactId>amqp-client</artifactId>
                    <version>5.1.1</version>
                </dependency>
    
            <!-- 配合slf4j使用  -->
            <!-- 日志记录 -->
            <dependency>
                <groupId>log4j</groupId>
                <artifactId>log4j</artifactId>
                <version>1.2.17</version>
            </dependency>
            <dependency>
                <groupId>org.slf4j</groupId>
                <artifactId>jcl-over-slf4j</artifactId>
                <version>${slf4j.version}</version>
            </dependency>
            <dependency>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-api</artifactId>
                <version>${slf4j.version}</version>
            </dependency>
            <dependency>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-log4j12</artifactId>
                <version>${slf4j.version}</version>
            </dependency>
            <dependency>
                <groupId>commons-logging</groupId>
                <artifactId>commons-logging</artifactId>
                <version>1.2</version>
            </dependency>
            <!-- https://mvnrepository.com/artifact/redis.clients/jedis -->
            <dependency>
                <groupId>redis.clients</groupId>
                <artifactId>jedis</artifactId>
                <version>2.9.0</version>
            </dependency>
    
    
        </dependencies>
    
        <build>
            <!--<finalName>sjd-yzbank-api</finalName>-->
            <plugins>
    
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-resources-plugin</artifactId>
                    <version>2.6</version>
                    <configuration>
                        <encoding>UTF-8</encoding>
                    </configuration>
                </plugin>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <version>2.20.1</version>
                    <configuration>
                        <skipTests>false</skipTests>
                    </configuration>
                </plugin>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-surefire-report-plugin</artifactId>
                    <version>2.20.1</version>
                    <!--<configuration>
                        <includes>
                            <include>**/*Test.java</include>
                        </includes>
                    </configuration>-->
                </plugin>
    
                <!-- clean插件 -->
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-clean-plugin</artifactId>
                </plugin>
    
                <!-- install插件 -->
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-install-plugin</artifactId>
                </plugin>
    
                <!-- deploy插件 -->
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-deploy-plugin</artifactId>
                </plugin>
    
                <!-- dependency插件 -->
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-dependency-plugin</artifactId>
                </plugin>
    
                <plugin>
                    <!-- https://mvnrepository.com/artifact/org.apache.maven.plugins/maven-compiler-plugin -->
                    <!-- 用于更好的编译,如jdk版本太低等问题 -->
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-compiler-plugin</artifactId>
                    <version>3.6.1</version>
                    <configuration>
                        <source>1.8</source>
                        <target>1.8</target>
                    </configuration>
                </plugin>
            </plugins>
        </build>
    </project>

    1.2. 添加一个 web.xml, 只加一个dispatcherServlet 和一个字符集转换过滤器

        <filter>
            <filter-name>SpringEncodingFilter</filter-name>
            <filter-class>org.springframework.web.filter.CharacterEncodingFilter</filter-class>
            <init-param>
                <param-name>encoding</param-name>
                <param-value>UTF-8</param-value>
            </init-param>
            <init-param>
                <param-name>forceEncoding</param-name>
                <param-value>true</param-value>
            </init-param>
        </filter>
        <filter-mapping>
            <filter-name>SpringEncodingFilter</filter-name>
            <url-pattern>/*</url-pattern>
        </filter-mapping>
        <!-- 防止Spring内存溢出监听器 -->
        <listener>
            <listener-class>org.springframework.web.util.IntrospectorCleanupListener</listener-class>
        </listener>
        <!-- 如下 listener 会查找 WEB-INF/applicationContext.xml 文件 -->
        <!--<listener>
            <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
        </listener>-->
        <!-- springMVC核心配置 -->
        <servlet>
            <servlet-name>dispatcherServlet</servlet-name>
            <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
            <init-param>
                <param-name>contextConfigLocation</param-name>
                <param-value>classpath:applicationContext.xml</param-value>
            </init-param>
            <load-on-startup>1</load-on-startup>
        </servlet>
        <!-- 拦截设置 -->
        <servlet-mapping>
            <servlet-name>dispatcherServlet</servlet-name>
            <url-pattern>/*</url-pattern>
        </servlet-mapping>
        <welcome-file-list>
            <welcome-file>index.html</welcome-file>
            <welcome-file>index.jsp</welcome-file>
        </welcome-file-list>

    1.3. 添加log4j.properties日志配置文件

    log4j.rootLogger=DEBUG,console,im,logFile
    log4j.additivity.org.apache=true
    # 控制台(console)
    log4j.appender.console=org.apache.log4j.ConsoleAppender
    log4j.appender.console.Threshold=DEBUG
    log4j.appender.console.ImmediateFlush=true
    log4j.appender.console.Target=System.out
    log4j.appender.console.layout=org.apache.log4j.PatternLayout
    log4j.appender.console.layout.ConversionPattern=%d(%r) [%t] %-5p %l: %m %x %n
    # 日志文件(logFile)
    log4j.appender.logFile=org.apache.log4j.FileAppender
    log4j.appender.logFile.Threshold=DEBUG
    log4j.appender.logFile.ImmediateFlush=true
    log4j.appender.logFile.Append=true
    log4j.appender.logFile.File=D:/logs/log.log4j
    log4j.appender.logFile.layout=org.apache.log4j.PatternLayout
    log4j.appender.logFile.layout.ConversionPattern=%d(%r) [%t] %-5p %l: %m %x %n
    # 回滚文件(rollingFile)
    log4j.appender.rollingFile=org.apache.log4j.RollingFileAppender
    log4j.appender.rollingFile.Threshold=DEBUG
    log4j.appender.rollingFile.ImmediateFlush=true
    log4j.appender.rollingFile.Append=true
    log4j.appender.rollingFile.File=D:/logs/log.log4j
    log4j.appender.rollingFile.MaxFileSize=200KB
    log4j.appender.rollingFile.MaxBackupIndex=50
    log4j.appender.rollingFile.layout=org.apache.log4j.PatternLayout
    log4j.appender.rollingFile.layout.ConversionPattern=%d(%r) [%t] %-5p %l: %m %x %n

    1.4. 添加一个HelloController, 返回一个 字符串

    @RestController
    @RequestMapping("/hello")
    public class HelloController {
    
        @RequestMapping(value = "/world", method = {RequestMethod.GET, RequestMethod.POST}, produces = "application/json")
        @ResponseBody
        public Object world(@ModelAttribute UserInfo info) {
            return "hello world!";
        }
    }

      好了,一切准备就绪!是时候让我们来看一下它的能力如何了!

    测试工具: jmeter + 台式测试机一台

    前提1:
      机器配置: 4c8g 笔记本
      网卡: 高通 QCA9377, 共享带宽: 100M, 网络测速: 下载:3.5MB/s 上传:475KB/s
      web容器: tomcat7, 运行模式: apr(apache portable runtime)

    压测过程如下:

      并发100-5组连续请求, TPS: 146.2, error: 0, 平均响应时间: 0.381s, 最大响应时间: 2.44s

      server端cpu有一瞬间的飙高,内存几乎无变化!

      下面,按照规律,翻倍并发,200-5组连续请求!看下数据!

      TPS增加了,为275;平均响应时间慢了点,0.429;没有 error。

      再翻倍并发量:400-5组连续请求:

      TPS下降了,为208;平均响应时间翻番,1.1秒;不过幸好还是没有error;

      再翻倍并发量:800-5组连续请求:

      TPS再次下降,为144;平均响应时间再翻番,3.6秒,这在生产环境已经不符合要求了!error仍为0;

       再翻倍: 1600-5组: server cpu 一直高居不下,直接jvm宕机。

     

      注意,此时已经有error出现了,1.09%的错误率! TPS继续下降: 93.4,平均响应时间继续翻倍:8.9秒;

      综上,springmvc的helloworld 能力差不多也就在1600了,因为已经有错误出现,在实际生产中已经完全不能接受了!

      不过,我还是想看一下server到底能承受多大压力,也就是jvm完全宕机!

      压到3000并发-5组:

      看起来还能响应,其实再server端,jvm已经挂掉了!所以结论是,tomcat7(apr模式)是扛不住3000并发的!

    jvm 宕机日志样例如下:

    十二月 12, 2018 11:54:25 上午 org.apache.tomcat.util.net.AprEndpoint$Poller run
    警告: Unexpected poller error
    java.lang.NullPointerException
        at org.apache.tomcat.util.net.AprEndpoint$AprSocketWrapper.access$400(AprEndpoint.java:2537)
        at org.apache.tomcat.util.net.AprEndpoint$Poller.run(AprEndpoint.java:1887)
        at java.lang.Thread.run(Thread.java:745)
    
    十二月 12, 2018 11:54:25 上午 org.apache.tomcat.util.net.AprEndpoint$Poller run
    警告: Unexpected poller error
    java.lang.NullPointerException
        at org.apache.tomcat.util.net.AprEndpoint$AprSocketWrapper.access$400(AprEndpoint.java:2537)
        at org.apache.tomcat.util.net.AprEndpoint$Poller.run(AprEndpoint.java:1887)
        at java.lang.Thread.run(Thread.java:745)
    
    #
    # A fatal error has been detected by the Java Runtime Environment:
    #
    #  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000000180008005, pid=13444, tid=0x0000000000002e34
    #
    # JRE version: Java(TM) SE Runtime Environment (8.0_101-b13) (build 1.8.0_101-b13)
    # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.101-b13 mixed mode windows-amd64 compressed oops)
    # Problematic frame:
    # C  [tcnative-1.dll+0x8005]
    #
    # Failed to write core dump. Minidumps are not enabled by default on client versions of Windows
    #
    # An error report file with more information is saved as:
    # D:servers	omcat-7.0.70inhs_err_pid13444.log
    #
    # If you would like to submit a bug report, please visit:
    #   http://bugreport.java.com/bugreport/crash.jsp
    # The crash happened outside the Java Virtual Machine in native code.
    # See problematic frame for where to report the bug.
    #
    Disconnected from the target VM, address: '127.0.0.1:58835', transport: 'socket'
    Disconnected from server

    其中,也指示了错误记录文件:hs_err_pid13444.log,其中大概是一些线程、栈、内存、cpu的总结内容!

      大概就是说, 

    #
    # A fatal error has been detected by the Java Runtime Environment:
    #
    #  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000000180008005, pid=13444, tid=0x0000000000002e34
    #
    # JRE version: Java(TM) SE Runtime Environment (8.0_101-b13) (build 1.8.0_101-b13)
    # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.101-b13 mixed mode windows-amd64 compressed oops)
    # Problematic frame:
    # C  [tcnative-1.dll+0x8005]
    #
    # Failed to write core dump. Minidumps are not enabled by default on client versions of Windows
    #
    # If you would like to submit a bug report, please visit:
    #   http://bugreport.java.com/bugreport/crash.jsp
    # The crash happened outside the Java Virtual Machine in native code.
    # See problematic frame for where to report the bug.
    #
    
    ---------------  T H R E A D  ---------------
    
    Current thread (0x000000001a1e5800):  JavaThread "http-apr-8080-Poller" daemon [_thread_in_native, id=11828, stack(0x000000001c420000,0x000000001c520000)]
    
    siginfo: ExceptionCode=0xc0000005, writing address 0x0000000000004774
    
    Registers:
    RAX=0x0000000036a64ec0, RBX=0x00000000356a2d60, RCX=0x0000000000004774, RDX=0x0000000036a64df8
    RSP=0x000000001c51f010, RBP=0x00000000000025e0, RSI=0x000000000000025e, RDI=0x00000000003ae258
    R8 =0x0000000000000000, R9 =0x00000000003960c0, R10=0x0000000000004000, R11=0x0000000036a64ec0
    R12=0x00000000000025e8, R13=0x00057ccb27bc9160, R14=0x00000000003ae2a8, R15=0x0000000000000001
    RIP=0x0000000180008005, EFLAGS=0x0000000000010246
    
    Top of Stack: (sp=0x000000001c51f010)
    0x000000001c51f010:   000000000000025e 00000000003ae258
    0x000000001c51f020:   000000001c51f088 000000001c51f090
    0x000000001c51f030:   000000001c51f500 0000000088a1ccb0
    0x000000001c51f040:   000000001a1e5800 000000001c51f0e0
    0x000000001c51f050:   0000000089172b08 0000000000000000
    0x000000001c51f060:   0000000000000001 000000001c51f0f0

     戳详情:

    #
    # A fatal error has been detected by the Java Runtime Environment:
    #
    #  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000000180008005, pid=13444, tid=0x0000000000002e34
    #
    # JRE version: Java(TM) SE Runtime Environment (8.0_101-b13) (build 1.8.0_101-b13)
    # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.101-b13 mixed mode windows-amd64 compressed oops)
    # Problematic frame:
    # C  [tcnative-1.dll+0x8005]
    #
    # Failed to write core dump. Minidumps are not enabled by default on client versions of Windows
    #
    # If you would like to submit a bug report, please visit:
    #   http://bugreport.java.com/bugreport/crash.jsp
    # The crash happened outside the Java Virtual Machine in native code.
    # See problematic frame for where to report the bug.
    #
    
    ---------------  T H R E A D  ---------------
    
    Current thread (0x000000001a1e5800):  JavaThread "http-apr-8080-Poller" daemon [_thread_in_native, id=11828, stack(0x000000001c420000,0x000000001c520000)]
    
    siginfo: ExceptionCode=0xc0000005, writing address 0x0000000000004774
    
    Registers:
    RAX=0x0000000036a64ec0, RBX=0x00000000356a2d60, RCX=0x0000000000004774, RDX=0x0000000036a64df8
    RSP=0x000000001c51f010, RBP=0x00000000000025e0, RSI=0x000000000000025e, RDI=0x00000000003ae258
    R8 =0x0000000000000000, R9 =0x00000000003960c0, R10=0x0000000000004000, R11=0x0000000036a64ec0
    R12=0x00000000000025e8, R13=0x00057ccb27bc9160, R14=0x00000000003ae2a8, R15=0x0000000000000001
    RIP=0x0000000180008005, EFLAGS=0x0000000000010246
    
    Top of Stack: (sp=0x000000001c51f010)
    0x000000001c51f010:   000000000000025e 00000000003ae258
    0x000000001c51f020:   000000001c51f088 000000001c51f090
    0x000000001c51f030:   000000001c51f500 0000000088a1ccb0
    0x000000001c51f040:   000000001a1e5800 000000001c51f0e0
    0x000000001c51f050:   0000000089172b08 0000000000000000
    0x000000001c51f060:   0000000000000001 000000001c51f0f0
    0x000000001c51f070:   0000000000000000 0000000002d9f968
    0x000000001c51f080:   000000001a1e59f8 00000000000003e4
    0x000000001c51f090:   00000000003aac38 0000000089033de8
    0x000000001c51f0a0:   000000001c51f0c0 0000000000000001
    0x000000001c51f0b0:   36a8db6836187b68 00000000ede7bdd0
    0x000000001c51f0c0:   000000008912ea98 0000000052cd606f
    0x000000001c51f0d0:   01d491ce629c2dc0 000000008912ce88
    0x000000001c51f0e0:   0000000088a1ccb0 0000000000000202
    0x000000001c51f0f0:   000000001c51f500 000000000389d324
    0x000000001c51f100:   01d491ce629c2dc0 0000000052c569bd 
    
    Instructions: (pc=0x0000000180008005)
    0x0000000180007fe5:   48 8b 94 24 80 00 00 00 48 8b 4f 10 e8 ca 8a 01
    0x0000000180007ff5:   00 4c 8b 5b 38 49 8b 13 48 8b 43 38 48 8b 48 08
    0x0000000180008005:   48 89 11 48 8b 43 38 48 8b 50 08 48 8b 43 38 48
    0x0000000180008015:   8b 08 48 89 51 08 48 8b 43 38 4c 89 30 48 8b 4f 
    
    
    Register to memory mapping:
    
    RAX=0x0000000036a64ec0 is an unknown value
    RBX=0x00000000356a2d60 is an unknown value
    RCX=0x0000000000004774 is an unknown value
    RDX=0x0000000036a64df8 is an unknown value
    RSP=0x000000001c51f010 is pointing into the stack for thread: 0x000000001a1e5800
    RBP=0x00000000000025e0 is an unknown value
    RSI=0x000000000000025e is an unknown value
    RDI=0x00000000003ae258 is an unknown value
    R8 =0x0000000000000000 is an unknown value
    R9 =0x00000000003960c0 is an unknown value
    R10=0x0000000000004000 is an unknown value
    R11=0x0000000036a64ec0 is an unknown value
    R12=0x00000000000025e8 is an unknown value
    R13=0x00057ccb27bc9160 is an unknown value
    R14=0x00000000003ae2a8 is an unknown value
    R15=0x0000000000000001 is an unknown value
    
    
    Stack: [0x000000001c420000,0x000000001c520000],  sp=0x000000001c51f010,  free space=1020k
    Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
    C  [tcnative-1.dll+0x8005]
    C  0x0000000002d9f968
    
    Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
    J 3241  org.apache.tomcat.jni.Poll.poll(JJ[JZ)I (0 bytes) @ 0x0000000002d9f8e2 [0x0000000002d9f880+0x62]
    J 6143% C1 org.apache.tomcat.util.net.AprEndpoint$Poller.run()V (2241 bytes) @ 0x000000000389d324 [0x0000000003899ec0+0x3464]
    j  java.lang.Thread.run()V+11
    v  ~StubRoutines::call_stub
    
    ---------------  P R O C E S S  ---------------
    
    Java Threads: ( => current thread )
      0x0000000020785000 JavaThread "http-apr-8080-exec-770" daemon [_thread_blocked, id=14488, stack(0x0000000035820000,0x0000000035920000)]
      0x0000000020784800 JavaThread "http-apr-8080-exec-769" daemon [_thread_blocked, id=5880, stack(0x0000000035930000,0x0000000035a30000)]
      0x0000000020783800 JavaThread "http-apr-8080-exec-768" daemon [_thread_blocked, id=8632, stack(0x0000000034db0000,0x0000000034eb0000)]
      0x0000000020783000 JavaThread "http-apr-8080-exec-767" daemon [_thread_blocked, id=11628, stack(0x0000000034500000,0x0000000034600000)]
      0x0000000020782000 JavaThread "http-apr-8080-exec-766" daemon [_thread_blocked, id=13600, stack(0x0000000034bd0000,0x0000000034cd0000)]
      0x0000000020781800 JavaThread "http-apr-8080-exec-765" daemon [_thread_blocked, id=11512, stack(0x00000000348b0000,0x00000000349b0000)]
      0x0000000020780800 JavaThread "http-apr-8080-exec-764" daemon [_thread_blocked, id=4848, stack(0x0000000032980000,0x0000000032a80000)]
      0x0000000020780000 JavaThread "http-apr-8080-exec-763" daemon [_thread_blocked, id=10048, stack(0x0000000033ef0000,0x0000000033ff0000)]
      0x000000002077f000 JavaThread "http-apr-8080-exec-762" daemon [_thread_blocked, id=12072, stack(0x00000000347a0000,0x00000000348a0000)]
      0x000000002395f000 JavaThread "http-apr-8080-exec-761" daemon [_thread_blocked, id=10588, stack(0x0000000034650000,0x0000000034750000)]
      0x000000002395e000 JavaThread "http-apr-8080-exec-760" daemon [_thread_blocked, id=14920, stack(0x00000000343e0000,0x00000000344e0000)]
      0x000000002395d800 JavaThread "http-apr-8080-exec-759" daemon [_thread_blocked, id=12232, stack(0x00000000341f0000,0x00000000342f0000)]
      0x000000002395c800 JavaThread "http-apr-8080-exec-758" daemon [_thread_blocked, id=9540, stack(0x0000000034000000,0x0000000034100000)]
      0x000000002395c000 JavaThread "http-apr-8080-exec-757" daemon [_thread_blocked, id=13092, stack(0x0000000031d30000,0x0000000031e30000)]
      0x000000002395b000 JavaThread "http-apr-8080-exec-756" daemon [_thread_blocked, id=4624, stack(0x0000000033c50000,0x0000000033d50000)]
      0x000000002395a800 JavaThread "http-apr-8080-exec-755" daemon [_thread_blocked, id=9964, stack(0x0000000033dd0000,0x0000000033ed0000)]
      0x0000000023959800 JavaThread "http-apr-8080-exec-754" daemon [_thread_blocked, id=9704, stack(0x0000000033b30000,0x0000000033c30000)]
      0x0000000023959000 JavaThread "http-apr-8080-exec-753" daemon [_thread_blocked, id=7484, stack(0x0000000033940000,0x0000000033a40000)]
      0x0000000023958000 JavaThread "http-apr-8080-exec-752" daemon [_thread_blocked, id=11976, stack(0x0000000033660000,0x0000000033760000)]
      0x0000000023957800 JavaThread "http-apr-8080-exec-751" daemon [_thread_blocked, id=15160, stack(0x00000000337c0000,0x00000000338c0000)]
      0x0000000023956800 JavaThread "http-apr-8080-exec-750" daemon [_thread_blocked, id=1320, stack(0x00000000332d0000,0x00000000333d0000)]
      0x0000000023956000 JavaThread "http-apr-8080-exec-749" daemon [_thread_blocked, id=8720, stack(0x0000000033560000,0x0000000033660000)]
      0x0000000023955000 JavaThread "http-apr-8080-exec-748" daemon [_thread_blocked, id=14136, stack(0x0000000033410000,0x0000000033510000)]
      0x0000000023954800 JavaThread "http-apr-8080-exec-747" daemon [_thread_blocked, id=10148, stack(0x00000000321c0000,0x00000000322c0000)]
      0x0000000023953800 JavaThread "http-apr-8080-exec-746" daemon [_thread_blocked, id=7368, stack(0x00000000331b0000,0x00000000332b0000)]
      0x0000000023953000 JavaThread "http-apr-8080-exec-745" daemon [_thread_blocked, id=11400, stack(0x0000000032fd0000,0x00000000330d0000)]
      0x0000000023952000 JavaThread "http-apr-8080-exec-744" daemon [_thread_blocked, id=10248, stack(0x0000000032de0000,0x0000000032ee0000)]
      0x0000000023951800 JavaThread "http-apr-8080-exec-743" daemon [_thread_blocked, id=5200, stack(0x0000000032c50000,0x0000000032d50000)]
      0x0000000023950800 JavaThread "http-apr-8080-exec-740" daemon [_thread_blocked, id=12396, stack(0x0000000032a80000,0x0000000032b80000)]
      0x0000000023950000 JavaThread "http-apr-8080-exec-742" daemon [_thread_blocked, id=6516, stack(0x0000000032880000,0x0000000032980000)]
      0x000000001e40b000 JavaThread "http-apr-8080-exec-741" daemon [_thread_blocked, id=8412, stack(0x0000000032710000,0x0000000032810000)]
      0x000000001e40a000 JavaThread "http-apr-8080-exec-739" daemon [_thread_blocked, id=8996, stack(0x00000000325e0000,0x00000000326e0000)]
      0x000000001e409800 JavaThread "http-apr-8080-exec-738" daemon [_thread_blocked, id=7712, stack(0x0000000031f40000,0x0000000032040000)]
      0x000000001e408800 JavaThread "http-apr-8080-exec-737" daemon [_thread_blocked, id=12004, stack(0x000000002ed90000,0x000000002ee90000)]
      0x000000001e408000 JavaThread "http-apr-8080-exec-736" daemon [_thread_blocked, id=9200, stack(0x0000000032470000,0x0000000032570000)]
      0x000000001e407000 JavaThread "http-apr-8080-exec-735" daemon [_thread_blocked, id=12752, stack(0x00000000322d0000,0x00000000323d0000)]
      0x000000001e406800 JavaThread "http-apr-8080-exec-734" daemon [_thread_blocked, id=12632, stack(0x0000000032040000,0x0000000032140000)]
      0x000000001e405800 JavaThread "http-apr-8080-exec-733" daemon [_thread_blocked, id=11596, stack(0x000000002f860000,0x000000002f960000)]
      0x000000001e405000 JavaThread "http-apr-8080-exec-732" daemon [_thread_blocked, id=4564, stack(0x00000000309f0000,0x0000000030af0000)]
      0x000000001e404000 JavaThread "http-apr-8080-exec-730" daemon [_thread_blocked, id=14540, stack(0x0000000031e30000,0x0000000031f30000)]
      0x000000001ac6b000 JavaThread "http-apr-8080-exec-731" daemon [_thread_blocked, id=2600, stack(0x0000000030e20000,0x0000000030f20000)]
      0x000000001ac6a000 JavaThread "http-apr-8080-exec-729" daemon [_thread_blocked, id=13624, stack(0x0000000031c20000,0x0000000031d20000)]
      0x000000001ac69800 JavaThread "http-apr-8080-exec-728" daemon [_thread_blocked, id=6428, stack(0x0000000031a60000,0x0000000031b60000)]
      0x000000001ac68800 JavaThread "http-apr-8080-exec-727" daemon [_thread_blocked, id=6384, stack(0x0000000025860000,0x0000000025960000)]
      0x000000001ac68000 JavaThread "http-apr-8080-exec-726" daemon [_thread_blocked, id=7200, stack(0x0000000031960000,0x0000000031a60000)]
      0x000000001ac67000 JavaThread "http-apr-8080-exec-725" daemon [_thread_blocked, id=14740, stack(0x00000000314c0000,0x00000000315c0000)]
      0x000000001ac66800 JavaThread "http-apr-8080-exec-724" daemon [_thread_blocked, id=12892, stack(0x00000000304f0000,0x00000000305f0000)]
      0x000000001ac65800 JavaThread "http-apr-8080-exec-723" daemon [_thread_blocked, id=7992, stack(0x0000000031780000,0x0000000031880000)]
      0x000000001ac65000 JavaThread "http-apr-8080-exec-722" daemon [_thread_blocked, id=4788, stack(0x00000000315e0000,0x00000000316e0000)]
      0x000000001ac64000 JavaThread "http-apr-8080-exec-721" daemon [_thread_blocked, id=10808, stack(0x00000000313b0000,0x00000000314b0000)]
      0x000000001b040000 JavaThread "http-apr-8080-exec-720" daemon [_thread_blocked, id=14832, stack(0x0000000030c00000,0x0000000030d00000)]
      0x000000001b03f800 JavaThread "http-apr-8080-exec-719" daemon [_thread_blocked, id=13892, stack(0x00000000311d0000,0x00000000312d0000)]
      0x000000001b03e800 JavaThread "http-apr-8080-exec-718" daemon [_thread_blocked, id=10408, stack(0x0000000030f30000,0x0000000031030000)]
      0x000000001b03e000 JavaThread "http-apr-8080-exec-717" daemon [_thread_blocked, id=7220, stack(0x0000000030d10000,0x0000000030e10000)]
      0x000000001b03d000 JavaThread "http-apr-8080-exec-716" daemon [_thread_blocked, id=14512, stack(0x0000000030af0000,0x0000000030bf0000)]
      0x000000001b03c800 JavaThread "http-apr-8080-exec-715" daemon [_thread_blocked, id=3068, stack(0x0000000030740000,0x0000000030840000)]
      0x000000001b03b800 JavaThread "http-apr-8080-exec-714" daemon [_thread_blocked, id=1004, stack(0x00000000305f0000,0x00000000306f0000)]
      0x000000001b03b000 JavaThread "http-apr-8080-exec-713" daemon [_thread_blocked, id=11156, stack(0x0000000029be0000,0x0000000029ce0000)]
      0x000000001b03a000 JavaThread "http-apr-8080-exec-712" daemon [_thread_blocked, id=6740, stack(0x00000000303f0000,0x00000000304f0000)]
      0x000000001b039800 JavaThread "http-apr-8080-exec-709" daemon [_thread_blocked, id=11852, stack(0x0000000030250000,0x0000000030350000)]
      0x000000002167e800 JavaThread "http-apr-8080-exec-711" daemon [_thread_blocked, id=13836, stack(0x00000000300f0000,0x00000000301f0000)]
      0x000000002167e000 JavaThread "http-apr-8080-exec-710" daemon [_thread_blocked, id=9528, stack(0x000000002fe40000,0x000000002ff40000)]
      0x000000002167d000 JavaThread "http-apr-8080-exec-708" daemon [_thread_blocked, id=9636, stack(0x000000002ff40000,0x0000000030040000)]
      0x000000002167c800 JavaThread "http-apr-8080-exec-707" daemon [_thread_blocked, id=7904, stack(0x000000002fc90000,0x000000002fd90000)]
      0x000000002167b800 JavaThread "http-apr-8080-exec-706" daemon [_thread_blocked, id=13048, stack(0x000000002f630000,0x000000002f730000)]
      0x000000002167b000 JavaThread "http-apr-8080-exec-705" daemon [_thread_blocked, id=9536, stack(0x000000002faf0000,0x000000002fbf0000)]
      0x000000002167a000 JavaThread "http-apr-8080-exec-703" daemon [_thread_blocked, id=3868, stack(0x000000002f960000,0x000000002fa60000)]
      0x0000000021679800 JavaThread "http-apr-8080-exec-704" daemon [_thread_blocked, id=13344, stack(0x000000002f760000,0x000000002f860000)]
      0x0000000021678800 JavaThread "http-apr-8080-exec-702" daemon [_thread_blocked, id=6812, stack(0x000000002f520000,0x000000002f620000)]
      0x0000000021678000 JavaThread "http-apr-8080-exec-701" daemon [_thread_blocked, id=6804, stack(0x000000002f360000,0x000000002f460000)]
      0x000000001bb23800 JavaThread "http-apr-8080-exec-700" daemon [_thread_blocked, id=10328, stack(0x000000002cc30000,0x000000002cd30000)]
      0x000000001bb22800 JavaThread "http-apr-8080-exec-699" daemon [_thread_blocked, id=14236, stack(0x000000002f210000,0x000000002f310000)]
      0x000000001bb22000 JavaThread "http-apr-8080-exec-698" daemon [_thread_blocked, id=3444, stack(0x000000002f080000,0x000000002f180000)]
      0x000000001bb21000 JavaThread "http-apr-8080-exec-697" daemon [_thread_blocked, id=2548, stack(0x000000002ee90000,0x000000002ef90000)]
      0x000000001bb20800 JavaThread "http-apr-8080-exec-696" daemon [_thread_blocked, id=13268, stack(0x000000002ec90000,0x000000002ed90000)]
      0x000000001bb1f800 JavaThread "http-apr-8080-exec-695" daemon [_thread_blocked, id=14668, stack(0x000000002eb50000,0x000000002ec50000)]
      0x000000001bb1f000 JavaThread "http-apr-8080-exec-694" daemon [_thread_blocked, id=10132, stack(0x000000002e2d0000,0x000000002e3d0000)]
      0x000000001bb1e000 JavaThread "http-apr-8080-exec-689" daemon [_thread_blocked, id=10860, stack(0x000000002e980000,0x000000002ea80000)]
      0x000000001bb1d800 JavaThread "http-apr-8080-exec-693" daemon [_thread_blocked, id=8664, stack(0x000000002e840000,0x000000002e940000)]
      0x000000001bb1c800 JavaThread "http-apr-8080-exec-692" daemon [_thread_blocked, id=3928, stack(0x000000002e730000,0x000000002e830000)]
      0x0000000021666800 JavaThread "http-apr-8080-exec-691" daemon [_thread_blocked, id=6380, stack(0x000000002e610000,0x000000002e710000)]
      0x0000000021665800 JavaThread "http-apr-8080-exec-690" daemon [_thread_blocked, id=8848, stack(0x000000002db20000,0x000000002dc20000)]
      0x0000000021665000 JavaThread "http-apr-8080-exec-688" daemon [_thread_blocked, id=7752, stack(0x000000002e1b0000,0x000000002e2b0000)]
      0x0000000021664000 JavaThread "http-apr-8080-exec-687" daemon [_thread_blocked, id=11700, stack(0x000000002dff0000,0x000000002e0f0000)]
      0x0000000021663800 JavaThread "http-apr-8080-exec-686" daemon [_thread_blocked, id=12084, stack(0x000000002de20000,0x000000002df20000)]
      0x0000000021662800 JavaThread "http-apr-8080-exec-685" daemon [_thread_blocked, id=13356, stack(0x000000002d6c0000,0x000000002d7c0000)]
      0x0000000021662000 JavaThread "http-apr-8080-exec-684" daemon [_thread_blocked, id=13484, stack(0x000000002da10000,0x000000002db10000)]
      0x0000000021661000 JavaThread "http-apr-8080-exec-683" daemon [_thread_blocked, id=8748, stack(0x000000002d900000,0x000000002da00000)]
      0x0000000021660800 JavaThread "http-apr-8080-exec-682" daemon [_thread_blocked, id=13808, stack(0x000000002d7c0000,0x000000002d8c0000)]
      0x000000002165f800 JavaThread "http-apr-8080-exec-681" daemon [_thread_blocked, id=8964, stack(0x000000002d580000,0x000000002d680000)]
      0x000000001a1e4800 JavaThread "http-apr-8080-exec-680" daemon [_thread_blocked, id=13084, stack(0x000000002d390000,0x000000002d490000)]
      0x00000000242ec000 JavaThread "http-apr-8080-exec-679" daemon [_thread_blocked, id=11716, stack(0x000000002c800000,0x000000002c900000)]
      0x00000000242ea800 JavaThread "http-apr-8080-exec-678" daemon [_thread_blocked, id=5008, stack(0x000000002a950000,0x000000002aa50000)]
      0x00000000242e7800 JavaThread "http-apr-8080-exec-677" daemon [_thread_blocked, id=3672, stack(0x000000002d200000,0x000000002d300000)]
      0x00000000242e5800 JavaThread "http-apr-8080-exec-676" daemon [_thread_blocked, id=6772, stack(0x000000002d090000,0x000000002d190000)]
      0x00000000242eb800 JavaThread "http-apr-8080-exec-675" daemon [_thread_blocked, id=8288, stack(0x000000002cf20000,0x000000002d020000)]
      0x00000000242e9000 JavaThread "http-apr-8080-exec-674" daemon [_thread_blocked, id=7048, stack(0x000000002ca30000,0x000000002cb30000)]
      0x00000000242ea000 JavaThread "http-apr-8080-exec-673" daemon [_thread_blocked, id=13932, stack(0x000000002cd40000,0x000000002ce40000)]
      0x00000000242e8800 JavaThread "http-apr-8080-exec-672" daemon [_thread_blocked, id=13524, stack(0x000000002c910000,0x000000002ca10000)]
      0x000000001a4f9000 JavaThread "http-apr-8080-exec-671" daemon [_thread_blocked, id=8072, stack(0x000000002c6b0000,0x000000002c7b0000)]
      0x000000001a4f6800 JavaThread "http-apr-8080-exec-670" daemon [_thread_blocked, id=9524, stack(0x000000002c570000,0x000000002c670000)]
      0x000000001a4fa800 JavaThread "http-apr-8080-exec-669" daemon [_thread_blocked, id=10388, stack(0x000000002c3d0000,0x000000002c4d0000)]
      0x000000001a4f9800 JavaThread "http-apr-8080-exec-668" daemon [_thread_blocked, id=7176, stack(0x000000002c1e0000,0x000000002c2e0000)]
      0x000000001a4fb000 JavaThread "http-apr-8080-exec-667" daemon [_thread_blocked, id=10288, stack(0x000000002bd40000,0x000000002be40000)]
      0x000000001a4f7800 JavaThread "http-apr-8080-exec-666" daemon [_thread_blocked, id=7480, stack(0x000000002c050000,0x000000002c150000)]
      0x000000001a4fc000 JavaThread "http-apr-8080-exec-665" daemon [_thread_blocked, id=2744, stack(0x000000002be90000,0x000000002bf90000)]
      0x000000001a4fc800 JavaThread "http-apr-8080-exec-664" daemon [_thread_blocked, id=6640, stack(0x000000002bc30000,0x000000002bd30000)]
      0x000000001d735800 JavaThread "http-apr-8080-exec-663" daemon [_thread_blocked, id=5784, stack(0x000000002b7f0000,0x000000002b8f0000)]
      0x000000001d736000 JavaThread "http-apr-8080-exec-662" daemon [_thread_blocked, id=10984, stack(0x000000002baf0000,0x000000002bbf0000)]
      0x000000001d733000 JavaThread "http-apr-8080-exec-661" daemon [_thread_blocked, id=5592, stack(0x000000002b930000,0x000000002ba30000)]
      0x000000001d731800 JavaThread "http-apr-8080-exec-660" daemon [_thread_blocked, id=5832, stack(0x000000002b680000,0x000000002b780000)]
      0x000000001d737000 JavaThread "http-apr-8080-exec-659" daemon [_thread_blocked, id=15260, stack(0x000000002a5c0000,0x000000002a6c0000)]
      0x000000001d734000 JavaThread "http-apr-8080-exec-658" daemon [_thread_blocked, id=12784, stack(0x000000002b4e0000,0x000000002b5e0000)]
      0x000000001d734800 JavaThread "http-apr-8080-exec-657" daemon [_thread_blocked, id=2488, stack(0x000000002b280000,0x000000002b380000)]
      0x000000001d737800 JavaThread "http-apr-8080-exec-656" daemon [_thread_blocked, id=12492, stack(0x000000002b120000,0x000000002b220000)]
      0x000000001d738800 JavaThread "http-apr-8080-exec-655" daemon [_thread_blocked, id=11388, stack(0x000000002a090000,0x000000002a190000)]
      0x000000001ed5f000 JavaThread "http-apr-8080-exec-654" daemon [_thread_blocked, id=4904, stack(0x000000002afe0000,0x000000002b0e0000)]
      0x000000001ed5b000 JavaThread "http-apr-8080-exec-653" daemon [_thread_blocked, id=6436, stack(0x000000002aee0000,0x000000002afe0000)]
      0x000000001ed5e000 JavaThread "http-apr-8080-exec-652" daemon [_thread_blocked, id=11532, stack(0x000000002ad50000,0x000000002ae50000)]
      0x000000001ed5c800 JavaThread "http-apr-8080-exec-651" daemon [_thread_blocked, id=12584, stack(0x000000002ac10000,0x000000002ad10000)]
      0x000000001ed61000 JavaThread "http-apr-8080-exec-650" daemon [_thread_blocked, id=13068, stack(0x000000002aa50000,0x000000002ab50000)]
      0x000000001ed5f800 JavaThread "http-apr-8080-exec-649" daemon [_thread_blocked, id=2000, stack(0x000000002a850000,0x000000002a950000)]
      0x000000001ed5b800 JavaThread "http-apr-8080-exec-648" daemon [_thread_blocked, id=7348, stack(0x000000002a6f0000,0x000000002a7f0000)]
      0x000000001ed60800 JavaThread "http-apr-8080-exec-647" daemon [_thread_blocked, id=6920, stack(0x000000002a1a0000,0x000000002a2a0000)]
      0x000000001ed5a000 JavaThread "http-apr-8080-exec-646" daemon [_thread_blocked, id=9844, stack(0x0000000029520000,0x0000000029620000)]
      0x000000001a0eb800 JavaThread "http-apr-8080-exec-645" daemon [_thread_blocked, id=13168, stack(0x0000000029e30000,0x0000000029f30000)]
      0x000000001a0ef800 JavaThread "http-apr-8080-exec-644" daemon [_thread_blocked, id=1520, stack(0x0000000029f30000,0x000000002a030000)]
      0x000000001a0f1000 JavaThread "http-apr-8080-exec-643" daemon [_thread_blocked, id=1268, stack(0x0000000029ce0000,0x0000000029de0000)]
      0x000000001a0f1800 JavaThread "http-apr-8080-exec-642" daemon [_thread_blocked, id=2644, stack(0x0000000029ae0000,0x0000000029be0000)]
      0x000000001a0f0000 JavaThread "http-apr-8080-exec-641" daemon [_thread_blocked, id=8884, stack(0x0000000029950000,0x0000000029a50000)]
      0x000000001a0ee000 JavaThread "http-apr-8080-exec-640" daemon [_thread_blocked, id=10704, stack(0x0000000029760000,0x0000000029860000)]
      0x000000001a0ed000 JavaThread "http-apr-8080-exec-639" daemon [_thread_blocked, id=11100, stack(0x0000000028e60000,0x0000000028f60000)]
      0x000000001a0ee800 JavaThread "http-apr-8080-exec-625" daemon [_thread_blocked, id=7720, stack(0x0000000029620000,0x0000000029720000)]
      0x000000001a0ec000 JavaThread "http-apr-8080-exec-638" daemon [_thread_blocked, id=1264, stack(0x0000000029410000,0x0000000029510000)]
      0x000000001a5a3000 JavaThread "http-apr-8080-exec-637" daemon [_thread_blocked, id=13576, stack(0x0000000028620000,0x0000000028720000)]
      0x000000001a5a2800 JavaThread "http-apr-8080-exec-636" daemon [_thread_blocked, id=4856, stack(0x0000000029280000,0x0000000029380000)]
      0x000000001a5a6000 JavaThread "http-apr-8080-exec-635" daemon [_thread_blocked, id=4580, stack(0x0000000029140000,0x0000000029240000)]
      0x000000001a5a7800 JavaThread "http-apr-8080-exec-634" daemon [_thread_blocked, id=11844, stack(0x0000000028f90000,0x0000000029090000)]
      0x000000001a5a4800 JavaThread "http-apr-8080-exec-633" daemon [_thread_blocked, id=11368, stack(0x0000000028ba0000,0x0000000028ca0000)]
      0x000000001a5a7000 JavaThread "http-apr-8080-exec-632" daemon [_thread_blocked, id=8928, stack(0x0000000028d10000,0x0000000028e10000)]
      0x000000001a5a9000 JavaThread "http-apr-8080-exec-631" daemon [_thread_blocked, id=6272, stack(0x0000000028960000,0x0000000028a60000)]
      0x000000001a5a4000 JavaThread "http-apr-8080-exec-630" daemon [_thread_blocked, id=7332, stack(0x0000000028830000,0x0000000028930000)]
      0x000000001a5a8800 JavaThread "http-apr-8080-exec-629" daemon [_thread_blocked, id=7672, stack(0x0000000028720000,0x0000000028820000)]
      0x000000001f38b800 JavaThread "http-apr-8080-exec-628" daemon [_thread_blocked, id=14600, stack(0x0000000028500000,0x0000000028600000)]
      0x000000001f38d800 JavaThread "http-apr-8080-exec-627" daemon [_thread_blocked, id=8272, stack(0x0000000027f20000,0x0000000028020000)]
      0x000000001f38c000 JavaThread "http-apr-8080-exec-626" daemon [_thread_blocked, id=14688, stack(0x00000000283b0000,0x00000000284b0000)]
      0x000000001f38a800 JavaThread "http-apr-8080-exec-624" daemon [_thread_blocked, id=10456, stack(0x00000000281e0000,0x00000000282e0000)]
      0x000000001f38f000 JavaThread "http-apr-8080-exec-623" daemon [_thread_blocked, id=15012, stack(0x0000000028060000,0x0000000028160000)]
      0x000000001f389800 JavaThread "http-apr-8080-exec-622" daemon [_thread_blocked, id=9692, stack(0x0000000027900000,0x0000000027a00000)]
      0x000000001f38e800 JavaThread "http-apr-8080-exec-621" daemon [_thread_blocked, id=6484, stack(0x0000000026b30000,0x0000000026c30000)]
      0x000000001f389000 JavaThread "http-apr-8080-exec-620" daemon [_thread_blocked, id=5840, stack(0x0000000027710000,0x0000000027810000)]
      0x000000001f388000 JavaThread "http-apr-8080-exec-619" daemon [_thread_blocked, id=12740, stack(0x000000001d250000,0x000000001d350000)]
      0x000000001e0c4800 JavaThread "http-apr-8080-exec-613" daemon [_thread_blocked, id=13252, stack(0x0000000027280000,0x0000000027380000)]
      0x000000001e0be800 JavaThread "http-apr-8080-exec-618" daemon [_thread_blocked, id=14384, stack(0x0000000027570000,0x0000000027670000)]
      0x000000001e0b6800 JavaThread "http-apr-8080-exec-617" daemon [_thread_blocked, id=13464, stack(0x00000000273b0000,0x00000000274b0000)]
      0x000000001e0bf800 JavaThread "http-apr-8080-exec-616" daemon [_thread_blocked, id=8000, stack(0x0000000027130000,0x0000000027230000)]
      0x000000001e0c2800 JavaThread "http-apr-8080-exec-615" daemon [_thread_blocked, id=7684, stack(0x0000000026e40000,0x0000000026f40000)]
      0x000000001e0bb800 JavaThread "http-apr-8080-exec-614" daemon [_thread_blocked, id=4924, stack(0x0000000026c60000,0x0000000026d60000)]
      0x000000001e0c0000 JavaThread "http-apr-8080-exec-612" daemon [_thread_blocked, id=5340, stack(0x0000000026780000,0x0000000026880000)]
      0x000000001e0bc800 JavaThread "http-apr-8080-exec-611" daemon [_thread_blocked, id=10568, stack(0x0000000026910000,0x0000000026a10000)]
      0x000000001e0b8000 JavaThread "http-apr-8080-exec-610" daemon [_thread_blocked, id=512, stack(0x0000000026650000,0x0000000026750000)]
      0x000000001e0c4000 JavaThread "http-apr-8080-exec-609" daemon [_thread_blocked, id=3196, stack(0x0000000026550000,0x0000000026650000)]
      0x000000001e0b7000 JavaThread "http-apr-8080-exec-608" daemon [_thread_blocked, id=11280, stack(0x0000000026380000,0x0000000026480000)]
      0x000000001e0b9800 JavaThread "http-apr-8080-exec-607" daemon [_thread_blocked, id=9860, stack(0x0000000026030000,0x0000000026130000)]
      0x000000001e0bb000 JavaThread "http-apr-8080-exec-606" daemon [_thread_blocked, id=14944, stack(0x0000000026240000,0x0000000026340000)]
      0x000000001e0c1000 JavaThread "http-apr-8080-exec-605" daemon [_thread_blocked, id=6944, stack(0x0000000026140000,0x0000000026240000)]
      0x000000001e0bd000 JavaThread "http-apr-8080-exec-604" daemon [_thread_blocked, id=9876, stack(0x0000000025e70000,0x0000000025f70000)]
      0x000000001e0c3000 JavaThread "http-apr-8080-exec-603" daemon [_thread_blocked, id=13192, stack(0x0000000025cd0000,0x0000000025dd0000)]
      0x000000001e0c1800 JavaThread "http-apr-8080-exec-602" daemon [_thread_blocked, id=11868, stack(0x00000000251e0000,0x00000000252e0000)]
      0x000000001e0ba000 JavaThread "http-apr-8080-exec-601" daemon [_thread_blocked, id=7288, stack(0x0000000025af0000,0x0000000025bf0000)]
      0x000000001e0c5800 JavaThread "http-apr-8080-exec-600" daemon [_thread_blocked, id=8228, stack(0x0000000025960000,0x0000000025a60000)]
      0x000000001e2e1800 JavaThread "http-apr-8080-exec-599" daemon [_thread_blocked, id=14164, stack(0x0000000025760000,0x0000000025860000)]
      0x000000001e2ee000 JavaThread "http-apr-8080-exec-598" daemon [_thread_blocked, id=10924, stack(0x0000000025600000,0x0000000025700000)]
      0x000000001e2e8000 JavaThread "http-apr-8080-exec-597" daemon [_thread_blocked, id=14928, stack(0x00000000254d0000,0x00000000255d0000)]
      0x000000001e2e5800 JavaThread "http-apr-8080-exec-596" daemon [_thread_blocked, id=8364, stack(0x0000000025310000,0x0000000025410000)]
      0x000000001e2ee800 JavaThread "http-apr-8080-exec-595" daemon [_thread_blocked, id=7508, stack(0x0000000024f80000,0x0000000025080000)]
      0x000000001e2ec800 JavaThread "http-apr-8080-exec-594" daemon [_thread_blocked, id=11068, stack(0x00000000250c0000,0x00000000251c0000)]
      0x000000001e2e8800 JavaThread "http-apr-8080-exec-593" daemon [_thread_blocked, id=10552, stack(0x0000000024e10000,0x0000000024f10000)]
      0x000000001e2eb000 JavaThread "http-apr-8080-exec-592" daemon [_thread_blocked, id=14956, stack(0x0000000024a10000,0x0000000024b10000)]
      0x000000001e2e7000 JavaThread "http-apr-8080-exec-591" daemon [_thread_blocked, id=8128, stack(0x0000000024650000,0x0000000024750000)]
      0x000000001e2ea000 JavaThread "http-apr-8080-exec-590" daemon [_thread_blocked, id=7880, stack(0x0000000024890000,0x0000000024990000)]
      0x000000001e2eb800 JavaThread "http-apr-8080-exec-589" daemon [_thread_blocked, id=2904, stack(0x0000000024760000,0x0000000024860000)]
      0x000000001e2e4800 JavaThread "http-apr-8080-exec-588" daemon [_thread_blocked, id=13972, stack(0x000000001aaf0000,0x000000001abf0000)]
      0x000000001e2ed000 JavaThread "http-apr-8080-exec-587" daemon [_thread_blocked, id=9868, stack(0x0000000023370000,0x0000000023470000)]
      0x000000001e2ef800 JavaThread "http-apr-8080-exec-586" daemon [_thread_blocked, id=12524, stack(0x00000000244f0000,0x00000000245f0000)]
      0x000000001e2e3000 JavaThread "http-apr-8080-exec-585" daemon [_thread_blocked, id=4500, stack(0x0000000019260000,0x0000000019360000)]
      0x000000001e2e0000 JavaThread "http-apr-8080-exec-584" daemon [_thread_blocked, id=3644, stack(0x0000000021e00000,0x0000000021f00000)]
      0x000000001e2e6800 JavaThread "http-apr-8080-exec-583" daemon [_thread_blocked, id=11864, stack(0x0000000019c60000,0x0000000019d60000)]
      0x000000001e2e4000 JavaThread "http-apr-8080-exec-582" daemon [_thread_blocked, id=8668, stack(0x000000001d050000,0x000000001d150000)]
      0x000000001e2e1000 JavaThread "http-apr-8080-exec-581" daemon [_thread_blocked, id=6356, stack(0x000000001bdf0000,0x000000001bef0000)]
      0x000000001f38d000 JavaThread "http-apr-8080-exec-538" daemon [_thread_blocked, id=4396, stack(0x00000000310a0000,0x00000000311a0000)]
      0x000000001a0f2800 JavaThread "http-apr-8080-exec-531" daemon [_thread_blocked, id=6632, stack(0x00000000308e0000,0x00000000309e0000)]
      0x000000001a4f8000 JavaThread "http-apr-8080-exec-505" daemon [_thread_blocked, id=13804, stack(0x000000002e4a0000,0x000000002e5a0000)]
      0x000000001a4f6000 JavaThread "http-apr-8080-exec-502" daemon [_thread_blocked, id=5144, stack(0x000000002dca0000,0x000000002dda0000)]
      0x000000001ed5d000 JavaThread "http-apr-8080-exec-475" daemon [_thread_blocked, id=14108, stack(0x000000002b3a0000,0x000000002b4a0000)]
      0x00000000242e7000 JavaThread "http-apr-8080-exec-468" daemon [_thread_blocked, id=8840, stack(0x000000002a390000,0x000000002a490000)]
      0x00000000242e6000 JavaThread "http-apr-8080-exec-463" daemon [_thread_blocked, id=2764, stack(0x000000002a4c0000,0x000000002a5c0000)]
      0x000000001e2e9800 JavaThread "http-apr-8080-exec-441" daemon [_thread_blocked, id=11860, stack(0x0000000028a60000,0x0000000028b60000)]
      0x000000001e2e2800 JavaThread "http-apr-8080-exec-430" daemon [_thread_blocked, id=10620, stack(0x0000000026f60000,0x0000000027060000)]
      0x000000001e0b8800 JavaThread "http-apr-8080-exec-420" daemon [_thread_blocked, id=12060, stack(0x0000000026a30000,0x0000000026b30000)]
      0x000000001a1ea000 JavaThread "ajp-apr-8019-AsyncTimeout" daemon [_thread_blocked, id=12688, stack(0x000000001d150000,0x000000001d250000)]
      0x000000001a1e9000 JavaThread "ajp-apr-8019-Acceptor-0" daemon [_thread_in_native, id=10124, stack(0x000000001cee0000,0x000000001cfe0000)]
      0x000000001a1e8800 JavaThread "ajp-apr-8019-Poller" daemon [_thread_blocked, id=3552, stack(0x000000001cd30000,0x000000001ce30000)]
      0x000000001a1e7800 JavaThread "http-apr-8080-AsyncTimeout" daemon [_thread_blocked, id=14876, stack(0x000000001cb60000,0x000000001cc60000)]
      0x000000001a1e7000 JavaThread "http-apr-8080-Acceptor-0" daemon [_thread_in_native, id=12780, stack(0x000000001ca10000,0x000000001cb10000)]
      0x000000001a1e6000 JavaThread "http-apr-8080-Sendfile" daemon [_thread_blocked, id=11020, stack(0x000000001c8e0000,0x000000001c9e0000)]
    =>0x000000001a1e5800 JavaThread "http-apr-8080-Poller" daemon [_thread_in_native, id=11828, stack(0x000000001c420000,0x000000001c520000)]
      0x000000001a1e4000 JavaThread "ContainerBackgroundProcessor[StandardEngine[Catalina]]" daemon [_thread_blocked, id=4780, stack(0x000000001c540000,0x000000001c640000)]
      0x000000001a1e3000 JavaThread "GC Daemon" daemon [_thread_blocked, id=7360, stack(0x000000001c0b0000,0x000000001c1b0000)]
      0x000000001a1d7800 JavaThread "RMI Scheduler(0)" daemon [_thread_blocked, id=15236, stack(0x000000001a970000,0x000000001aa70000)]
      0x0000000018884800 JavaThread "RMI TCP Accept-0" daemon [_thread_in_native, id=11560, stack(0x00000000198c0000,0x00000000199c0000)]
      0x0000000018873800 JavaThread "RMI TCP Accept-1099" daemon [_thread_in_native, id=12684, stack(0x0000000019b30000,0x0000000019c30000)]
      0x000000001881f800 JavaThread "RMI TCP Accept-0" daemon [_thread_in_native, id=1868, stack(0x0000000019620000,0x0000000019720000)]
      0x00000000185fb800 JavaThread "Service Thread" daemon [_thread_blocked, id=13392, stack(0x0000000019510000,0x0000000019610000)]
      0x0000000018575800 JavaThread "C1 CompilerThread2" daemon [_thread_blocked, id=6968, stack(0x0000000019410000,0x0000000019510000)]
      0x0000000018570000 JavaThread "C2 CompilerThread1" daemon [_thread_blocked, id=6488, stack(0x0000000019030000,0x0000000019130000)]
      0x000000001856a800 JavaThread "C2 CompilerThread0" daemon [_thread_blocked, id=7528, stack(0x0000000019160000,0x0000000019260000)]
      0x000000001855c000 JavaThread "JDWP Command Reader" daemon [_thread_in_native, id=13196, stack(0x0000000018ed0000,0x0000000018fd0000)]
      0x0000000018557800 JavaThread "JDWP Event Helper Thread" daemon [_thread_blocked, id=15308, stack(0x0000000018da0000,0x0000000018ea0000)]
      0x000000001714f000 JavaThread "JDWP Transport Listener: dt_socket" daemon [_thread_blocked, id=6536, stack(0x0000000018930000,0x0000000018a30000)]
      0x0000000017148800 JavaThread "Attach Listener" daemon [_thread_blocked, id=8788, stack(0x0000000017f40000,0x0000000018040000)]
      0x0000000017144000 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=6084, stack(0x0000000018a90000,0x0000000018b90000)]
      0x000000001712c000 JavaThread "Finalizer" daemon [_thread_blocked, id=2112, stack(0x0000000018430000,0x0000000018530000)]
      0x00000000170e4800 JavaThread "Reference Handler" daemon [_thread_blocked, id=14692, stack(0x0000000018290000,0x0000000018390000)]
      0x0000000001e9f000 JavaThread "main" [_thread_in_native, id=10256, stack(0x00000000026e0000,0x00000000027e0000)]
    
    Other Threads:
      0x00000000170dd000 VMThread [stack: 0x00000000180a0000,0x00000000181a0000] [id=13616]
      0x0000000018889800 WatcherThread [stack: 0x0000000019d80000,0x0000000019e80000] [id=9576]
    
    VM state:not at safepoint (normal execution)
    
    VM Mutex/Monitor currently owned by a thread: None
    
    Heap:
     PSYoungGen      total 583680K, used 465274K [0x00000000d8380000, 0x00000000ffc80000, 0x0000000100000000)
      eden space 518656K, 89% used [0x00000000d8380000,0x00000000f4896a70,0x00000000f7e00000)
      from space 65024K, 2% used [0x00000000f7e00000,0x00000000f7f48000,0x00000000fbd80000)
      to   space 62976K, 0% used [0x00000000fbf00000,0x00000000fbf00000,0x00000000ffc80000)
     ParOldGen       total 133632K, used 61052K [0x0000000088a00000, 0x0000000090c80000, 0x00000000d8380000)
      object space 133632K, 45% used [0x0000000088a00000,0x000000008c59f140,0x0000000090c80000)
     Metaspace       used 33765K, capacity 34314K, committed 34816K, reserved 1079296K
      class space    used 3802K, capacity 4019K, committed 4096K, reserved 1048576K
    
    Card table byte_map: [0x0000000011ba0000,0x0000000011f60000] byte_map_base: 0x000000001175b000
    
    Marking Bits: (ParMarkBitMap*) 0x000000005325a6d0
     Begin Bits: [0x00000000127e0000, 0x00000000145b8000)
     End Bits:   [0x00000000145b8000, 0x0000000016390000)
    
    Polling page: 0x0000000001e70000
    
    CodeCache: size=245760Kb used=19834Kb max_used=21178Kb free=225925Kb
     bounds [0x00000000027e0000, 0x0000000003d10000, 0x00000000117e0000]
     total_blobs=5219 nmethods=4771 adapters=367
     compilation: enabled
    
    Compilation events (10 events):
    Event: 2108.116 Thread 0x0000000018575800 6307 % !   3       org.apache.tomcat.util.net.AprEndpoint$Poller::run @ 672 (2241 bytes)
    Event: 2108.137 Thread 0x0000000018575800 nmethod 6307% 0x0000000003cf3210 code [0x0000000003cf4080, 0x0000000003d00608]
    Event: 2110.648 Thread 0x0000000018570000 6308       4       java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject::addConditionWaiter (64 bytes)
    Event: 2110.650 Thread 0x0000000018570000 nmethod 6308 0x0000000003aa89d0 code [0x0000000003aa8b20, 0x0000000003aa8ca8]
    Event: 2121.424 Thread 0x000000001856a800 6309       4       java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject::doSignal (43 bytes)
    Event: 2121.442 Thread 0x000000001856a800 nmethod 6309 0x0000000003715010 code [0x0000000003715180, 0x00000000037154e8]
    Event: 2125.553 Thread 0x0000000018575800 6310       1       java.util.concurrent.ConcurrentHashMap::spread (10 bytes)
    Event: 2125.553 Thread 0x0000000018575800 nmethod 6310 0x0000000003aa8710 code [0x0000000003aa8860, 0x0000000003aa8950]
    Event: 2125.553 Thread 0x0000000018575800 6311       3       java.util.concurrent.ConcurrentHashMap::get (162 bytes)
    Event: 2125.555 Thread 0x0000000018575800 nmethod 6311 0x000000000362b2d0 code [0x000000000362b4e0, 0x000000000362c0b8]
    
    GC Heap History (10 events):
    Event: 61.902 GC heap before
    {Heap before GC invocations=18 (full 3):
     PSYoungGen      total 572928K, used 510464K [0x00000000d8380000, 0x00000000ffa00000, 0x0000000100000000)
      eden space 510464K, 100% used [0x00000000d8380000,0x00000000f7600000,0x00000000f7600000)
      from space 62464K, 0% used [0x00000000fbd00000,0x00000000fbd00000,0x00000000ffa00000)
      to   space 67584K, 0% used [0x00000000f7600000,0x00000000f7600000,0x00000000fb800000)
     ParOldGen       total 133632K, used 59508K [0x0000000088a00000, 0x0000000090c80000, 0x00000000d8380000)
      object space 133632K, 44% used [0x0000000088a00000,0x000000008c41d130,0x0000000090c80000)
     Metaspace       used 33530K, capacity 34122K, committed 34560K, reserved 1079296K
      class space    used 3794K, capacity 4019K, committed 4096K, reserved 1048576K
    Event: 61.952 GC heap after
    Heap after GC invocations=18 (full 3):
     PSYoungGen      total 578048K, used 1568K [0x00000000d8380000, 0x0000000100000000, 0x0000000100000000)
      eden space 510464K, 0% used [0x00000000d8380000,0x00000000d8380000,0x00000000f7600000)
      from space 67584K, 2% used [0x00000000f7600000,0x00000000f7788000,0x00000000fb800000)
      to   space 67584K, 0% used [0x00000000fbe00000,0x00000000fbe00000,0x0000000100000000)
     ParOldGen       total 133632K, used 59516K [0x0000000088a00000, 0x0000000090c80000, 0x00000000d8380000)
      object space 133632K, 44% used [0x0000000088a00000,0x000000008c41f130,0x0000000090c80000)
     Metaspace       used 33530K, capacity 34122K, committed 34560K, reserved 1079296K
      class space    used 3794K, capacity 4019K, committed 4096K, reserved 1048576K
    }
    Event: 90.777 GC heap before
    {Heap before GC invocations=19 (full 3):
     PSYoungGen      total 578048K, used 512032K [0x00000000d8380000, 0x0000000100000000, 0x0000000100000000)
      eden space 510464K, 100% used [0x00000000d8380000,0x00000000f7600000,0x00000000f7600000)
      from space 67584K, 2% used [0x00000000f7600000,0x00000000f7788000,0x00000000fb800000)
      to   space 67584K, 0% used [0x00000000fbe00000,0x00000000fbe00000,0x0000000100000000)
     ParOldGen       total 133632K, used 59516K [0x0000000088a00000, 0x0000000090c80000, 0x00000000d8380000)
      object space 133632K, 44% used [0x0000000088a00000,0x000000008c41f130,0x0000000090c80000)
     Metaspace       used 33562K, capacity 34122K, committed 34560K, reserved 1079296K
      class space    used 3799K, capacity 4019K, committed 4096K, reserved 1048576K
    Event: 90.819 GC heap after
    Heap after GC invocations=19 (full 3):
     PSYoungGen      total 583168K, used 640K [0x00000000d8380000, 0x00000000fff00000, 0x0000000100000000)
      eden space 516608K, 0% used [0x00000000d8380000,0x00000000d8380000,0x00000000f7c00000)
      from space 66560K, 0% used [0x00000000fbe00000,0x00000000fbea0000,0x00000000fff00000)
      to   space 67072K, 0% used [0x00000000f7c00000,0x00000000f7c00000,0x00000000fbd80000)
     ParOldGen       total 133632K, used 60396K [0x0000000088a00000, 0x0000000090c80000, 0x00000000d8380000)
      object space 133632K, 45% used [0x0000000088a00000,0x000000008c4fb140,0x0000000090c80000)
     Metaspace       used 33562K, capacity 34122K, committed 34560K, reserved 1079296K
      class space    used 3799K, capacity 4019K, committed 4096K, reserved 1048576K
    }
    Event: 269.712 GC heap before
    {Heap before GC invocations=20 (full 3):
     PSYoungGen      total 583168K, used 517248K [0x00000000d8380000, 0x00000000fff00000, 0x0000000100000000)
      eden space 516608K, 100% used [0x00000000d8380000,0x00000000f7c00000,0x00000000f7c00000)
      from space 66560K, 0% used [0x00000000fbe00000,0x00000000fbea0000,0x00000000fff00000)
      to   space 67072K, 0% used [0x00000000f7c00000,0x00000000f7c00000,0x00000000fbd80000)
     ParOldGen       total 133632K, used 60396K [0x0000000088a00000, 0x0000000090c80000, 0x00000000d8380000)
      object space 133632K, 45% used [0x0000000088a00000,0x000000008c4fb140,0x0000000090c80000)
     Metaspace       used 33593K, capacity 34122K, committed 34560K, reserved 1079296K
      class space    used 3799K, capacity 4019K, committed 4096K, reserved 1048576K
    Event: 270.048 GC heap after
    Heap after GC invocations=20 (full 3):
     PSYoungGen      total 583680K, used 1088K [0x00000000d8380000, 0x00000000ffe80000, 0x0000000100000000)
      eden space 516608K, 0% used [0x00000000d8380000,0x00000000d8380000,0x00000000f7c00000)
      from space 67072K, 1% used [0x00000000f7c00000,0x00000000f7d10000,0x00000000fbd80000)
      to   space 66048K, 0% used [0x00000000fbe00000,0x00000000fbe00000,0x00000000ffe80000)
     ParOldGen       total 133632K, used 60452K [0x0000000088a00000, 0x0000000090c80000, 0x00000000d8380000)
      object space 133632K, 45% used [0x0000000088a00000,0x000000008c509140,0x0000000090c80000)
     Metaspace       used 33593K, capacity 34122K, committed 34560K, reserved 1079296K
      class space    used 3799K, capacity 4019K, committed 4096K, reserved 1048576K
    }
    Event: 315.217 GC heap before
    {Heap before GC invocations=21 (full 3):
     PSYoungGen      total 583680K, used 517696K [0x00000000d8380000, 0x00000000ffe80000, 0x0000000100000000)
      eden space 516608K, 100% used [0x00000000d8380000,0x00000000f7c00000,0x00000000f7c00000)
      from space 67072K, 1% used [0x00000000f7c00000,0x00000000f7d10000,0x00000000fbd80000)
      to   space 66048K, 0% used [0x00000000fbe00000,0x00000000fbe00000,0x00000000ffe80000)
     ParOldGen       total 133632K, used 60452K [0x0000000088a00000, 0x0000000090c80000, 0x00000000d8380000)
      object space 133632K, 45% used [0x0000000088a00000,0x000000008c509140,0x0000000090c80000)
     Metaspace       used 33601K, capacity 34122K, committed 34560K, reserved 1079296K
      class space    used 3801K, capacity 4019K, committed 4096K, reserved 1048576K
    Event: 315.239 GC heap after
    Heap after GC invocations=21 (full 3):
     PSYoungGen      total 583168K, used 384K [0x00000000d8380000, 0x00000000ffd00000, 0x0000000100000000)
      eden space 518656K, 0% used [0x00000000d8380000,0x00000000d8380000,0x00000000f7e00000)
      from space 64512K, 0% used [0x00000000fbe00000,0x00000000fbe60000,0x00000000ffd00000)
      to   space 65024K, 0% used [0x00000000f7e00000,0x00000000f7e00000,0x00000000fbd80000)
     ParOldGen       total 133632K, used 61028K [0x0000000088a00000, 0x0000000090c80000, 0x00000000d8380000)
      object space 133632K, 45% used [0x0000000088a00000,0x000000008c599140,0x0000000090c80000)
     Metaspace       used 33601K, capacity 34122K, committed 34560K, reserved 1079296K
      class space    used 3801K, capacity 4019K, committed 4096K, reserved 1048576K
    }
    Event: 1570.093 GC heap before
    {Heap before GC invocations=22 (full 3):
     PSYoungGen      total 583168K, used 519040K [0x00000000d8380000, 0x00000000ffd00000, 0x0000000100000000)
      eden space 518656K, 100% used [0x00000000d8380000,0x00000000f7e00000,0x00000000f7e00000)
      from space 64512K, 0% used [0x00000000fbe00000,0x00000000fbe60000,0x00000000ffd00000)
      to   space 65024K, 0% used [0x00000000f7e00000,0x00000000f7e00000,0x00000000fbd80000)
     ParOldGen       total 133632K, used 61028K [0x0000000088a00000, 0x0000000090c80000, 0x00000000d8380000)
      object space 133632K, 45% used [0x0000000088a00000,0x000000008c599140,0x0000000090c80000)
     Metaspace       used 33675K, capacity 34250K, committed 34560K, reserved 1079296K
      class space    used 3801K, capacity 4019K, committed 4096K, reserved 1048576K
    Event: 1570.102 GC heap after
    Heap after GC invocations=22 (full 3):
     PSYoungGen      total 583680K, used 1312K [0x00000000d8380000, 0x00000000ffc80000, 0x0000000100000000)
      eden space 518656K, 0% used [0x00000000d8380000,0x00000000d8380000,0x00000000f7e00000)
      from space 65024K, 2% used [0x00000000f7e00000,0x00000000f7f48000,0x00000000fbd80000)
      to   space 62976K, 0% used [0x00000000fbf00000,0x00000000fbf00000,0x00000000ffc80000)
     ParOldGen       total 133632K, used 61052K [0x0000000088a00000, 0x0000000090c80000, 0x00000000d8380000)
      object space 133632K, 45% used [0x0000000088a00000,0x000000008c59f140,0x0000000090c80000)
     Metaspace       used 33675K, capacity 34250K, committed 34560K, reserved 1079296K
      class space    used 3801K, capacity 4019K, committed 4096K, reserved 1048576K
    }
    
    Deoptimization events (10 events):
    Event: 388.516 Thread 0x0000000021649800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x000000000331c6d4 method=java.util.concurrent.ThreadPoolExecutor.getTask()Ljava/lang/Runnable; @ 113
    Event: 901.816 Thread 0x000000001a1d7800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x0000000002b4bdf4 method=java.util.HashMap$HashIterator.<init>(Ljava/util/HashMap;)V @ 45
    Event: 901.816 Thread 0x000000001a1d7800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x000000000366e29c method=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.addConditionWaiter()Ljava/util/concurrent/locks/AbstractQueuedSynchronizer$Node; @ 15
    Event: 2104.757 Thread 0x000000001a1e5800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x0000000003ce559c method=org.apache.tomcat.util.net.AprEndpoint$Poller.run()V @ 1362
    Event: 2104.992 Thread 0x000000001a1e5800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x0000000003a18e3c method=org.apache.tomcat.util.net.AprEndpoint$Poller.addToPoller(JI)Z @ 23
    Event: 2105.392 Thread 0x000000001a1e5800 Uncommon trap: reason=class_check action=maybe_recompile pc=0x00000000038ef390 method=java.lang.Throwable.<init>()V @ 24
    Event: 2105.399 Thread 0x000000001a1e5800 Uncommon trap: reason=class_check action=maybe_recompile pc=0x00000000038ef390 method=java.lang.Throwable.<init>()V @ 24
    Event: 2105.421 Thread 0x000000001a1e5800 Uncommon trap: reason=class_check action=maybe_recompile pc=0x00000000038ef390 method=java.lang.Throwable.<init>()V @ 24
    Event: 2125.243 Thread 0x000000001a1e7000 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00000000032866a0 method=java.util.concurrent.ConcurrentHashMap.putVal(Ljava/lang/Object;Ljava/lang/Object;Z)Ljava/lang/Object; @ 331
    Event: 2125.553 Thread 0x000000001a1e5800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x000000000329a180 method=java.util.concurrent.ConcurrentHashMap.get(Ljava/lang/Object;)Ljava/lang/Object; @ 86
    
    Internal exceptions (10 events):
    Event: 2121.066 Thread 0x000000001a1e5800 Implicit null exception at 0x00000000039f91ac to 0x00000000039f91bb
    Event: 2121.066 Thread 0x000000001a1e5800 Exception <a 'java/lang/NullPointerException'> (0x00000000f4811098) thrown at [C:
    eworkspace8-2-build-windows-amd64-cygwinjdk8u1017261hotspotsrcsharevm
    untimesharedRuntime.cpp, line 605]
    Event: 2121.069 Thread 0x000000001a1e5800 Implicit null exception at 0x00000000039f91ac to 0x00000000039f91bb
    Event: 2121.070 Thread 0x000000001a1e5800 Exception <a 'java/lang/NullPointerException'> (0x00000000f481c240) thrown at [C:
    eworkspace8-2-build-windows-amd64-cygwinjdk8u1017261hotspotsrcsharevm
    untimesharedRuntime.cpp, line 605]
    Event: 2121.100 Thread 0x000000001a1e5800 Implicit null exception at 0x00000000039f91ac to 0x00000000039f91bb
    Event: 2121.101 Thread 0x000000001a1e5800 Exception <a 'java/lang/NullPointerException'> (0x00000000f4827648) thrown at [C:
    eworkspace8-2-build-windows-amd64-cygwinjdk8u1017261hotspotsrcsharevm
    untimesharedRuntime.cpp, line 605]
    Event: 2121.104 Thread 0x000000001a1e5800 Implicit null exception at 0x00000000039f91ac to 0x00000000039f91bb
    Event: 2121.104 Thread 0x000000001a1e5800 Exception <a 'java/lang/NullPointerException'> (0x00000000f4832818) thrown at [C:
    eworkspace8-2-build-windows-amd64-cygwinjdk8u1017261hotspotsrcsharevm
    untimesharedRuntime.cpp, line 605]
    Event: 2121.108 Thread 0x000000001a1e5800 Implicit null exception at 0x00000000039f91ac to 0x00000000039f91bb
    Event: 2121.108 Thread 0x000000001a1e5800 Exception <a 'java/lang/NullPointerException'> (0x00000000f483d9a8) thrown at [C:
    eworkspace8-2-build-windows-amd64-cygwinjdk8u1017261hotspotsrcsharevm
    untimesharedRuntime.cpp, line 605]
    
    Events (10 events):
    Event: 2125.243 Thread 0x000000001a1e7000 Uncommon trap: trap_request=0xffffff65 fr.pc=0x00000000032866a0
    Event: 2125.243 Thread 0x000000001a1e7000 DEOPT PACKING pc=0x00000000032866a0 sp=0x000000001cb0ec60
    Event: 2125.243 Thread 0x000000001a1e7000 DEOPT UNPACKING pc=0x000000000282582a sp=0x000000001cb0ec08 mode 2
    Event: 2125.243 loading class java/util/concurrent/ConcurrentHashMap$TreeNode
    Event: 2125.244 loading class java/util/concurrent/ConcurrentHashMap$TreeNode done
    Event: 2125.244 loading class java/util/concurrent/ConcurrentHashMap$TreeBin
    Event: 2125.244 loading class java/util/concurrent/ConcurrentHashMap$TreeBin done
    Event: 2125.552 Thread 0x000000001a1e5800 Uncommon trap: trap_request=0xffffff65 fr.pc=0x000000000329a180
    Event: 2125.553 Thread 0x000000001a1e5800 DEOPT PACKING pc=0x000000000329a180 sp=0x000000001c51f380
    Event: 2125.553 Thread 0x000000001a1e5800 DEOPT UNPACKING pc=0x000000000282582a sp=0x000000001c51f320 mode 2
    
    
    Dynamic libraries:
    0x000000013f910000 - 0x000000013f947000     D:Program FilesJavajdk1.8.0_101injava.exe
    0x00000000774a0000 - 0x000000007764a000     C:WindowsSYSTEM32
    tdll.dll
    0x0000000077280000 - 0x000000007739f000     C:Windowssystem32kernel32.dll
    0x000007fefd2e0000 - 0x000007fefd34a000     C:Windowssystem32KERNELBASE.dll
    0x000007feff6d0000 - 0x000007feff7ab000     C:Windowssystem32ADVAPI32.dll
    0x000007fefeaf0000 - 0x000007fefeb8f000     C:Windowssystem32msvcrt.dll
    0x000007fefd6e0000 - 0x000007fefd6ff000     C:WindowsSYSTEM32sechost.dll
    0x000007feff1e0000 - 0x000007feff30d000     C:Windowssystem32RPCRT4.dll
    0x00000000773a0000 - 0x000000007749a000     C:Windowssystem32USER32.dll
    0x000007fefed20000 - 0x000007fefed87000     C:Windowssystem32GDI32.dll
    0x000007fefed90000 - 0x000007fefed9e000     C:Windowssystem32LPK.dll
    0x000007feff110000 - 0x000007feff1da000     C:Windowssystem32USP10.dll
    0x000007fefbc40000 - 0x000007fefbe34000     C:WindowsWinSxSamd64_microsoft.windows.common-controls_6595b64144ccf1df_6.0.7601.18837_none_fa3b1e3d17594757COMCTL32.dll
    0x000007feff390000 - 0x000007feff401000     C:Windowssystem32SHLWAPI.dll
    0x000007fefda70000 - 0x000007fefda9e000     C:Windowssystem32IMM32.DLL
    0x000007fefd700000 - 0x000007fefd809000     C:Windowssystem32MSCTF.dll
    0x000007fefd1a0000 - 0x000007fefd1d0000     C:Windowssystem32
    vinitx.dll
    0x000007fefd190000 - 0x000007fefd19c000     C:Windowssystem32VERSION.dll
    0x00000000744c0000 - 0x00000000744c6000     C:Program FilesNVIDIA CorporationCoProcManagerdetoured.dll
    0x000007fefb190000 - 0x000007fefb1c8000     C:Program FilesNVIDIA CorporationCoProcManager
    vd3d9wrapx.dll
    0x000007feff4f0000 - 0x000007feff6c7000     C:Windowssystem32SETUPAPI.dll
    0x000007fefd2a0000 - 0x000007fefd2d6000     C:Windowssystem32CFGMGR32.dll
    0x000007feff410000 - 0x000007feff4ea000     C:Windowssystem32OLEAUT32.dll
    0x000007fefe880000 - 0x000007fefea7c000     C:Windowssystem32ole32.dll
    0x000007fefd480000 - 0x000007fefd49a000     C:Windowssystem32DEVOBJ.dll
    0x000007fefb060000 - 0x000007fefb084000     C:Program FilesNVIDIA CorporationCoProcManager
    vdxgiwrapx.dll
    0x000000006a3a0000 - 0x000000006a472000     D:Program FilesJavajdk1.8.0_101jreinmsvcr100.dll
    0x0000000052a40000 - 0x00000000532da000     D:Program FilesJavajdk1.8.0_101jreinserverjvm.dll
    0x000007fefb0f0000 - 0x000007fefb0f9000     C:Windowssystem32WSOCK32.dll
    0x000007fefe830000 - 0x000007fefe87d000     C:Windowssystem32WS2_32.dll
    0x000007fefeae0000 - 0x000007fefeae8000     C:Windowssystem32NSI.dll
    0x000007fefadf0000 - 0x000007fefae2b000     C:Windowssystem32WINMM.dll
    0x0000000077670000 - 0x0000000077677000     C:Windowssystem32PSAPI.DLL
    0x000000006c230000 - 0x000000006c23f000     D:Program FilesJavajdk1.8.0_101jreinverify.dll
    0x000000006c0c0000 - 0x000000006c0e9000     D:Program FilesJavajdk1.8.0_101jreinjava.dll
    0x000000006ba30000 - 0x000000006ba65000     D:Program FilesJavajdk1.8.0_101jreinjdwp.dll
    0x000000006c030000 - 0x000000006c038000     D:Program FilesJavajdk1.8.0_101jrein
    pt.dll
    0x000000006c1e0000 - 0x000000006c1f6000     D:Program FilesJavajdk1.8.0_101jreinzip.dll
    0x000007fefdaa0000 - 0x000007fefe82a000     C:Windowssystem32SHELL32.dll
    0x000007fefd270000 - 0x000007fefd27f000     C:Windowssystem32profapi.dll
    0x000000006ba20000 - 0x000000006ba29000     D:Program FilesJavajdk1.8.0_101jreindt_socket.dll
    0x000007fefc9f0000 - 0x000007fefca45000     C:Windowssystem32mswsock.dll
    0x000007fefc3c0000 - 0x000007fefc3c7000     C:WindowsSystem32wshtcpip.dll
    0x0000000073120000 - 0x000000007312d000     D:Program FilesJavajdk1.8.0_101jreinmanagement.dll
    0x000000006c1c0000 - 0x000000006c1da000     D:Program FilesJavajdk1.8.0_101jrein
    et.dll
    0x000007fefc9e0000 - 0x000007fefc9e7000     C:WindowsSystem32wship6.dll
    0x000000006be10000 - 0x000000006be21000     D:Program FilesJavajdk1.8.0_101jrein
    io.dll
    0x000007fefac10000 - 0x000007fefac25000     C:Windowssystem32NLAapi.dll
    0x000007feeed90000 - 0x000007feeeda5000     C:Windowssystem32
    apinsp.dll
    0x000007feeed70000 - 0x000007feeed89000     C:Windowssystem32pnrpnsp.dll
    0x000007fefc870000 - 0x000007fefc8cb000     C:Windowssystem32DNSAPI.dll
    0x000007feeed60000 - 0x000007feeed6b000     C:WindowsSystem32winrnr.dll
    0x000007feeed50000 - 0x000007feeed60000     C:Windowssystem32wshbth.dll
    0x000007fefa030000 - 0x000007fefa057000     C:Windowssystem32IPHLPAPI.DLL
    0x000007fefa020000 - 0x000007fefa02b000     C:Windowssystem32WINNSI.DLL
    0x000007fef4580000 - 0x000007fef4588000     C:Windowssystem32
    asadhlp.dll
    0x000007fef9c30000 - 0x000007fef9c83000     C:WindowsSystem32fwpuclnt.dll
    0x000007fefca50000 - 0x000007fefca68000     C:Windowssystem32CRYPTSP.dll
    0x000007fefc750000 - 0x000007fefc797000     C:Windowssystem32
    saenh.dll
    0x0000000002440000 - 0x000000000245e000     C:Windowssystem32USERENV.dll
    0x000007fefd0c0000 - 0x000007fefd0cf000     C:Windowssystem32CRYPTBASE.dll
    0x000007fef9c00000 - 0x000007fef9c18000     C:Windowssystem32dhcpcsvc.DLL
    0x000007fef9be0000 - 0x000007fef9bf1000     C:Windowssystem32dhcpcsvc6.DLL
    0x0000000180000000 - 0x000000018020a000     D:servers	omcat-7.0.70in	cnative-1.dll
    0x0000000069dd0000 - 0x0000000069df4000     D:Program FilesJavajdk1.8.0_101jreinsunec.dll
    0x000000006b9a0000 - 0x000000006b9ab000     D:Program FilesJavajdk1.8.0_101jreinsunmscapi.dll
    0x000007fefd4b0000 - 0x000007fefd61d000     C:Windowssystem32CRYPT32.dll
    0x000007fefd260000 - 0x000007fefd26f000     C:Windowssystem32MSASN1.dll
    0x000007fee5320000 - 0x000007fee5445000     C:Windowssystem32dbghelp.dll
    
    VM Arguments:
    jvm_args: -agentlib:jdwp=transport=dt_socket,address=127.0.0.1:58835,suspend=y,server=n -Dcom.sun.management.jmxremote= -Dcom.sun.management.jmxremote.port=1099 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Djava.rmi.server.hostname=127.0.0.1 -Djdk.tls.ephemeralDHKeySize=2048 -Djava.util.logging.config.file=C:Usersxxx.IntelliJIdea2016.1system	omcatUnnamed_mvn_test_localconflogging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=D:servers	omcat-7.0.70endorsed -Dcatalina.base=C:Usersxxx.IntelliJIdea2016.1system	omcatUnnamed_mvn_test_local -Dcatalina.home=D:servers	omcat-7.0.70 -Djava.io.tmpdir=D:servers	omcat-7.0.70	emp 
    java_command: org.apache.catalina.startup.Bootstrap start
    java_class_path (initial): D:servers	omcat-7.0.70inootstrap.jar;D:servers	omcat-7.0.70in	omcat-juli.jar
    Launcher Type: SUN_STANDARD
    
    Environment Variables:
    JAVA_HOME=D:Program FilesJavajdk1.8.0_101
    JRE_HOME=D:Program FilesJavajdk1.8.0_101
    CLASSPATH=D:servers	omcat-7.0.70inootstrap.jar;D:servers	omcat-7.0.70in	omcat-juli.jar
    PATH=C:ProgramDataOracleJavajavapath;C:Windowssystem32;C:Windows;C:WindowsSystem32Wbem;C:WindowsSystem32WindowsPowerShellv1.0;C:Program Files (x86)NVIDIA CorporationPhysXCommon;D:Program FilesGitcmd;D:Program FilesTortoiseGitin;D:Program FilesJavajdk1.8.0_101in;D:Program FilesJavajdk1.8.0_101jrein;C:Program FilesTortoiseSVNin;D:Program Filesapache-maven-3.3.9in;D:xamppphp;C:ProgramDataComposerSetupin;D:Program Files
    odejs;D:Program Files
    odejs;D:Program Filesgradle-4.1in;C:UsersxxxAppDataLocalProgramsPythonPython36Scripts;C:UsersxxxAppDataLocalProgramsPythonPython36;C:UsersxxxAppDataRoamingComposervendorin;C:UsersxxxAppDataRoaming
    pm;D:Program Files
    odejs;C:Program Files (x86)OpenVPNin;C:UsersxxxAppDataLocalProgramsFiddler;C:UsersxxxAppDataLocalProgramsPythonPython36;
    USERNAME=xxx
    OS=Windows_NT
    PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 142 Stepping 9, GenuineIntel
    
    
    
    ---------------  S Y S T E M  ---------------
    
    OS: Windows 7 , 64 bit Build 7601 (6.1.7601.23714)
    
    CPU:total 4 (2 cores per cpu, 2 threads per core) family 6 model 142 stepping 9, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, avx, avx2, aes, clmul, erms, 3dnowpref, lzcnt, ht, tsc, tscinvbit, bmi1, bmi2, adx
    
    Memory: 4k page, physical 7822340k(3261452k free), swap 15642820k(8502936k free)
    
    vm_info: Java HotSpot(TM) 64-Bit Server VM (25.101-b13) for windows-amd64 JRE (1.8.0_101-b13), built on Jun 22 2016 01:21:29 by "java_re" with MS VC++ 10.0 (VS2010)
    
    time: Wed Dec 12 11:54:30 2018
    elapsed time: 2125 seconds (0d 0h 35m 25s)
    View Code

      降到2800,也依然jvm挂了!

       降到2600,jvm没挂,但是错误量较多,由于错误导致并发只跑到12000就未能继续进行了,数据不准:

       改了下失败策略后,2600,还是挂了,重启后可以扛住压力!

      压到4840个请求就挂了!

      2500并发,挂!

      2300并发,挂!

      2200并发,操作系统提示jre没有响应,被迫关闭java进程!

      2000并发,挂!

      1900并发,扛住了!cpu在80左右跳动!内存不变!

      看来,1900还行,极限就2000吧!

      好了,tomcat7看来是没辙了!

      换tomcat8 的 nio 看下效果!

    前提2:

      tomcat8, nio 模式运行!

     NIO介绍如下:

    Java NIO: Channels and Buffers(通道和缓冲区)

    标准的IO基于字节流和字符流进行操作的,而NIO是基于通道(Channel)和缓冲区(Buffer)进行操作,数据总是从通道读取到缓冲区中,或者从缓冲区写入到通道中。

    Java NIO: Non-blocking IO(非阻塞IO)

    Java NIO可以让你非阻塞的使用IO,例如:当线程从通道读取数据到缓冲区时,线程还是可以进行其他事情。当数据被写入到缓冲区时,线程可以继续处理它。从缓冲区写入通道也类似。

    Java NIO: Selectors(选择器)

    Java NIO引入了选择器的概念,选择器用于监听多个通道的事件(比如:连接打开,数据到达)。因此,单个的线程可以监听多个数据通道。

    直接从2000并发开测!

    并发2000-5组连续请求!

      server ok, 但是,仍然存在数错误,有几个并发请求卡死! TPS: 62.9, 平均响应: 14.7s。

      server 的cpu,内存情况如下:

      server ok, TPS: 77.2, error: 55%, 平均响应时间: 25s。只能说,服务端没死,但是基本已经不怎么可用了!

      到最后,我压到 6000 的并发时,server 仍然没有挂掉!

      所以,nio,是比较强悍的!

  • 相关阅读:
    leetcode 347. Top K Frequent Elements
    581. Shortest Unsorted Continuous Subarray
    leetcode 3. Longest Substring Without Repeating Characters
    leetcode 217. Contains Duplicate、219. Contains Duplicate II、220. Contains Duplicate、287. Find the Duplicate Number 、442. Find All Duplicates in an Array 、448. Find All Numbers Disappeared in an Array
    leetcode 461. Hamming Distance
    leetcode 19. Remove Nth Node From End of List
    leetcode 100. Same Tree、101. Symmetric Tree
    leetcode 171. Excel Sheet Column Number
    leetcode 242. Valid Anagram
    leetcode 326. Power of Three
  • 原文地址:https://www.cnblogs.com/yougewe/p/10118681.html
Copyright © 2011-2022 走看看