Java Web c3p0 pool池泄漏优化与日志分析

问题跟踪：

近期在整合SSH(spring、springmvc、hibernate)项目，提供给第三方服务。每当调用内存池达到上限之后，外界调用服务直接失败，提示[cannot open connection]。经过九牛二虎之力，才找到是c3p0 pool池内存泄漏的原因。类似jdbc，session使用之后需要手动关闭：

public static Basicinfo getBean(Integer iduser) {
    Basicinfo u = null;
    Transaction tx = null;
    Session sess = NewHibernateUtil.getSessionFactory().openSession();
    try {

        Query q = sess.createQuery("from Basicinfo where iduser=" + iduser);
        u = (Basicinfo) q.uniqueResult();
        if (u == null) {
            tx = sess.beginTransaction();   //line 69
            u = new Basicinfo();
            u.setIduser(iduser);
            tx.commit();
        }           
    } catch (Exception ex) {
        ex.printStackTrace();
        if(tx != null) {
            try {
             tx.rollback();
            } catch(Exception e){e.printStackTrace;}
        }
    } finally {
        if(sess!=null) {
         sess.close();
        }
    }
    return u;
}

 SSM框架中spring进行的事务管理，事务返回之后会自动关闭session。此时获得session的方式包含三种：

1、getCurrentSession()
获得当前会话中的session,该session有容器自行维护管理，spring可以代理事务。

2、this.getSession()
从当前的执行中获得或创建一个hibernate的session对象，自己关闭，释放连接资源，spring可以代理事务。

3、openSession()
调用函数自行创建一个数据库的连接，并将其打开，在使用Spring操作非查询语句的请况下，Spring的事务对该session对象不起到事务管理的作用，所以该session对象应当由程序员自己关闭，释放连接资源。

应用程序中dao层使用spring的HibernateDaoSupport模板,通过spring来管理session生命周期,则首选getCurrentSession() ;

跟踪 c3p0 pool池连接：

<property name="maxIdleTime">
    <!--最大空闲时间,超过空闲时间未使用则连接被丢弃。若为0则永不丢弃。Default: 0 -->
          <value>30</value>
</property>
<property name="unreturnedConnectionTimeout">
　　　　　<!-- 自动超时回收Connection 单位秒 -->
            <value>40</value>
</property>
 <property name="debugUnreturnedConnectionStackTraces">
             <value>true</value>
</property>        

 参数一：maxIdleTime 最大空闲时间,超过空闲时间未使用则连接被丢弃。若为0则永不丢弃。Default: 0 。

参数二：debugUnreturnedConnectionStackTraces 参数肯定是设置为true的，打印堆栈跟踪信息。

参数三：unreturnedConnectionTimeout 在连接被应用程序 checkout后指定时间内未checkin则由连接缓冲池执行kill操作。maxIdleTime设置30秒，unreturnedConnectionTimeout设置40秒，达到最大存活，链接不能被缓冲池正常关闭，则出现泄漏。此时，再过10秒后连接缓冲池主动执行kill。 

debugUnreturnedConnectionStackTraces 和 unreturnedConnectionTimeout 详细介绍 http://www.mchange.com/projects/c3p0/#configuring_to_debug_and_workaround_broken_clients

打印信息：

09:22:17,909 INFO  [com.mchange.v2.resourcepool.BasicResourcePool] (Timer-1) A checked-out resource is overdue, and will be destroyed: com.mchange.v2.c3p0.impl.NewPooledConnection@1e029db
09:22:17,911 INFO  [com.mchange.v2.resourcepool.BasicResourcePool] (Timer-1) Logging the stack trace by which the overdue resource was checked-out.: java.lang.Exception: DEBUG ONLY: Overdue resource check-out stack trace.
    at com.mchange.v2.resourcepool.BasicResourcePool.checkoutResource(BasicResourcePool.java:506) [c3p0-0.9.1.2.jar:0.9.1.2]
    at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C3P0PooledConnectionPool.java:525) [c3p0-0.9.1.2.jar:0.9.1.2]
    at com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource.getConnection(AbstractPoolBackedDataSource.java:128) [c3p0-0.9.1.2.jar:0.9.1.2]
    at org.springframework.orm.hibernate3.LocalDataSourceConnectionProvider.getConnection(LocalDataSourceConnectionProvider.java:83) [spring-orm-3.2.8.RELEASE.jar:3.2.8.RELEASE]
    at org.hibernate.jdbc.ConnectionManager.openConnection(ConnectionManager.java:446) [hibernate3.jar:]
    at org.hibernate.jdbc.ConnectionManager.getConnection(ConnectionManager.java:167) [hibernate3.jar:]
    at org.hibernate.jdbc.AbstractBatcher.prepareQueryStatement(AbstractBatcher.java:161) [hibernate3.jar:]
    at org.hibernate.loader.Loader.prepareQueryStatement(Loader.java:1573) [hibernate3.jar:]
    at org.hibernate.loader.Loader.doQuery(Loader.java:696) [hibernate3.jar:]
    at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:259) [hibernate3.jar:]
    at org.hibernate.loader.Loader.doList(Loader.java:2228) [hibernate3.jar:]
    at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2125) [hibernate3.jar:]
    at org.hibernate.loader.Loader.list(Loader.java:2120) [hibernate3.jar:]
    at org.hibernate.loader.custom.CustomLoader.list(CustomLoader.java:312) [hibernate3.jar:]
    at org.hibernate.impl.SessionImpl.listCustomQuery(SessionImpl.java:1722) [hibernate3.jar:]
    at org.hibernate.impl.AbstractSessionImpl.list(AbstractSessionImpl.java:165) [hibernate3.jar:]
    at org.hibernate.impl.SQLQueryImpl.list(SQLQueryImpl.java:175) [hibernate3.jar:]
    at com.pec.dao.impl.ServiceIntTypeDaoImpl.getIntTypes(ServiceIntTypeDaoImpl.java:45) [classes:]
    at com.pec.dao.impl.CacheDaoImpl.uploadInterfaceCache(CacheDaoImpl.java:80) [classes:]
    at com.pec.service.impl.CacheServiceImpl.uploadInterfaceCache(CacheServiceImpl.java:135) [classes:]
    at com.pec.service.impl.CacheServiceImpl$$FastClassBySpringCGLIB$$53861b3b.invoke(<generated>) [spring-core-3.2.8.RELEASE.jar:]
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) [spring-core-3.2.8.RELEASE.jar:3.2.8.RELEASE]
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:700) [spring-aop-3.2.8.RELEASE.jar:3.2.8.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) [spring-aop-3.2.8.RELEASE.jar:3.2.8.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:96) [spring-tx-3.2.8.RELEASE.jar:3.2.8.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:260) [spring-tx-3.2.8.RELEASE.jar:3.2.8.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:94) [spring-tx-3.2.8.RELEASE.jar:3.2.8.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) [spring-aop-3.2.8.RELEASE.jar:3.2.8.RELEASE]
    at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91) [spring-aop-3.2.8.RELEASE.jar:3.2.8.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) [spring-aop-3.2.8.RELEASE.jar:3.2.8.RELEASE]
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:633) [spring-aop-3.2.8.RELEASE.jar:3.2.8.RELEASE]
    at com.pec.service.impl.CacheServiceImpl$$EnhancerBySpringCGLIB$$4db83b52.uploadInterfaceCache(<generated>) [spring-core-3.2.8.RELEASE.jar:]
    at com.pec.controller.PecInterfaceController.pecInterface(PecInterfaceController.java:138) [classes:]
    at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source) [:1.7.0_25]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_25]
    at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_25]
    at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:215) [spring-web-3.2.8.RELEASE.jar:3.2.8.RELEASE]
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:132) [spring-web-3.2.8.RELEASE.jar:3.2.8.RELEASE]
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104) [spring-webmvc-3.2.8.RELEASE.jar:3.2.8.RELEASE]
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:745) [spring-webmvc-3.2.8.RELEASE.jar:3.2.8.RELEASE]
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:686) [spring-webmvc-3.2.8.RELEASE.jar:3.2.8.RELEASE]
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:80) [spring-webmvc-3.2.8.RELEASE.jar:3.2.8.RELEASE]
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:925) [spring-webmvc-3.2.8.RELEASE.jar:3.2.8.RELEASE]
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:856) [spring-webmvc-3.2.8.RELEASE.jar:3.2.8.RELEASE]
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:953) [spring-webmvc-3.2.8.RELEASE.jar:3.2.8.RELEASE]
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:855) [spring-webmvc-3.2.8.RELEASE.jar:3.2.8.RELEASE]
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final]
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:829) [spring-webmvc-3.2.8.RELEASE.jar:3.2.8.RELEASE]
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) [jbossweb-7.0.13.Final.jar:]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) [jbossweb-7.0.13.Final.jar:]
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88) [spring-web-3.2.8.RELEASE.jar:3.2.8.RELEASE]
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-3.2.8.RELEASE.jar:3.2.8.RELEASE]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) [jbossweb-7.0.13.Final.jar:]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) [jbossweb-7.0.13.Final.jar:]
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) [jbossweb-7.0.13.Final.jar:]
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) [jbossweb-7.0.13.Final.jar:]
    at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) [jboss-as-web-7.1.1.Final.jar:7.1.1.Final]
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) [jbossweb-7.0.13.Final.jar:]
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) [jbossweb-7.0.13.Final.jar:]
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) [jbossweb-7.0.13.Final.jar:]
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) [jbossweb-7.0.13.Final.jar:]
    at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) [jbossweb-7.0.13.Final.jar:]
    at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) [jbossweb-7.0.13.Final.jar:]
    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:518) [jbossweb-7.0.13.Final.jar:]
    at org.jboss.threads.SimpleDirectExecutor.execute(SimpleDirectExecutor.java:33)
    at org.jboss.threads.QueueExecutor.runTask(QueueExecutor.java:801)
    at org.jboss.threads.QueueExecutor.access$100(QueueExecutor.java:45)
    at org.jboss.threads.QueueExecutor$Worker.run(QueueExecutor.java:842)
    at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25]
    at org.jboss.threads.JBossThread.run(JBossThread.java:122)

核心问题发现dao层使用openSession()获得session而没有进行手动关闭：

@Override
    public List<String> getIntTypes(JSONObject js) {
        // 创建相关SQL语句
        StringBuffer sb = new StringBuffer();

        sb.append("select  ");
        sb.append("        int_type ");
        sb.append("  from ");
        sb.append("        interface_admin.interface_service_int_type");
        sb.append("  where ");
        sb.append("        service_id ='" + js.getString("p_service_id")+ "'");
        sb.append("        order by int_type_level asc");

        SQLQuery q = this.getSessionFactory().openSession().createSQLQuery(sb.toString()) ;
        
        /** 推荐使用
         * SQLQuery q = this.getSessionFactory().getCurrentSession().createSQLQuery(sb.toString()) ; 
         * SQLQuery q = this.getSession().createSQLQuery(sb.toString()) ;
         * 
         */
        
        if ( !CollectionUtils.isNullOrEmpty(q) ) {
            @SuppressWarnings("unchecked")
            List<String> beanList = q.list();
            return beanList;
        }

        return null;
    }

项目结构：

spring事务管理配置：

<!--5、Spring 配置声明式事物 -->
    <!-- 配置事务 -->
    <bean id="transactionManager"
        class="org.springframework.orm.hibernate3.HibernateTransactionManager">
        <property name="sessionFactory" ref="sessionFactory"></property>
    </bean>
    
    <!-- 配置事务范围 -->
    <tx:advice id="txAdvice" transaction-manager="transactionManager">
        <tx:attributes>
            <tx:method name="get*" read-only="false" propagation="NOT_SUPPORTED" />
            <tx:method name="find*" read-only="false" propagation="NOT_SUPPORTED" />
            <tx:method name="download*" propagation="REQUIRED" />
            <tx:method name="save*" propagation="REQUIRED" />
            <tx:method name="upload*" propagation="REQUIRED" />
            <tx:method name="update*" propagation="REQUIRED" />
            <tx:method name="insert*" propagation="REQUIRED" />
            <tx:method name="delete*" propagation="REQUIRED" />
            <tx:method name="create*" propagation="REQUIRED" />
            <tx:method name="anscy*" propagation="REQUIRED" />
        </tx:attributes>
    </tx:advice>
    
    <!-- 定义切面 -->
    <!-- <aop:config>里面有一个"proxy-target-class"属性，true，那么基于类的代理将起作用;false或者这个属性被省略，那么基于接口的代理将起作用 -->
    <aop:config proxy-target-class="true">
        <aop:pointcut id="pointcut" expression="execution(* com.pec.service.impl.*Impl.*(..))"  />
        <aop:advisor advice-ref="txAdvice" pointcut-ref="pointcut" />
    </aop:config>

此时spring事务管理范围是： com.pec.service.impl包下，以Impl结尾的类下所有方法。事务返回之后，spring会对连接进行管理。