遍历多个tomcat日志文件，找出含有ERROR 和Exception 的日志，并把该行日志输出到另一个文件中：（这里为了体现python模块导入的知识，所有建立了多个文件夹和模块）
项目结构：
consetting.py:
# 日志文件目录 F_PATH = r'C:UsersshenpingPycharmProjectsShenping_TESTday_5scriptglivelogs' # 错误日志存储目录 D_PATH = r'C:UsersshenpingPycharmProjectsShenping_TESTday_5scriptglivedata'
rwfile.py:
def op_file(filename,content =None): f = open(filename,'a+',encoding='utf-8') f.seek(0) if content: f.writelines(content) res = None else: res = f.readlines() f.close() return res
operationlog.py:
import re,os,datetime from lib.rwfile import op_file # 遍历日志文件： # 找出含有 error 或 exception 的行，按格式写入文件 def operation_log(filename,f_path,d_path): index = 0 log_file = os.path.join(f_path, filename) f = op_file(log_file) lis = [] # 存储错误日志 for j in f: index += 1 m = re.search('error',j,re.IGNORECASE) n = re.search('exception',j,re.IGNORECASE) if m or n: data_file = os.path.join(d_path, str(datetime.date.today())+'_tomcat.log') log_str = "文件名："+filename+" 第"+str(index)+"行："+j+' ' lis.append(log_str) op_file(data_file,lis)
start.py:
# 启动文件 import os,sys cur_path = os.path.abspath(__file__) base_dir = os.path.dirname(os.path.dirname(cur_path)) sys.path.insert(0,base_dir) from conf.consetting import F_PATH,D_PATH from lib.operationlog import operation_log log_name = os.listdir(F_PATH) for x in log_name: operation_log(x,F_PATH,D_PATH) print('========================================华丽的分割线=========================================================')
运行后结果：
文件名：catalina.2017-10-21.log 第5行：21-Oct-2017 14:16:32.118 ERROR [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [Abandoned connection cleanup thread] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread: 文件名：catalina.2017-10-21.log 第9行：21-Oct-2017 14:16:32.119 ERROR [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [threadDeathWatcher-2-1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread: 文件名：catalina.2017-10-21.log 第14行：21-Oct-2017 14:16:32.120 ERROR [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [logback-1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread: 文件名：catalina.2017-10-21.log 第515行：21-Oct-2017 16:08:39.217 ERROR [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [glive-task-pool-1-thread-12] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread: 文件名：catalina.2017-10-22.log 第216行：22-Oct-2017 15:17:49.169 Exception [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [Abandoned connection cleanup thread] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread: 文件名：catalina.2017-10-22.log 第248行：22-Oct-2017 15:17:54.089 Exception [main] org.apache.tomcat.util.digester.SetPropertiesRule.begin [SetPropertiesRule]{Server/Service/Engine/Host/Context} Setting property 'debug' to '0' did not find a matching property. 文件名：catalina.2017-10-22.log 第249行：22-Oct-2017 15:17:54.113 Exception [main] org.apache.tomcat.util.digester.SetPropertiesRule.begin [SetPropertiesRule]{Server/Service/Engine/Host/Valve} Setting property 'resolveHosts' to 'false' did not find a matching property. 文件名：catalina.2017-10-23.log 第3行：23-Oct-2017 10:51:55.461 IllegalStateException [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc The web application [ROOT] registered the JDBC driver [com.mysql.jdbc.Driver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered. 文件名：catalina.2017-10-23.log 第4行：23-Oct-2017 10:51:55.462 ERROR [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc The web application [ROOT] registered the JDBC driver [com.mysql.fabric.jdbc.FabricMySQLDriver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered. 文件名：catalina.2017-10-23.log 第9行：23-Oct-2017 10:51:55.464 IllegalStateException [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [threadDeathWatcher-3-1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread: 文件名：catalina.2017-10-23.log 第3356行：23-Oct-2017 21:47:50.795 IllegalStateException [logback-1] org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading Illegal access: this web application instance has been stopped already. Could not load [ch.qos.logback.core.spi.ContextAwareBaseBeanINFO]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access. 文件名：catalina.2017-10-23.log 第3357行： java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [ch.qos.logback.core.spi.ContextAwareBaseBeanINFO]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access. 文件名：catalina.2017-10-23.log 第3398行：23-Oct-2017 21:47:50.800 ERROR [logback-1] org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading Illegal access: this web application instance has been stopped already. Could not load [java.lang.ObjectBeanINFO]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access. 文件名：catalina.2017-10-23.log 第3399行： java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [java.lang.ObjectBeanINFO]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access. 文件名：catalina.2017-10-23.log 第3444行：23-Oct-2017 21:47:50.802 ERROR [logback-1] org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading Illegal access: this web application instance has been stopped already. Could not load [java.lang.ObjectCustomizer]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access. 文件名：catalina.2017-10-23.log 第3445行： java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [java.lang.ObjectCustomizer]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access. 文件名：catalina.2017-10-23.log 第3491行： java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [ch.qos.logback.core.spi.ContextAwareBaseCustomizer]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access. 文件名：catalina.2017-10-23.log 第3532行： java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [ch.qos.logback.core.rolling.RollingPolicyBaseCustomizer]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access. 文件名：catalina.2017-10-23.log 第3570行： java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [ch.qos.logback.core.rolling.TimeBasedRollingPolicyCustomizer]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access. 文件名：catalina.2017-10-23.log 第3605行： java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [ch.qos.logback.core.rolling.helper.DateTokenConverter]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access. 文件名：catalina.2017-10-23.log 第3637行： java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [ch.qos.logback.core.status.INFOStatus]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access. 文件名：catalina.2017-10-23.log 第3667行： java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [ch.qos.logback.core.status.INFOStatus]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access. 文件名：catalina.2017-10-24.log 第5行：24-Oct-2017 08:02:51.428 IllegalStateException [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [Abandoned connection cleanup thread] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread: 文件名：catalina.2017-10-24.log 第520行：24-Oct-2017 14:42:09.350 Exception [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [Abandoned connection cleanup thread] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread: 文件名：catalina.2017-10-24.log 第670行：24-Oct-2017 14:42:09.374 Exception [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [ROOT] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@e5d976d]) and a value of type [io.netty.util.internal.InternalThreadLocalMap] (value [io.netty.util.internal.InternalThreadLocalMap@b0bd2a4]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak. 文件名：catalina.2017-10-24.log 第682行：24-Oct-2017 14:42:09.376 Exception [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [ROOT] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@55c7591]) and a value of type [io.grpc.Context] (value [io.grpc.Context@390c5a57]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak. 文件名：catalina.2017-10-24.log 第704行：24-Oct-2017 14:42:09.378 Exception [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [ROOT] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@55c7591]) and a value of type [io.grpc.Context] (value [io.grpc.Context@390c5a57]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.