log4j多线程以及分文件输出日志

前段时间项目里因为多线程的问题对log4j进行了一下学习，今天有空汇总一下。

需求是有一个多线程程序，需要对每个线程的日志单独按级别存储。如下图所示。

 项目代码不方便发出，简单写一个demo。

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.ThreadContext;

public class TestLog {
    private static Logger log = LogManager.getLogger(TestLog.class);
    public static void main(String[] args) {

        Thread[] threads = new Thread[10];
        for(int i=0;i<10;i++){
            threads[i] = new TestThread("-"+i);
        }
        for(Thread thread : threads) {
            thread.start();
        }
    }
}

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.ThreadContext;

class TestThread extends Thread {

    private static Logger log = LogManager.getLogger(TestLog.class);
    public TestThread(String name) {
        super(name);
    }

    @Override
    public void run() {
        try {
            System.out.println(Thread.currentThread().getName());
            log.info(Thread.currentThread().getName());
            log.error("error:"+Thread.currentThread().getName());
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

一、log4j

1.每个线程中输出一份文件（此方法不通，留痕）

　　方法一是可以通过filter进行过滤。

　　但是考虑到需要对不同线程的日志按级别进行输出，还需要进一步改造。

　　因此考虑，首先在log4j.properties中对日志进行过滤，，将不同级别的日志输出到不同文件。可参考https://blog.csdn.net/liuxiao723846/article/details/69295428　　

log4j.rootLogger=info,stdout,infolog,errorlog
 
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=[%d{MM-dd HH:mm:ss}] [%p] [%c:%L] %m%n
 
log4j.appender.infolog = org.apache.log4j.DailyRollingFileAppender
log4j.appender.infolog.Threshold = INFO
log4j.appender.infolog.File = /data/logs/logtest/test.log
log4j.appender.infolog.layout = org.apache.log4j.PatternLayout
log4j.appender.infolog.layout.ConversionPattern = [%d{MM-dd HH:mm:ss}] [%p] [%c:%L] %m%n
log4j.appender.infolog.filter.infoFilter = org.apache.log4j.varia.LevelRangeFilter
log4j.appender.infolog.filter.infoFilter.LevelMin = INFO
log4j.appender.infolog.filter.infoFilter.LevelMax = INFO
 
log4j.appender.warnlog = org.apache.log4j.DailyRollingFileAppender
log4j.appender.warnlog.Threshold = WARN
log4j.appender.warnlog.File = /data/logs/logtest/test_warn.log
log4j.appender.warnlog.layout = org.apache.log4j.PatternLayout
log4j.appender.warnlog.layout.ConversionPattern = [%d{MM-dd HH:mm:ss}] [%p] [%c:%L] %m%n
log4j.appender.warnlog.filter.warnFilter = org.apache.log4j.varia.LevelRangeFilter
log4j.appender.warnlog.filter.warnFilter.LevelMin = WARN
log4j.appender.warnlog.filter.warnFilter.LevelMax=WARN

　　然后，在每个线程中对输出的日志名称进行重命名。对TestThread类进行改造

class TestThread extends Thread {

    private static final Logger log = Loggger.getLogger(TestLog.class);
    public TestThread(String name) {
        super(name);
    }

    @Override
    public void run() {
        try {
FileAppender appender = (FileAppender) log.getRootLogger().getAppender("errorlog");
            appender.setFile("C:/Test/SystemOutError"+Thread.currentThread().getName()+".log");
            appender.activateOptions();
            FileAppender appender = (FileAppender) log.getRootLogger().getAppender("infolog");
            appender.setFile("C:/Test/SystemOut"+Thread.currentThread().getName()+".log");
            appender.activateOptions();

            System.out.println(Thread.currentThread().getName());
            log.info(Thread.currentThread().getName());
            log.error("error:"+Thread.currentThread().getName());
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

　　但是，该方法能够按照线程生成不同的文件，并且文件分为不同的级别，例如下图。但是打开后发现error的内容都会存储在thread1的文件中，thread2中为空文件。

　　搜索了一下是因为log4j是单例模式，参考https://blog.csdn.net/guan0005/article/details/84139142，摘录其中一段。　

　　但log4j中有个陷阱：单例模式。在log4j中，配置的每个Logger，都只有一个实例。即在不同线程中，取到的同名Logger，全都是同一个实例。那么，之前的思路就　　　　行不通了。除非为每个线程配置一个特定的Logger，或者配置一定数量的Logger，组成一个Logger池，各线程争用Logger池中的Logger实例。但这两种方法要么不够灵活，要么逻辑复杂，并不是我想要的方案。

分析：
既然问题出在单例模式上，那我们只要在创建子线程的时候，为线程创建一个私有的Logger实例，不就行了吗。遗憾的是，log4j并没有提供创建全新Logger实例的接口给我们使用，Logger对象只能通过Logger.getLogger(*)方法获取，而不能new一个Logger对象。怎么办？只能重写Logger类，或创建Logger的子类了。

 　　上篇文章中给出了创建子类的方式，后续给出。

2.threadlocal

待补充。

二、log4j2

重点参考这篇文章https://my.oschina.net/u/2300159/blog/887687

导入两个包

<dependencies>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-api</artifactId>
            <version>2.12.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-core</artifactId>
            <version>2.12.0</version>
        </dependency>
    </dependencies>

配置log4j2.xml

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="OFF">
    <Appenders>
        <Routing name="Routing">
            <Routes pattern="$${ctx:ROUTINGKEY}">
                <!-- This route is chosen if ThreadContext has a value for ROUTINGKEY
                     (other than the value 'special' which had its own route above).
                     The value dynamically determines the name of the log file. -->
                <Route>
                    <RollingFile name="Rolling-${ctx:ROUTINGKEY}" fileName="logs/SystemOut-${ctx:ROUTINGKEY}.log"
                                 filePattern="./logs/${date:yyyy-MM}/${ctx:ROUTINGKEY}-SystemOut-%d{yyyy-MM-dd}-%i.log.gz">
                        <PatternLayout>
                            <pattern>%d{ISO8601} [%t] %p %c{3} - %m%n</pattern>
                        </PatternLayout>
                        <Policies>
                            <TimeBasedTriggeringPolicy interval="6" modulate="true"/>
                            <SizeBasedTriggeringPolicy size="10 MB"/>
                        </Policies>
                    </RollingFile>
                </Route>

            </Routes>
        </Routing>
        <!--很直白，Console指定了结果输出到控制台-->
        <Console name="ConsolePrint" target="SYSTEM_OUT">
            <PatternLayout pattern="%d{yyyy.MM.dd HH:mm:ss z} %t %-5level %class{36} %L %M - %msg%xEx%n"/>
        </Console>
    </Appenders>
    <Loggers>
        <!-- 级别顺序（低到高）：TRACE < DEBUG < INFO < WARN < ERROR < FATAL -->
        <Root level="DEBUG" includeLocation="true">
            <!--AppenderRef中的ref值必须是在前面定义的appenders-->
            <AppenderRef ref="Routing"/>
            <AppenderRef ref="ConsolePrint"/>
        </Root>
    </Loggers>
</Configuration>

该xml中有一个变量${ctx:ROUTINGKEY}，此变量需要在代码中设定，对TestThread进行修改

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.ThreadContext;
import org.apache.logging.log4j.core.appender.FileAppender;

class TestThread extends Thread {

    private static Logger log = LogManager.getLogger(TestLog.class);
    public TestThread(String name) {
        super(name);
    }

    @Override
    public void run() {
        ThreadContext.put("ROUTINGKEY", Thread.currentThread().getName());
        try {
            System.out.println(Thread.currentThread().getName());
            log.info(Thread.currentThread().getName());
            log.error("error:"+Thread.currentThread().getName());
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

点击运行，便可以按照线程生成日志。

 

 该部分代码可以

2.不使用ThreadContext

首先修改xml配置

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="OFF">
    <Appenders>
        <Routing name="Routing">
            <Routes pattern="$${thread:threadName}">
                <Route>
                    <RollingFile name="logFile-${thread:threadName}"
                                 fileName="logs/concurrent-${thread:threadName}.log"
                                 filePattern="logs/concurrent-${thread:threadName}-%d{MM-dd-yyyy}-%i.log">
                        <PatternLayout pattern="%d %-5p [%t] %C{2} - %m%n"/>
                        <Policies>
                            <SizeBasedTriggeringPolicy size="50 MB"/>
                        </Policies>
                        <DefaultRolloverStrategy max="100"/>
                    </RollingFile>
                </Route>
            </Routes>
        </Routing>
        <Async name="async" bufferSize="1000" includeLocation="true">
            <AppenderRef ref="Routing"/>
        </Async>
        <!--很直白，Console指定了结果输出到控制台-->
        <Console name="ConsolePrint" target="SYSTEM_OUT">
            <PatternLayout pattern="%d{yyyy.MM.dd HH:mm:ss z} %t %-5level %class{36} %L %M - %msg%xEx%n"/>
        </Console>
    </Appenders>
    <Loggers>
        <Root level="info" includeLocation="true">
            <AppenderRef ref="async"/>
            <AppenderRef ref="ConsolePrint"/>
        </Root>
    </Loggers>
</Configuration>

然后实现StrLookup

import org.apache.logging.log4j.core.LogEvent;
import org.apache.logging.log4j.core.config.plugins.Plugin;
import org.apache.logging.log4j.core.lookup.StrLookup;

@Plugin(name = "thread", category = StrLookup.CATEGORY)
public class ThreadLookup implements StrLookup {
    @Override
    public String lookup(String key) {
        return Thread.currentThread().getName();
    }

    @Override
    public String lookup(LogEvent event, String key) {
        return event.getThreadName() == null ? Thread.currentThread().getName()
                : event.getThreadName();
    }

}

ThreadLookup类可以获取到代码中的线程名称，所以我们要对每个线程分别命名，用于区分日志名称。可以通过Thread.currentThread().setName(name);设置。
首先主函数

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.ThreadContext;

import java.util.concurrent.*;

public class TestLog {
    private static Logger log = LogManager.getLogger(TestLog.class);
    public static void main(String[] args) {
        testValidate();

//        Thread[] threads = new Thread[10];
//        for(int i=0;i<10;i++){
//            threads[i] = new TestThread("-"+i);
//        }
//        for(Thread thread : threads) {
//            thread.start();
//        }

//        new Thread(() -> {
//            log.info("info");
//            log.debug("debug");
//            log.error("error");
//            ThreadContext.remove("ROUTINGKEY");
//        }).start();
//        new Thread(() -> {
//            log.info("info");
//            log.debug("debug");
//            log.error("error");
//            ThreadContext.remove("ROUTINGKEY");
//        }).start();
    }
    private static  void testValidate(){
        int threadNum = 3;
        int totalNum = 3;
        ConcurrentLinkedDeque<Future> futureList = new ConcurrentLinkedDeque<>();
        CountDownLatch begin = new CountDownLatch(1);
        ExecutorService executor = Executors.newFixedThreadPool(threadNum);
        for(int i=0;i<totalNum;i++){
            Future submit = executor.submit(new TestThread("name"+i));
            futureList.add(submit);
        }
        begin.countDown();
        executor.shutdown();
    }
}

主函数对TestThread进行调用，在TestThread中完成对于线程的命名。

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.ThreadContext;
import org.apache.logging.log4j.core.appender.FileAppender;

import java.util.concurrent.Callable;

class TestThread implements Callable {

    private static Logger log = LogManager.getLogger(TestLog.class);
    String name;
    public TestThread(String name) {
        this.name = name;
    }

//    @Override
//    public void run() {
////        ThreadContext.put("ROUTINGKEY", Thread.currentThread().getName());
//        try {
//            System.out.println(Thread.currentThread().getName());
//            log.info(Thread.currentThread().getName());
//            log.error("error:"+Thread.currentThread().getName());
//            Upload upload = new Upload();
//            upload.testUpload(Thread.currentThread().getName());
//        } catch (Exception e) {
//            e.printStackTrace();
//        }
//    }
    @Override
    public Object call(){
        Thread.currentThread().setName(name);
        Upload upload = new Upload();
        upload.testUpload(name);

        return null;
    }
}

TestThread调用upload方法，进行日志的输出。

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;

public class Upload {
    private static Logger log = LogManager.getLogger(Upload.class);
    public void testUpload(String name){
        log.info("in testUpload" + name);
    }
}