How do I get multiple process to log to the same file?
Before you even start trying any of the alternatives provided, ask yourself whether you really need to have multiple processes log to the same file, then don't do it ;-).
FileAppender offers pluggable locking models for this usecase but all existing implementations have issues and drawbacks.
By default the FileAppender holds an exclusive write lock on the log file while it is logging. This prevents other processes from writing to the file. This model is known to break down with (at least on some versions of) Mono on Linux and log files may get corrupted as soon as another process tries to access the log file.
MinimalLock only acquires the write lock while a log is being written. This allows multiple processes to interleave交错 writes to the same file, albeit即使 with a considerable loss in performance.
InterProcessLock doesn't lock the file at all but synchronizes using a system wide Mutex. This will only work if all processes cooperate (and use the same locking model). The acquisition and release of a Mutex for every log entry to be written will result in a loss of performance, but the Mutex is preferable to the use of MinimalLock.
If you use RollingFileAppender things become even worse as several process may try to start rolling the log file concurrently. RollingFileAppender completely ignores the locking model when rolling files, rolling files is simply not compatible with this scenario.
A better alternative is to have your processes log to RemotingAppenders. Using the RemoteLoggingServerPlugin (or IRemoteLoggingSink) a process can receive all the events and log them to a single log file. One of the examples shows how to use the RemoteLoggingServerPlugin.
log4net-2.0.8examples et2.0Remoting
默认的log4net的lock配置是写独占的
Intermittent log4net RollingFileAppender locked file issue
在appender中，配置lock 如下
<lockingModel type="log4net.Appender.FileAppender+MinimalLock"/>
就可以确保其他进程可以操作文件
但是不足之处是，会有性能问题
<lockingModel type="log4net.Appender.FileAppender+MinimalLock" />
to your <appender /> element. There is some performance impact because this means that log4net will lock the file, write to it, and unlock it for each write operation (as opposed to the default behavior, which acquires and holds onto the lock for a long time).
One implication of the default behavior is that if you're using it under a Web site that is being executed under multiple worker processes running on the same machine, each one will try to acquire and hold onto that lock indefinitely, and two of them are just going to lose. Changing the locking model to the minimal lock works around this issue.
(When debugging, ungraceful terminations and spinning up lots of new worker processes is exactly the type of thing that's likely to happen.)
Good luck!
FileAppender.LockingModel Property
Property Value
The LockingModel used to lock the file.
Remarks
Gets or sets the LockingModel used to handle locking of the file.
There are three built in locking models, FileAppender.ExclusiveLock, FileAppender.MinimalLock and FileAppender.InterProcessLock .
The first locks the file from the start of logging to the end,
the second locks only for the minimal amount of time when logging each message and
the last synchronizes processes using a named system wide Mutex.
The default locking model is the FileAppender.ExclusiveLock.
FileAppender.ExclusiveLock Class
Remarks
Open the file once for writing and hold it open until CloseFile is called. Maintains an exclusive lock on the file during this time.
FileAppender.MinimalLock Class
Remarks
Opens the file once for each AcquireLock/ReleaseLock cycle, thus holding the lock for the minimal amount of time. This method of locking is considerably slower than FileAppender.ExclusiveLock but allows other processes to move/delete the log file whilst logging continues.
Log4net Thread-Safe but not Process-Safe
How to address the issue
There are two ways that we can configure the RollingFileAppender to work in a web garden. One option is to configure it to use what is known as minimal lock on the log file. With this option log4net will only lock the file for the duration of each log operation. This option reduces the chances that more than one process will collide as they keep the file locked to a minimum. However, there is no guarantee that under load they might actually collide. Additionally there is a performance penalty in opening and closing the file on each log entry. I am not sure if the penalty is too high but it's worth testing if you decide to use this option. Below is an example of how to configure the RollingFileAppender to use minimal locking (notice the lockingModel setting in line 2.)
<appender name="LogFileAppender" type="log4net.Appender.RollingFileAppender,log4net"> <lockingModel type="log4net.Appender.FileAppender+MinimalLock" /> <param name="File" value=".\App_Data\LogFile" /> <param name="AppendToFile" value="true" /> <datePattern value=".yyyy-MM-dd'.txt'" /> <rollingStyle value="Composite" /> <maxSizeRollBackups value="5" /> <maximumFileSize value="100KB" /> <staticLogFileName value="true" /> <layout type="log4net.Layout.PatternLayout"> <conversionPattern value="[LOGENTRY] %date %-5level %logger{2} - %message %newline" /> <layout> <appender>
The second option to take care of the problem is to make sure that each IIS worker process writes to its own file so that there are no collisions at all. With this option if the web garden is configured to use three worker processes then you'll have three simultaneous log files (one for each active worker process.) Each of them can use an exclusive lock on the file as these files are not shared across processes. This method is guaranteed not to have collisions and allows you to keep using the fast exclusive lock method. The disadvantage is that now you have multiple log files to look at rather than a single consolidated one. Below is an example on how to configure the RollingFileAppender to create one file for each process. Notice how line 2 now has been configured to use a PatternString to name the file and the inclusion of pattern %processid to make each file unique by each process ID.
<appender name="LogFileAppender" type="log4net.Appender.RollingFileAppender,log4net"> <file type="log4net.Util.PatternString" value=".\App_Data\Log[%processid]" /> <param name="AppendToFile" value="true" /> <datePattern value=".yyyy-MM-dd'.txt'" /> <rollingStyle value="Composite" /> <maxSizeRollBackups value="5" /> <maximumFileSize value="100KB" /> <staticLogFileName value="true" /> <layout type="log4net.Layout.PatternLayout"> <conversionPattern value="[LOGENTRY] %date %-5level %logger{2} - %message %newline" /> <layout> <appender>
But I thought log4net was thread-safe?
Log4net is indeed thread-safe as it is clearly indicated on log4net's FAQ. A lot of people get confused on this because the documentation for the RollingFileAppender says that the appender is not thread safe. In reality the log4net code takes this into account (see here and here) and makes sure logging is thread-safe. Some people have even run tests to prove this.
Update Dec/2014: Apparently some people have found that log4net can be "deadlock-prone" and they have logged an issue about this. Thanks John Zabroski for the heads up!
However, the fact that log4net is thread-safe does not mean that is process-safe to write to a single file. This is not log4net's fault, there is no such thing as the ability to have multiple processes write concurrently to the same text file. We use relational databases (not text files) when we need to have multiple processes write to a single repository of data.
In summary
If you are using log4net's RollingFileAppender in an environment where IIS is configured to use multiple worker processes (i.e. a web garden) you must ensure that the RollingFileAppender uses one file per worker process rather than a single file.