Busybox的syslogd认识与使用

关键词：rcS、start-stop-daemon、syslogd、syslog()、klogd、klogctl()、syslog.conf、/dev/log、facility/level等等。

syslog用来记录应用程序或者硬件设备的日志；通过syslogd这个进程记录系统有关事件记录，也可以记录应用程序运作事件。

syslogd日志记录器由两个守护进程（klogd，syslogd）和一个配置文件（syslog.conf）组成。

syslogd和klogd是很有意思的守护进程，syslogd是一个分发器，它将接收到的所有日志按照/etc/syslog.conf的配置策略发送到这些日志应该去的地方，当然也包括从klogd接收到的日志。

klogd首先接收内核的日志，然后将之发送给syslogd。

klogd不使用配置文件，它负责截获内核消息，它既可以独立使用也可以作为syslogd的客户端运行。

syslogd默认使用/etc/syslog.conf作为配置文件，负责截获应用程序消息，还可以截获klogd向其转发的内核消息。支持internet/unix domain sockets的特性使得这两个工具可以用于记录本地和远程的日志。

 

图：系统日志概览

注：引用自《Linux/UNIX系统编程》第37.5.1 概述。

1.Busybox中的syslog

在busybox中执行make menuconfig，或者在buildroot中执行make busybox-menuconfig都可以对busybox进行配置。

打开menuconfig之后，进入System Logging Utilities：

syslog功能主要包括两大部分syslogd/klogd和配置文件syslog.conf。前者是daemon程序，后者是针对daemon的配置文件。

2.syslogd配置syslog.conf

syslog.conf包括对syslogd的配置，每个配置包括两个域：selector和action。

2.1 syslog.conf的selector

selector有两部分组成：facility和level，两者之间用"."分隔；多个facility或者level之间用","分隔；多个selector之间用";"分隔。

同时selector中可以使用通配符：

Name	Facility	Level
err/warn/...	任何一个facility	显示大于等于此级别的log
*	任何一个facility	任何一个级别
=	无效	只有该级别log才生效
!	无效	除了该级别，其他log生效
None	无效	不存储该facility任何log

2.2 syslog.conf的action

action主要是当信息满足selector时，该日志消息的输出方向。

一般分为三种：(1)存储到普通文件中；(2)写入到管道中；(3)远程转发到其它主机上。

常规文件	管道文件	远程转发
普通的文件名：/xx/bb	|文件名	@hostname

2.3 Facility和Level

提到syslog.con中的facility和level，需要看一下他在代码中对应关系。

相关facility和level都在syslog.h中定义，两这个共同组成一个32位数值；低3bit(0-3)表示level，其余部分(3-24)表示facility。

facilitynames[]中的名称对应syslog.conf中的facility，后面的值对应syslog()中的facility代码。

/* facility codes */
#define    LOG_KERN    (0<<3)    /* kernel messages */
#define    LOG_USER    (1<<3)    /* random user-level messages */
#define    LOG_MAIL    (2<<3)    /* mail system */
#define    LOG_DAEMON    (3<<3)    /* system daemons */
#define    LOG_AUTH    (4<<3)    /* security/authorization messages */
#define    LOG_SYSLOG    (5<<3)    /* messages generated internally by syslogd */
#define    LOG_LPR        (6<<3)    /* line printer subsystem */
#define    LOG_NEWS    (7<<3)    /* network news subsystem */
#define    LOG_UUCP    (8<<3)    /* UUCP subsystem */
#define    LOG_CRON    (9<<3)    /* clock daemon */
#define    LOG_AUTHPRIV    (10<<3)    /* security/authorization messages (private) */
#define    LOG_FTP        (11<<3)    /* ftp daemon */

    /* other codes through 15 reserved for system use */
#define    LOG_LOCAL0    (16<<3)    /* reserved for local use */
#define    LOG_LOCAL1    (17<<3)    /* reserved for local use */
#define    LOG_LOCAL2    (18<<3)    /* reserved for local use */
#define    LOG_LOCAL3    (19<<3)    /* reserved for local use */
#define    LOG_LOCAL4    (20<<3)    /* reserved for local use */
#define    LOG_LOCAL5    (21<<3)    /* reserved for local use */
#define    LOG_LOCAL6    (22<<3)    /* reserved for local use */
#define    LOG_LOCAL7    (23<<3)    /* reserved for local use */

#define    LOG_NFACILITIES    24    /* current number of facilities */
#define    LOG_FACMASK    0x03f8    /* mask to extract facility part */
                /* facility of pri */
#define    LOG_FAC(p)    (((p) & LOG_FACMASK) >> 3)

#ifdef SYSLOG_NAMES
CODE facilitynames[] =
  {
    { "auth", LOG_AUTH },
    { "authpriv", LOG_AUTHPRIV },
    { "cron", LOG_CRON },
    { "daemon", LOG_DAEMON },
    { "ftp", LOG_FTP },
    { "kern", LOG_KERN },
    { "lpr", LOG_LPR },
    { "mail", LOG_MAIL },
    { "mark", INTERNAL_MARK },        /* INTERNAL */
    { "news", LOG_NEWS },
    { "security", LOG_AUTH },        /* DEPRECATED */
    { "syslog", LOG_SYSLOG },
    { "user", LOG_USER },
    { "uucp", LOG_UUCP },
    { "local0", LOG_LOCAL0 },
    { "local1", LOG_LOCAL1 },
    { "local2", LOG_LOCAL2 },
    { "local3", LOG_LOCAL3 },
    { "local4", LOG_LOCAL4 },
    { "local5", LOG_LOCAL5 },
    { "local6", LOG_LOCAL6 },
    { "local7", LOG_LOCAL7 },
    { NULL, -1 }
  };
#endif

prioritynames定义了syslog.conf中等级和syslog()关系：

#define    LOG_EMERG    0    /* system is unusable */
#define    LOG_ALERT    1    /* action must be taken immediately */
#define    LOG_CRIT    2    /* critical conditions */
#define    LOG_ERR        3    /* error conditions */
#define    LOG_WARNING    4    /* warning conditions */
#define    LOG_NOTICE    5    /* normal but significant condition */
#define    LOG_INFO    6    /* informational */
#define    LOG_DEBUG    7    /* debug-level messages */

CODE prioritynames[] =
  {
    { "alert", LOG_ALERT },
    { "crit", LOG_CRIT },
    { "debug", LOG_DEBUG },
    { "emerg", LOG_EMERG },
    { "err", LOG_ERR },
    { "error", LOG_ERR },        /* DEPRECATED */
    { "info", LOG_INFO },
    { "none", INTERNAL_NOPRI },        /* INTERNAL */
    { "notice", LOG_NOTICE },
    { "panic", LOG_EMERG },        /* DEPRECATED */
    { "warn", LOG_WARNING },        /* DEPRECATED */
    { "warning", LOG_WARNING },
    { NULL, -1 }
  }; 

2.4 syslog.conf实例

#syslog.conf
kern,user.*                                 /var/log/messages    #all messages of kern and user facilities
kern.!err                                   /var/log/critical    #all messages of kern facility with priorities lower than err (warn, notice ...)
*.*;auth,authpriv.none                      /var/log/noauth    #all messages except ones with auth and authpriv facilities
kern,user.*;kern.!=notice;*.err;syslog.none /var/log/OMG    #some whicked rule just as an example =)
*.*                                         /dev/null        #this prevents from logging to default log file (-O FILE or /var/log/messages) 

3.syslogd相关API

syslogd相关API主要有如下几个，其中openlog()调用是可选的，它建立一个系统日志工具的连接，并未后续的syslog调用设置默认设置。

ident参数是一个指向字符串的指针，syslog()输出的每条消息都会包含这个字符串。其他facility和level都在facilitynames[]和prioritynames[]中有定义。

syslog()用于写入一条日志消息，priority是facility和level的OR值。

#include <syslog.h>
void openlog(const char *ident, int log_options, int facility);
void syslog(int priority, const char *format, ...);
int setlogmask(int mask_priority);
void closelog(void);

下面重点看看openlog()的log_options选项：

#define    LOG_PID        0x01    /* log the pid with each message */------------------------log中包含进程pid。
#define    LOG_CONS    0x02    /* log on the console if errors in sending */-----------------如果syslogd无法正确输出，输出到控制台作为替代。
#define    LOG_ODELAY    0x04    /* delay open until first syslog() (default) */
#define    LOG_NDELAY    0x08    /* don't delay open */
#define    LOG_NOWAIT    0x10    /* don't wait for console forks: DEPRECATED */
#define    LOG_PERROR    0x20    /* log to stderr as well */

4.syslogd使用

下面从rcS中启动syslog功能、syslog服务、syslogd/klogd、syslog()几个方面，介绍如何使用syslog功能。

4.1 syslog的启动

在rcS中配置启动syslog服务：

#To start syslog
/etc/init.d/S01logging start

S01logging中启动syslogd和klogd两个daemon：

#!/bin/sh
#
# Start logging
#

SYSLOGD_ARGS=-n---------------参照syslogd option。
KLOGD_ARGS=-n-----------------参照klogd option。
[ -r /etc/default/logging ] && . /etc/default/logging

start() {
    printf "Starting logging: "
    start-stop-daemon -b -S -q -m -p /var/run/syslogd.pid --exec /sbin/syslogd -- $SYSLOGD_ARGS------------b表示background运行，-S是启动daemon，-q表示quiet，-m -p表示将当前daemon的pid写入/var/run/syslog.pid中；--exec表示daemon对应的可执行文件， SYSLOGD_ARGS表示可选参数。
    start-stop-daemon -b -S -q -m -p /var/run/klogd.pid --exec /sbin/klogd -- $KLOGD_ARGS
    echo "OK"
}

stop() {
    printf "Stopping logging: "
    start-stop-daemon -K -q -p /var/run/syslogd.pid---------------------------------------------------------K表示停止daemon。
    start-stop-daemon -K -q -p /var/run/klogd.pid
    echo "OK"
}

case "$1" in
  start)
    start
    ;;
  stop)
    stop
    ;;
  restart|reload)
    stop
    start
    ;;
  *)
    echo "Usage: $0 {start|stop|restart}"
    exit 1
esac

exit $?

4.2 syslogd和klogd选项

通过syslogd -h和klogd -h可以了解不同选项作用。

BusyBox v1.27.2 (2019-03-24 12:37:18 CST) multi-call binary.

Usage: syslogd [OPTIONS]

System logging utility

        -n              Run in foreground------------------------------------------------作为后台进程运行。
        -R HOST[:PORT]  Log to HOST:PORT (default PORT:514)------------------------------log输出到指定远程服务器。
        -L              Log locally and via network (default is network only if -R)
        -C[size_kb]     Log to shared mem buffer (use logread to read it)----------------log输出到共享内存，通过logread读取。
        -K              Log to kernel printk buffer (use dmesg to read it)---------------log输出到内核printk缓存中，通过dmesg读取。
        -O FILE         Log to FILE (default: /var/log/messages, stdout if -)------------log输出到指定文件中。
        -s SIZE         Max size (KB) before rotation (default 200KB, 0=off)-------------在log达到一定大小后，循环log到messgas.0中，后续依次变动。
        -b N            N rotated logs to keep (default 1, max 99, 0=purge)--------------保持的messages个数，messages.0最新，数字越大越老。一次覆盖。
        -l N            Log only messages more urgent than prio N (1-8)------------------设置记录的log优先级，注意这里如果设置-l 7，是不会记录等级7，即LOG_DEBUG的，只会记录LOG_INFO及更高优先级。
        -S              Smaller output---------------------------------------------------是否显示hostname，以及user.info之类的详细信息。
        -f FILE         Use FILE as config (default:/etc/syslog.conf)--------------------指定conf文件。

BusyBox v1.27.2 (2019-03-24 12:37:18 CST) multi-call binary.

Usage: klogd [-c N] [-n]

Kernel logger

        -c N    Print to console messages more urgent than prio N (1-8)------------------将错误高于N优先级的log输出到console。
        -n      Run in foreground--------------------------------------------------------转到后台运行。

4.3 使用openlog()/syslog()/closelog()

编写syslog，源码如下：

#include <syslog.h>                                         
#include <stdio.h>                                          
#include <stdarg.h>                                         
                                                            
int main(int argc, char** argv)                                              
{                                                           
    int log_test;                                           
    /*打开日志*/                                            
    openlog("log_test ", LOG_PID|LOG_CONS, LOG_USER);   
    while(1)    
    {
        /*写日志*/                                              
        syslog(LOG_INFO, "%s: PID information, pid=%d", argv[0], getpid());  
        syslog(LOG_DEBUG,"%s: debug message",argv[0]);
        usleep(1000);
    }
    /*关闭日志*/                                            
    closelog();                                             
}

启动syslog之后，可以看出/var/log/messages在不停增加。

Jan  1 08:10:56 log_test [176]: ./syslog: PID information, pid=176---------------------log_test是openlog()中的ident，[176]对应LOG_PID。
Jan  1 08:10:56 log_test [176]: ./syslog: debug message
Jan  1 08:10:56 log_test [176]: ./syslog: PID information, pid=176
Jan  1 08:10:56 log_test [176]: ./syslog: debug message
...

如果现实完整信息：

Jan  1 08:17:15 XXXX user.info log_test [176]: ./syslog: PID information, pid=176
Jan  1 08:17:15 XXXX user.debug log_test [176]: ./syslog: debug message
Jan  1 08:17:15 XXXX user.info log_test [176]: ./syslog: PID information, pid=176
...

4.4 syslog.conf中使用facility和level过滤log

在syslog.conf中设置filter，显示local0的debug及更高优先级log、不显示local1的log、显示local2的所有log，显示local3的err及更高优先级log：

local0.info            /var/log/local.log
local1.none             /var/log/local.log
local2.*                /var/log/local.log
local3.err              /var/log/local.log

测试程序如下：

#include <syslog.h>                                         
#include <stdio.h>                                          
#include <stdarg.h>                                         
                                                            
int main(int argc, char** argv)                                              
{                                                           
    int log_test;                                           
    /*打开日志*/                                            
    openlog("log_test ", LOG_PID|LOG_CONS, LOG_USER);   
    /*写日志*/                                              
    syslog(LOG_LOCAL0|LOG_INFO, "%s: PID information, pid=%d", argv[0], getpid());  
    syslog(LOG_LOCAL0|LOG_DEBUG,"%s: debug message",argv[0]);

    syslog(LOG_LOCAL1|LOG_INFO, "%s: PID information, pid=%d", argv[0], getpid());  
    syslog(LOG_LOCAL1|LOG_DEBUG,"%s: debug message",argv[0]);

    syslog(LOG_LOCAL2|LOG_INFO, "%s: PID information, pid=%d", argv[0], getpid());  
    syslog(LOG_LOCAL2|LOG_DEBUG,"%s: debug message",argv[0]);

    syslog(LOG_LOCAL3|LOG_INFO, "%s: PID information, pid=%d", argv[0], getpid());  
    syslog(LOG_LOCAL3|LOG_DEBUG,"%s: debug message",argv[0]);

    syslog(LOG_LOCAL4|LOG_INFO, "%s: PID information, pid=%d", argv[0], getpid());  
    syslog(LOG_LOCAL4|LOG_DEBUG,"%s: debug message",argv[0]);


    /*关闭日志*/                                            
    closelog();                                             
}

最终log输出/var/log/local.log：

Jan  1 08:11:54 xxxx local0.info log_test [259]: ./syslog: PID information, pid=259
Jan  1 08:11:54 xxxx local2.info log_test [259]: ./syslog: PID information, pid=259
Jan  1 08:11:54 xxxx local2.debug log_test [259]: ./syslog: debug message

5. busybox中syslogd和klogd分析

syslogd和klogd在busybox的sysklogd目录中，入口是syslogd_main()和klogd_main()。

通过系统日志概览可知：

1.内核log由klogd从内核中取出转发到/dev/log；klogctl()从kmsg的ring buffer中获取数据，通过syslog()发送到/dev/log。

2.用户空间的log直接通过syslog()发送到/dev/log。

3.syslogd从syslog.conf获取配置信息。

4.syslogd从/dev/log接收数据，或者从UDP端口514接收数据。

5.syslogd输出到磁盘文件、控制台、FIFO等。

5.1 klogd和klogctl()

klogd从通过klogctl()将内核log从ring buffer中取出，然后调用syslog()通过/dev/log发送。

这里的一个核心函数是klogctl()，通过对内核kmsg ring buffer的设置，从中取出内容。

#include <sys/klog.h>

int klogctl(int type, char *bufp, int len);

klogctl()的type，决定了后面内容的意义。

       SYSLOG_ACTION_CLOSE (0)
              Close the log.  Currently a NOP.

       SYSLOG_ACTION_OPEN (1)
              Open the log.  Currently a NOP.

       SYSLOG_ACTION_READ (2)-----------------------------等待内核log非空时，从中读取len大小的log到bufp中。返回值是实际读取内容大小，已经读取的部分会从内核ring buffer中移除。

       SYSLOG_ACTION_READ_ALL (3)
              Read all messages remaining in the ring buffer, placing them in the buffer pointed to  by  bufp.
              The call reads the last len bytes from the log buffer (nondestructively), but will not read more
              than was written into the buffer since the last "clear  ring  buffer"  command  (see  command  5
              below)).  The call returns the number of bytes read.

       SYSLOG_ACTION_READ_CLEAR (4)------------------------读取所有的内核log，然后清空。

       SYSLOG_ACTION_CLEAR (5)

       SYSLOG_ACTION_CONSOLE_OFF (6)-----------------------通过保存console_loglevel，然后将其设置到最低，来关闭console的显示。bufp和len参数忽略。

       SYSLOG_ACTION_CONSOLE_ON (7)-------------------------打开console显示。bufp和len参数忽略。

       SYSLOG_ACTION_CONSOLE_LEVEL (8)----------------------设置console的等级，1-8之间。详细如下：
　　　　　　　　#define KERN_EMERG "<0>" /* system is unusable */ 
　　　　　　　　#define KERN_ALERT "<1>" /* action must be taken immediately */ 
　　　　　　　　#define KERN_CRIT "<2>" /* critical conditions */ 
　　　　　　　　#define KERN_ERR "<3>" /* error conditions */ 
　　　　　　　　#define KERN_WARNING "<4>" /* warning conditions */ 
　　　　　　　　#define KERN_NOTICE "<5>" /* normal but significant condition */ 
　　　　　　　　#define KERN_INFO "<6>" /* informational */ 
　　　　　　　　#define KERN_DEBUG "<7>" /* debug-level messages */

       SYSLOG_ACTION_SIZE_UNREAD (9) (since Linux 2.4.10)---返回内核中可读buffer大小，bufp和len参数忽略。

       SYSLOG_ACTION_SIZE_BUFFER (10) (since Linux 2.6.6)---内核中log ring buffer大小。

klogd用到的相关klogctl()包括：

static void klogd_open(void)
{
    /* "Open the log. Currently a NOP" */
    klogctl(1, NULL, 0);
}

static void klogd_setloglevel(int lvl)
{
    klogctl(8, NULL, lvl);
}

static int klogd_read(char *bufp, int len)
{
    return klogctl(2, bufp, len);
}

static void klogd_close(void)
{
    klogctl(7, NULL, 0); /* "7 -- Enable printk's to console" */
    klogctl(0, NULL, 0); /* "0 -- Close the log. Currently a NOP" */
}

klogd_main()走读如下：

int klogd_main(int argc, char **argv) MAIN_EXTERNALLY_VISIBLE;
int klogd_main(int argc UNUSED_PARAM, char **argv)
{
    int i = 0;
    char *opt_c;
    int opt;
    int used;

    setup_common_bufsiz();----------------------------------------------------分配COMMON_BUFSIZE大小的bufer缓存，指针bb_common_bufsiz1。

    opt = getopt32(argv, "c:n", &opt_c);
    if (opt & OPT_LEVEL) {-----------------------------------------------------获取log等级参数
        /* Valid levels are between 1 and 8 */
        i = xatou_range(opt_c, 1, 8);
    }
    if (!(opt & OPT_FOREGROUND)) {---------------------------------------------klogd是否后台运行。
        bb_daemonize_or_rexec(DAEMON_CHDIR_ROOT, argv);
    }

    logmode = LOGMODE_SYSLOG;

    /* klogd_open() before openlog(), since it might use fixed fd 3,
     * and openlog() also may use the same fd 3 if we swap them:
     */
    klogd_open();
    openlog("kernel", 0, LOG_KERN);---------------------------------------------所有的内核log，都以kernel作为ident。

    if (i)
        klogd_setloglevel(i);---------------------------------------------------从传入log等级参数设置到内核进行过滤。

    signal(SIGHUP, SIG_IGN);----------------------------------------------------忽略SIGHUP信号。
    /* We want klogd_read to not be restarted, thus _norestart: */
    bb_signals_recursive_norestart(BB_FATAL_SIGS, record_signo);----------------在record_signo()中记录接收到的信号bb_got_signal。

    syslog(LOG_NOTICE, "klogd started: %s", bb_banner);

    write_pidfile(CONFIG_PID_FILE_PATH "/klogd.pid");

    used = 0;
    while (!bb_got_signal) {----------------------------------------------------没有受到信号中断，则while循环。
        int n;
        int priority;
        char *start;

        /* "2 -- Read from the log." */
        start = log_buffer + used;-----------------------------------------------log_buffer指向bb_common_bufsiz1。
        n = klogd_read(start, KLOGD_LOGBUF_SIZE-1 - used);-----------------------读取KLOGD_LOGBUF_SIZE-1 - used大小log到start。
        if (n < 0) {
            if (errno == EINTR)
                continue;
            bb_perror_msg(READ_ERROR);
            break;
        }
        start[n] = '';

        /* Process each newline-terminated line in the buffer */
        start = log_buffer;
        while (1) {
            char *newline = strchrnul(start, '
');------------------------------根据
进行分行处理，一行一个syslog(),newline指向下一行头部。

            if (*newline == '') {
                /* This line is incomplete */

                /* move it to the front of the buffer */
                overlapping_strcpy(log_buffer, start);
                used = newline - start;
                if (used < KLOGD_LOGBUF_SIZE-1) {
                    /* buffer isn't full */
                    break;
                }
                /* buffer is full, log it anyway */
                used = 0;
                newline = NULL;
            } else {
                *newline++ = '';
            }

            /* Extract the priority */
            priority = LOG_INFO;--------------------------------------------------默认等级
            if (*start == '<') {
                start++;
                if (*start)
                    priority = strtoul(start, &start, 10);------------------------等级转换。
                if (*start == '>')
                    start++;
            }
            /* Log (only non-empty lines) */
            if (*start)
                syslog(priority, "%s", start);------------------------------------获取等级和一行内容start，通过syslog打印。

            if (!newline)
                break;
            start = newline;------------------------------------------------------start指向下一行首部，循环处理。
        }
    }

    klogd_close();----------------------------------------------------------------关闭klog。
    syslog(LOG_NOTICE, "klogd: exiting");
    remove_pidfile(CONFIG_PID_FILE_PATH "/klogd.pid");
    if (bb_got_signal)
        kill_myself_with_sig(bb_got_signal);
    return EXIT_FAILURE;
}

klogctl()函数通过调用syslog系统调用，来从内核中获取msg。

SYSCALL_DEFINE3(syslog, int, type, char __user *, buf, int, len)
{
    return do_syslog(type, buf, len, SYSLOG_FROM_READER);
}


int do_syslog(int type, char __user *buf, int len, int source)
{
    bool clear = false;
    static int saved_console_loglevel = LOGLEVEL_DEFAULT;
    int error;

    error = check_syslog_permissions(type, source);
    if (error)
        goto out;

    switch (type) {
    case SYSLOG_ACTION_CLOSE:    /* Close log */
        break;
    case SYSLOG_ACTION_OPEN:    /* Open log */--------------------------CLOSE和OPEN没有任何作用。
        break;
    case SYSLOG_ACTION_READ:    /* Read from log */
        error = -EINVAL;
        if (!buf || len < 0)
            goto out;
        error = 0;
        if (!len)
            goto out;
        if (!access_ok(VERIFY_WRITE, buf, len)) {
            error = -EFAULT;
            goto out;
        }
        error = wait_event_interruptible(log_wait,
                         syslog_seq != log_next_seq);
        if (error)
            goto out;
        error = syslog_print(buf, len);
        break;
    /* Read/clear last kernel messages */
    case SYSLOG_ACTION_READ_CLEAR:
        clear = true;-------------------------------------------------可以看出这里并不做实际清空，只是SYSLOG_ACTION_READ_ALL之后才会清空。
        /* FALL THRU */
    /* Read last kernel messages */
    case SYSLOG_ACTION_READ_ALL:
        error = -EINVAL;
        if (!buf || len < 0)
            goto out;
        error = 0;
        if (!len)
            goto out;
        if (!access_ok(VERIFY_WRITE, buf, len)) {
            error = -EFAULT;
            goto out;
        }
        error = syslog_print_all(buf, len, clear);--------------------将所有出书到buf中，但是有长度len限制，只保存最新部分。
        break;
    /* Clear ring buffer */
    case SYSLOG_ACTION_CLEAR:
        syslog_print_all(NULL, 0, true);------------------------------仅仅做清空操作。
        break;
    /* Disable logging to console */
    case SYSLOG_ACTION_CONSOLE_OFF:
        if (saved_console_loglevel == LOGLEVEL_DEFAULT)
            saved_console_loglevel = console_loglevel;
        console_loglevel = minimum_console_loglevel;
        break;
    /* Enable logging to console */
    case SYSLOG_ACTION_CONSOLE_ON:
        if (saved_console_loglevel != LOGLEVEL_DEFAULT) {
            console_loglevel = saved_console_loglevel;
            saved_console_loglevel = LOGLEVEL_DEFAULT;
        }
        break;
    /* Set level of messages printed to console */
    case SYSLOG_ACTION_CONSOLE_LEVEL:
        error = -EINVAL;
        if (len < 1 || len > 8)
            goto out;
        if (len < minimum_console_loglevel)
            len = minimum_console_loglevel;
        console_loglevel = len;----------------------------------------设置console_loglevel等级。
        /* Implicitly re-enable logging to console */
        saved_console_loglevel = LOGLEVEL_DEFAULT;
        error = 0;
        break;
    /* Number of chars in the log buffer */
    case SYSLOG_ACTION_SIZE_UNREAD:
...
        break;
    /* Size of the log buffer */
    case SYSLOG_ACTION_SIZE_BUFFER:
        error = log_buf_len;
        break;
    default:
        error = -EINVAL;
        break;
    }
out:
    return error;
}


static int syslog_print(char __user *buf, int size)
{
    char *text;
    struct printk_log *msg;
    int len = 0;

    text = kmalloc(LOG_LINE_MAX + PREFIX_MAX, GFP_KERNEL);
    if (!text)
        return -ENOMEM;

    while (size > 0) {
        size_t n;
        size_t skip;

        raw_spin_lock_irq(&logbuf_lock);
        if (syslog_seq < log_first_seq) {
            /* messages are gone, move to first one */
            syslog_seq = log_first_seq;
            syslog_idx = log_first_idx;
            syslog_prev = 0;
            syslog_partial = 0;
        }
        if (syslog_seq == log_next_seq) {
            raw_spin_unlock_irq(&logbuf_lock);
            break;
        }

        skip = syslog_partial;
        msg = log_from_idx(syslog_idx);
        n = msg_print_text(msg, syslog_prev, true, text,
                   LOG_LINE_MAX + PREFIX_MAX);
        if (n - syslog_partial <= size) {
            /* message fits into buffer, move forward */
            syslog_idx = log_next(syslog_idx);
            syslog_seq++;
            syslog_prev = msg->flags;
            n -= syslog_partial;
            syslog_partial = 0;
        } else if (!len){
            /* partial read(), remember position */
            n = size;
            syslog_partial += n;
        } else
            n = 0;
        raw_spin_unlock_irq(&logbuf_lock);

        if (!n)
            break;

        if (copy_to_user(buf, text + skip, n)) {
            if (!len)
                len = -EFAULT;
            break;
        }

        len += n;
        size -= n;
        buf += n;
    }

    kfree(text);
    return len;
}

static int syslog_print_all(char __user *buf, int size, bool clear)
{
    char *text;
    int len = 0;

    text = kmalloc(LOG_LINE_MAX + PREFIX_MAX, GFP_KERNEL);
    if (!text)
        return -ENOMEM;

    raw_spin_lock_irq(&logbuf_lock);
    if (buf) {
        u64 next_seq;
        u64 seq;
        u32 idx;
        enum log_flags prev;

        /*
         * Find first record that fits, including all following records,
         * into the user-provided buffer for this dump.
         */
        seq = clear_seq;
        idx = clear_idx;
        prev = 0;
        while (seq < log_next_seq) {
            struct printk_log *msg = log_from_idx(idx);

            len += msg_print_text(msg, prev, true, NULL, 0);---------------------------获取所有log大小。
            prev = msg->flags;
            idx = log_next(idx);
            seq++;
        }

        /* move first record forward until length fits into the buffer */
        seq = clear_seq;
        idx = clear_idx;
        prev = 0;
        while (len > size && seq < log_next_seq) {
            struct printk_log *msg = log_from_idx(idx);

            len -= msg_print_text(msg, prev, true, NULL, 0);---------------------------找到满足剩余长度小于size大小的seq，后面将seq之后所有的log发送到text中。保证满足buf尺寸前提下，取最新的log。
            prev = msg->flags;
            idx = log_next(idx);
            seq++;
        }

        /* last message fitting into this dump */
        next_seq = log_next_seq;

        len = 0;
        while (len >= 0 && seq < next_seq) {
            struct printk_log *msg = log_from_idx(idx);
            int textlen;

            textlen = msg_print_text(msg, prev, true, text,
                         LOG_LINE_MAX + PREFIX_MAX);----------------------------------将msg内容输出到text中，第3参数true表示要输出给syslog()使用，所以需要加上facility和level头。
            if (textlen < 0) {
                len = textlen;
                break;
            }
            idx = log_next(idx);
            seq++;
            prev = msg->flags;

            raw_spin_unlock_irq(&logbuf_lock);
            if (copy_to_user(buf + len, text, textlen))
                len = -EFAULT;
            else
                len += textlen;
            raw_spin_lock_irq(&logbuf_lock);

            if (seq < log_first_seq) {
                /* messages are gone, move to next one */
                seq = log_first_seq;
                idx = log_first_idx;
                prev = 0;
            }
        }
    }

    if (clear) {-------------------------------------------------------------------------清空所有buffer。
        clear_seq = log_next_seq;
        clear_idx = log_next_idx;
    }
    raw_spin_unlock_irq(&logbuf_lock);

    kfree(text);
    return len;
}

5.2 syslog()函数

在glibc库的misc/syslog.c中，定义了openlog()/syslog()/closelog()函数。

这里重点看一下syslog()都做了哪些内容。

ldbl_hidden_def (__syslog, syslog)

void
__syslog(int pri, const char *fmt, ...)
{
    va_list ap;

    va_start(ap, fmt);
    __vsyslog_chk(pri, -1, fmt, ap);
    va_end(ap);
}

void
__vsyslog_chk(int pri, int flag, const char *fmt, va_list ap)
{
    struct tm now_tm;
    time_t now;
    int fd;
    FILE *f;
    char *buf = 0;
    size_t bufsize = 0;
    size_t msgoff;
#ifndef NO_SIGPIPE
     struct sigaction action, oldaction;
     int sigpipe;
#endif
    int saved_errno = errno;
    char failbuf[3 * sizeof (pid_t) + sizeof "out of memory []"];
...

    /* Build the message in a memory-buffer stream.  */
    f = __open_memstream (&buf, &bufsize);
    if (f == NULL)
      {
...
      }
    else
      {
        __fsetlocking (f, FSETLOCKING_BYCALLER);
        fprintf (f, "<%d>", pri);---------------------------------------------------输出pri到f，pri包括facility和level。
        (void) time (&now);
        f->_IO_write_ptr += __strftime_l (f->_IO_write_ptr,
                          f->_IO_write_end
                          - f->_IO_write_ptr,
                          "%h %e %T ",
                          __localtime_r (&now, &now_tm),
                          _nl_C_locobj_ptr);----------------------------------------输出log产生的时间。
        msgoff = ftell (f);
        if (LogTag == NULL)
          LogTag = __progname;
        if (LogTag != NULL)
          __fputs_unlocked (LogTag, f);
        if (LogStat & LOG_PID)
          fprintf (f, "[%d]", (int) __getpid ());-----------------------------------在使能LOG_PID之后，输出pid到f。
        if (LogTag != NULL)
          {
        __putc_unlocked (':', f);
        __putc_unlocked (' ', f);
          }

        /* Restore errno for %m format.  */
        __set_errno (saved_errno);

        /* We have the header.  Print the user's format into the
               buffer.  */
        if (flag == -1)--------------------------------------------------------------根据flag是否有效，输出flag、fmt、ap内容到f。
          vfprintf (f, fmt, ap);
        else
          __vfprintf_chk (f, flag, fmt, ap);

        /* Close the memory stream; this will finalize the data
           into a malloc'd buffer in BUF.  */
        fclose (f);
      }
...

    /* Prepare for multiple users.  We have to take care: open and
       write are cancellation points.  */
    struct cleanup_arg clarg;
    clarg.buf = buf;
    clarg.oldaction = NULL;
    __libc_cleanup_push (cancel_handler, &clarg);
    __libc_lock_lock (syslog_lock);

...
    /* Get connected, output the message to the local logger. */
    if (!connected)
        openlog_internal(LogTag, LogStat | LOG_NDELAY, 0);---------------------------如果还没有和/dev/log连接上，调用openlog_internal()连接，操作句柄是LogFile。/* If we have a SOCK_STREAM connection, also send ASCII NUL as
       a record terminator.  */
    if (LogType == SOCK_STREAM)
      ++bufsize;

    if (!connected || __send(LogFile, buf, bufsize, send_flags) < 0)-----------------buf中存放上面组装好的log，通过send()发送到/dev/log。
      {
        if (connected)---------------------------------------------------------------如果通过openlog()打开过，那么send异常有可能是socket异常。需要重新关闭打开。
          {
        /* Try to reopen the syslog connection.  Maybe it went
           down.  */
        closelog_internal ();
        openlog_internal(LogTag, LogStat | LOG_NDELAY, 0);
          }

        if (!connected || __send(LogFile, buf, bufsize, send_flags) < 0)
          {
        closelog_internal ();    /* attempt re-open next time */----------------------关闭LogFile。
        /*...
          }
      }

#ifndef NO_SIGPIPE
    if (sigpipe == 0)
        __sigaction (SIGPIPE, &oldaction, (struct sigaction *) NULL);
#endif

    /* End of critical section.  */
    __libc_cleanup_pop (0);
    __libc_lock_unlock (syslog_lock);

    if (buf != failbuf)
        free (buf);
}

openlog()和closelog()主要通过openlog_internal()和closelog_internal()实现。

static void
openlog_internal(const char *ident, int logstat, int logfac)
{
    if (ident != NULL)
        LogTag = ident;
    LogStat = logstat;
    if (logfac != 0 && (logfac &~ LOG_FACMASK) == 0)
        LogFacility = logfac;

    int retry = 0;
    while (retry < 2) {
        if (LogFile == -1) {
            SyslogAddr.sun_family = AF_UNIX;
            (void)strncpy(SyslogAddr.sun_path, _PATH_LOG,
                      sizeof(SyslogAddr.sun_path));
            if (LogStat & LOG_NDELAY) {
              LogFile = __socket(AF_UNIX, LogType | SOCK_CLOEXEC, 0);---------------------打开一个socket。
              if (LogFile == -1)
                return;
            }
        }
        if (LogFile != -1 && !connected)
        {
            int old_errno = errno;
            if (__connect(LogFile, &SyslogAddr, sizeof(SyslogAddr))-----------------------将LogFile和/dev/log socket连接，后面可以传输数据。
                == -1)
            {
...
            } else
                connected = 1;
        }
        break;
    }
}

static void
closelog_internal (void)
{
  if (!connected)
    return;

  __close (LogFile);
  LogFile = -1;
  connected = 0;
}

5.3 syslogd解析

syslogd_main()中主要是解析各种选项，转换成参数。

static void do_syslogd(void) NORETURN;
static void do_syslogd(void)
{
#if ENABLE_FEATURE_REMOTE_LOG
    llist_t *item;
#endif
#if ENABLE_FEATURE_SYSLOGD_DUP
    int last_sz = -1;
    char *last_buf;
    char *recvbuf = G.recvbuf;
#else
#define recvbuf (G.recvbuf)
#endif

    /* Set up signal handlers (so that they interrupt read()) */
    signal_no_SA_RESTART_empty_mask(SIGTERM, record_signo);
    signal_no_SA_RESTART_empty_mask(SIGINT, record_signo);
    //signal_no_SA_RESTART_empty_mask(SIGQUIT, record_signo);
    signal(SIGHUP, SIG_IGN);
#ifdef SYSLOGD_MARK
    signal(SIGALRM, do_mark);
    alarm(G.markInterval);
#endif
    xmove_fd(create_socket(), STDIN_FILENO);-------------------------------------创建/dev/log socket，并且将句柄dup到STDIN_FILENO。从socket接收数据。

    if (option_mask32 & OPT_circularlog)
        ipcsyslog_init();

    if (option_mask32 & OPT_kmsg)
        kmsg_init();

    timestamp_and_log_internal("syslogd started: BusyBox v" BB_VER);

    while (!bb_got_signal) {
        ssize_t sz;

#if ENABLE_FEATURE_SYSLOGD_DUP
        last_buf = recvbuf;
        if (recvbuf == G.recvbuf)
            recvbuf = G.recvbuf + MAX_READ;
        else
            recvbuf = G.recvbuf;
#endif
 read_again:
        sz = read(STDIN_FILENO, recvbuf, MAX_READ - 1);----------------------------------从STDIN_FILENO读取syslog()输入的数据。
        if (sz < 0) {
            if (!bb_got_signal)
                bb_perror_msg("read from %s", _PATH_LOG);
            break;
        }

        /* Drop trailing '
' and NULs (typically there is one NUL) */
        while (1) {
            if (sz == 0)
                goto read_again;
            /* man 3 syslog says: "A trailing newline is added when needed".
             * However, neither glibc nor uclibc do this:
             * syslog(prio, "test")   sends "test" to /dev/log,
             * syslog(prio, "test
") sends "test
".
             * IOW: newline is passed verbatim!
             * I take it to mean that it's syslogd's job
             * to make those look identical in the log files. */
            if (recvbuf[sz-1] != '' && recvbuf[sz-1] != '
')---------------------------''作为不同syslog()的分隔符；'
'作为换行符。
                break;
            sz--;
        }

...
        if (!ENABLE_FEATURE_REMOTE_LOG || (option_mask32 & OPT_locallog)) {
            recvbuf[sz] = ''; /* ensure it *is* NUL terminated */
            split_escape_and_log(recvbuf, sz);---------------------------------------------这里的recvbuf已经根据''和'
'进行分割，里面分析facility和level，进行转发。
        }
    } /* while (!bb_got_signal) */

    timestamp_and_log_internal("syslogd exiting");
    remove_pidfile(CONFIG_PID_FILE_PATH "/syslogd.pid");
    ipcsyslog_cleanup();
    if (option_mask32 & OPT_kmsg)
        kmsg_cleanup();
    kill_myself_with_sig(bb_got_signal);
#undef recvbuf
}

split_escape_and_log()将消息分割出prio和msg，进行处理。

static void split_escape_and_log(char *tmpbuf, int len)
{
    char *p = tmpbuf;

    tmpbuf += len;
    while (p < tmpbuf) {
        char c;
        char *q = G.parsebuf;
        int pri = (LOG_USER | LOG_NOTICE);

        if (*p == '<') {-------------------------------------------此处的tmpbuf都是在__vsyslog_chk()中组装，这里开头应该是'<pri>'，从中解析出pri。
            /* Parse the magic priority number */
            pri = bb_strtou(p + 1, &p, 10);
            if (*p == '>')
                p++;
            if (pri & ~(LOG_FACMASK | LOG_PRIMASK))
                pri = (LOG_USER | LOG_NOTICE);
        }

        while ((c = *p++)) {---------------------------------------遇到''停止。而且一定会有。
            if (c == '
')
                c = ' ';
            if (!(c & ~0x1f) && c != '	') {
                *q++ = '^';
                c += '@'; /* ^@, ^A, ^B... */
            }
            *q++ = c;
        }
        *q = '';

        /* Now log it */
        timestamp_and_log(pri, G.parsebuf, q - G.parsebuf);
    }
}

static void timestamp_and_log(int pri, char *msg, int len)
{
    char *timestamp;
    time_t now;

    /* Jan 18 00:11:22 msg... */
    /* 01234567890123456 */
    if (len < 16 || msg[3] != ' ' || msg[6] != ' '
     || msg[9] != ':' || msg[12] != ':' || msg[15] != ' '
    ) {----------------------------------------------------------------------------------没有时间戳的msg处理，添加timestamp。
        time(&now);
    #if 0
        timestamp = ctime(&now) + 4; /* skip day of week */
    #else
        char time_str[16];
        struct timespec ts;
        int ret = 0;

        timestamp = time_str;
        ret = clock_gettime(CLOCK_MONOTONIC, &ts);
        snprintf(timestamp, 16, "%8ld%c%6ld", ts.tv_sec, '.', ts.tv_nsec);
    #endif
    } else {
        now = 0;
        timestamp = msg;
#if 1-------------------------------------------------------------------------------------修改时间戳的来源，使用更高精度的时间戳。
        struct timespec ts;
        int ret = 0;

        ret = clock_gettime(CLOCK_MONOTONIC, &ts);
        snprintf(timestamp, 16, "%8ld%c%6ld", ts.tv_sec, '.', ts.tv_nsec);
#endif
        msg += 16;
    }
    timestamp[15] = '';-----------------------------------------------------------------将msg分割成timestamp和msg两部分、

    if (option_mask32 & OPT_kmsg) {
        log_to_kmsg(pri, msg);
        return;
    }

    if (option_mask32 & OPT_small)-------------------------------------------------------根据syslogd选项'-S'来决定输出内容。
        sprintf(G.printbuf, "%s %s
", timestamp, msg);
    else {
        char res[20];
        parse_fac_prio_20(pri, res);
        sprintf(G.printbuf, "%s %.64s %s %s
", timestamp, G.hostname, res, msg);--------增加hostname和facility/level输出。
    }

    /* Log message locally (to file or shared mem) */
#if ENABLE_FEATURE_SYSLOGD_CFG
    {
        bool match = 0;
        logRule_t *rule;
        uint8_t facility = LOG_FAC(pri);
        uint8_t prio_bit = 1 << LOG_PRI(pri);

        for (rule = G.log_rules; rule; rule = rule->next) {
            if (rule->enabled_facility_priomap[facility] & prio_bit) {--------------------如果当前msg的prio满足/etc/syslog.conf中某一个条件，则将log内容输出到对应的rule->file中。如果一个msd满足多个rule，那么输出多次。
                log_locally(now, G.printbuf, rule->file);
                match = 1;
            }
        }
        if (match)-------------------------------------------------------------------------如果msg满足rule中的一个，则退出。不会进入下面的全局性/var/log/messages。
            return;
    }
#endif
    if (LOG_PRI(pri) < G.logLevel) {
#if ENABLE_FEATURE_IPC_SYSLOG
        if ((option_mask32 & OPT_circularlog) && G.shbuf) {
            log_to_shmem(G.printbuf);
            return;
        }
#endif
        log_locally(now, G.printbuf, &G.logFile);-------------------------------------------不在/etc/syslog.conf中的msg，满足设置的loglevel后。
    }
}

 log_locally()将log保存到指定的文件。

/* Print a message to the log file. */
static void log_locally(time_t now, char *msg, logFile_t *log_file)
{
#ifdef SYSLOGD_WRLOCK
    struct flock fl;
#endif
    int len = strlen(msg);

    /* fd can't be 0 (we connect fd 0 to /dev/log socket) */----------------------------log_file->fd句柄0给/dev/log使用，1给stdout使用。
    /* fd is 1 if "-O -" is in use */
    if (log_file->fd > 1) {
        if (!now)
            now = time(NULL);
        if (log_file->last_log_time != now) {-------------------------------------------每一秒钟开关一次log_file，让用户有删除log_file的机会。
            log_file->last_log_time = now;
            close(log_file->fd);
            goto reopen;
        }
    }
    else if (log_file->fd == 1) {
        /* We are logging to stdout: do nothing */
    }
    else {
        if (LONE_DASH(log_file->path)) {
            log_file->fd = 1;
            /* log_file->isRegular = 0; - already is */
        } else {
 reopen:
            log_file->fd = open(log_file->path, O_WRONLY | O_CREAT
                    | O_NOCTTY | O_APPEND | O_NONBLOCK,
                    0666);----------------------------------------------------------------打开log_file文件。
            if (log_file->fd < 0) {-------------------------------------------------------打开失败输出到/dev/console。
                /* cannot open logfile? - print to /dev/console then */
                int fd = device_open(DEV_CONSOLE, O_WRONLY | O_NOCTTY | O_NONBLOCK);
                if (fd < 0)
                    fd = 2; /* then stderr, dammit */
                full_write(fd, msg, len);
                if (fd != 2)
                    close(fd);
                return;
            }
#if ENABLE_FEATURE_ROTATE_LOGFILE
            {
                struct stat statf;
                log_file->isRegular = (fstat(log_file->fd, &statf) == 0 && S_ISREG(statf.st_mode));
                /* bug (mostly harmless): can wrap around if file > 4gb */
                log_file->size = statf.st_size;
            }
#endif
        }
    }

#ifdef SYSLOGD_WRLOCK
    fl.l_whence = SEEK_SET;
    fl.l_start = 0;
    fl.l_len = 1;
    fl.l_type = F_WRLCK;
    fcntl(log_file->fd, F_SETLKW, &fl);
#endif

#if ENABLE_FEATURE_ROTATE_LOGFILE
    if (G.logFileSize && log_file->isRegular && log_file->size > G.logFileSize) {---------判断当前写入是否会发生尺寸溢出，超出分隔尺寸的话。进行文件分隔。
        if (G.logFileRotate) { /* always 0..99 */
            int i = strlen(log_file->path) + 3 + 1;
            char oldFile[i];
            char newFile[i];
            i = G.logFileRotate - 1;
            /* rename: f.8 -> f.9; f.7 -> f.8; ... */
            while (1) {-------------------------------------------------------------------将已有的文件名末尾数字加1，文件名末尾达到最大数字的文件将被覆盖丢弃。
                sprintf(newFile, "%s.%d", log_file->path, i);
                if (i == 0) break;
                sprintf(oldFile, "%s.%d", log_file->path, --i);
                /* ignore errors - file might be missing */
                rename(oldFile, newFile);
            }
            /* newFile == "f.0" now */
            rename(log_file->path, newFile);----------------------------------------------将messages变成messages.0.
        }

        unlink(log_file->path);
#ifdef SYSLOGD_WRLOCK
        fl.l_type = F_UNLCK;
        fcntl(log_file->fd, F_SETLKW, &fl);
#endif
        close(log_file->fd);
        goto reopen;
    }
/* TODO: what to do on write errors ("disk full")? */
    len = full_write(log_file->fd, msg, len);---------------------------------------------将msg写入文件。
    if (len > 0)
        log_file->size += len;
#else
    full_write(log_file->fd, msg, len);
#endif

#ifdef SYSLOGD_WRLOCK
    fl.l_type = F_UNLCK;
    fcntl(log_file->fd, F_SETLKW, &fl);
#endif
}

经过如上分析可知内核log如何被klogd通过klogctl()获取，调用syslog()转发；应用如何通过syslog()发送消息到/dev/log；syslod是如何从/dev/log接收信息，并进行分发的。

5.3.1 解析syslog.conf

static void parse_syslogdcfg(const char *file)
{
    char *t;
    logRule_t **pp_rule;
    /* tok[0] set of selectors */
    /* tok[1] file name */
    /* tok[2] has to be NULL */
    char *tok[3];
    parser_t *parser;

    parser = config_open2(file ? file : "/etc/syslog.conf",
                file ? xfopen_for_read : fopen_for_read);
    if (!parser)
        /* didn't find default /etc/syslog.conf */
        /* proceed as if we built busybox without config support */
        return;

    /* use ptr to ptr to avoid checking whether head was initialized */
    pp_rule = &G.log_rules;
    /* iterate through lines of config, skipping comments */
    while (config_read(parser, tok, 3, 2, "# 	", PARSE_NORMAL | PARSE_MIN_DIE)) {-------------------------读取syslog.conf每一行。
        char *cur_selector;
        logRule_t *cur_rule;

        /* unexpected trailing token? */
        if (tok[2])
            goto cfgerr;

        cur_rule = *pp_rule = xzalloc(sizeof(*cur_rule));

        cur_selector = tok[0];
        /* iterate through selectors: "kern.info;kern.!err;..." */
        do {-----------------------------------------------------------------------------------------------编译一行中每一个selector。
            const CODE *code;
            char *next_selector;
            uint8_t negated_prio; /* "kern.!err" */
            uint8_t single_prio;  /* "kern.=err" */
            uint32_t facmap; /* bitmap of enabled facilities */
            uint8_t primap;  /* bitmap of enabled priorities */
            unsigned i;

            next_selector = strchr(cur_selector, ';');
            if (next_selector)
                *next_selector++ = '';

            t = strchr(cur_selector, '.');
            if (!t)
                goto cfgerr;
            *t++ = ''; /* separate facility from priority */

            negated_prio = 0;
            single_prio = 0;-------------------------------------------------------------------------------解析selector中的priority，五种情况！、=、*、none、info/notice/debu/err。
            if (*t == '!') {-------------------------------------------------------------------------------排除特定优先级。
                negated_prio = 1;
                ++t;
            }
            if (*t == '=') {-------------------------------------------------------------------------------指定特定优先级。
                single_prio = 1;
                ++t;
            }

            /* parse priority */
            if (*t == '*')---------------------------------------------------------------------------------所有优先级。
                primap = 0xff; /* all 8 log levels enabled */
            else {
                uint8_t priority;
                code = find_by_name(t, prioritynames);
                if (!code)
                    goto cfgerr;
                primap = 0;
                priority = code->c_val;
                if (priority == INTERNAL_NOPRI) {-----------------------------------------------------------INTERNAL_NOPRI为0x10，这里需要取反，所以所有优先级都不显示。
                    /* ensure we take "enabled_facility_priomap[fac] &= 0" branch below */
                    negated_prio = 1;
                } else {
                    priority = 1 << priority;
                    do {
                        primap |= priority;
                        if (single_prio)---------------------------------------------------------------------特定优先级情况，只置一位。
                            break;
                        priority >>= 1;----------------------------------------------------------------------将当前优先级以及更高优先级位置位。
                    } while (priority);
                    if (negated_prio)------------------------------------------------------------------------'!'情况，下面enabled_facility_priomap还会特殊处理。
                        primap = ~primap;
                }
            }

            /* parse facility */------------------------------------------------------------------------------解析facility情况。
            if (*cur_selector == '*')-------------------------------------------------------------------------所有facility。
                facmap = (1<<LOG_NFACILITIES) - 1;
            else {
                char *next_facility;
                facmap = 0;
                t = cur_selector;
                /* iterate through facilities: "kern,daemon.<priospec>" */
                do {------------------------------------------------------------------------------------------多facility情况。
                    next_facility = strchr(t, ',');
                    if (next_facility)
                        *next_facility++ = '';
                    code = find_by_name(t, facilitynames);
                    if (!code)
                        goto cfgerr;
                    /* "mark" is not a real facility, skip it */
                    if (code->c_val != INTERNAL_MARK)
                        facmap |= 1<<(LOG_FAC(code->c_val));
                    t = next_facility;
                } while (t);
            }

            /* merge result with previous selectors */
            for (i = 0; i < LOG_NFACILITIES; ++i) {------------------------------------------------------------合并多selector数据，合并到cur_rule中。
                if (!(facmap & (1<<i)))
                    continue;
                if (negated_prio)
                    cur_rule->enabled_facility_priomap[i] &= primap;
                else
                    cur_rule->enabled_facility_priomap[i] |= primap;
            }

            cur_selector = next_selector;
        } while (cur_selector);

        /* check whether current file name was mentioned in previous rules or
         * as global logfile (G.logFile).
         */
        if (strcmp(G.logFile.path, tok[1]) == 0) {
            cur_rule->file = &G.logFile;
            goto found;
        }
        /* temporarily use cur_rule as iterator, but *pp_rule still points
         * to currently processing rule entry.
         * NOTE: *pp_rule points to the current (and last in the list) rule.
         */
        for (cur_rule = G.log_rules; cur_rule != *pp_rule; cur_rule = cur_rule->next) {
            if (strcmp(cur_rule->file->path, tok[1]) == 0) {
                /* found - reuse the same file structure */
                (*pp_rule)->file = cur_rule->file;
                cur_rule = *pp_rule;
                goto found;
            }
        }
        cur_rule->file = xzalloc(sizeof(*cur_rule->file));
        cur_rule->file->fd = -1;
        cur_rule->file->path = xstrdup(tok[1]);--------------------------------------------------------一行一个rule，设置输出文件路径。
 found:
        pp_rule = &cur_rule->next;
    }
    config_close(parser);
    return;

 cfgerr:
    bb_error_msg_and_die("error in '%s' at line %d",
            file ? file : "/etc/syslog.conf",
            parser->lineno);
}

5.3.2 输出syslog到/dev/kmsg

如果在syslogd选项中加-K，那么就会将syslog()输出写到/dev/kmsg中。通过dmesg可以读取。

static void kmsg_init(void)
{
    G.kmsgfd = xopen("/dev/kmsg", O_WRONLY);------------------------------------------打开/dev/kmsg文件。

    /*
     * kernel < 3.5 expects single char printk KERN_* priority prefix,
     * from 3.5 onwards the full syslog facility/priority format is supported
     */
    if (get_linux_version_code() < KERNEL_VERSION(3,5,0))
        G.primask = LOG_PRIMASK;
    else
        G.primask = -1;
}

static void kmsg_cleanup(void)
{
    if (ENABLE_FEATURE_CLEAN_UP)
        close(G.kmsgfd);
}

/* Write message to /dev/kmsg */
static void log_to_kmsg(int pri, const char *msg)
{
    /*
     * kernel < 3.5 expects single char printk KERN_* priority prefix,
     * from 3.5 onwards the full syslog facility/priority format is supported
     */
    pri &= G.primask;

    full_write(G.kmsgfd, G.printbuf, sprintf(G.printbuf, "<%d>%s
", pri, msg));-------加上pri，和msg写入/dev/kmsg。
}

5.3.3 ipc和logread功能

/* our shared key (syslogd.c and logread.c must be in sync) */
enum { KEY_ID = 0x414e4547 }; /* "GENA" */

static void ipcsyslog_cleanup(void)
{
    if (G.shmid != -1) {
        shmdt(G.shbuf);
    }
    if (G.shmid != -1) {
        shmctl(G.shmid, IPC_RMID, NULL);
    }
    if (G.s_semid != -1) {
        semctl(G.s_semid, 0, IPC_RMID, 0);
    }
}

static void ipcsyslog_init(void)
{
    if (DEBUG)
        printf("shmget(%x, %d,...)
", (int)KEY_ID, G.shm_size);

    G.shmid = shmget(KEY_ID, G.shm_size, IPC_CREAT | 0644);
    if (G.shmid == -1) {
        bb_perror_msg_and_die("shmget");
    }

    G.shbuf = shmat(G.shmid, NULL, 0);
    if (G.shbuf == (void*) -1L) { /* shmat has bizarre error return */
        bb_perror_msg_and_die("shmat");
    }

    memset(G.shbuf, 0, G.shm_size);
    G.shbuf->size = G.shm_size - offsetof(struct shbuf_ds, data) - 1;
    /*G.shbuf->tail = 0;*/

    /* we'll trust the OS to set initial semval to 0 (let's hope) */
    G.s_semid = semget(KEY_ID, 2, IPC_CREAT | IPC_EXCL | 1023);
    if (G.s_semid == -1) {
        if (errno == EEXIST) {
            G.s_semid = semget(KEY_ID, 2, 0);
            if (G.s_semid != -1)
                return;
        }
        bb_perror_msg_and_die("semget");
    }
}

/* Write message to shared mem buffer */
static void log_to_shmem(const char *msg)
{
    int old_tail, new_tail;
    int len;

    if (semop(G.s_semid, G.SMwdn, 3) == -1) {
        bb_perror_msg_and_die("SMwdn");
    }

    /* Circular Buffer Algorithm:
     * --------------------------
     * tail == position where to store next syslog message.
     * tail's max value is (shbuf->size - 1)
     * Last byte of buffer is never used and remains NUL.
     */
    len = strlen(msg) + 1; /* length with NUL included */
 again:
    old_tail = G.shbuf->tail;
    new_tail = old_tail + len;
    if (new_tail < G.shbuf->size) {
        /* store message, set new tail */
        memcpy(G.shbuf->data + old_tail, msg, len);
        G.shbuf->tail = new_tail;
    } else {
        /* k == available buffer space ahead of old tail */
        int k = G.shbuf->size - old_tail;
        /* copy what fits to the end of buffer, and repeat */
        memcpy(G.shbuf->data + old_tail, msg, k);
        msg += k;
        len -= k;
        G.shbuf->tail = 0;
        goto again;
    }
    if (semop(G.s_semid, G.SMwup, 1) == -1) {
        bb_perror_msg_and_die("SMwup");
    }
    if (DEBUG)
        printf("tail:%d
", G.shbuf->tail);
}

int logread_main(int argc, char **argv) MAIN_EXTERNALLY_VISIBLE;
int logread_main(int argc UNUSED_PARAM, char **argv)
{
    unsigned cur;
    int log_semid; /* ipc semaphore id */
    int log_shmid; /* ipc shared memory id */
    int follow = getopt32(argv, "fF");

    INIT_G();

    log_shmid = shmget(KEY_ID, 0, 0);
    if (log_shmid == -1)
        bb_perror_msg_and_die("can't %s syslogd buffer", "find");

    /* Attach shared memory to our char* */
    shbuf = shmat(log_shmid, NULL, SHM_RDONLY);
    if (shbuf == NULL)
        bb_perror_msg_and_die("can't %s syslogd buffer", "access");

    log_semid = semget(KEY_ID, 0, 0);
    if (log_semid == -1)
        error_exit("can't get access to semaphores for syslogd buffer");

    bb_signals(BB_FATAL_SIGS, interrupted);

    /* Suppose atomic memory read */
    /* Max possible value for tail is shbuf->size - 1 */
    cur = shbuf->tail;

    /* Loop for -f or -F, one pass otherwise */
    do {
        unsigned shbuf_size;
        unsigned shbuf_tail;
        const char *shbuf_data;
#if ENABLE_FEATURE_LOGREAD_REDUCED_LOCKING
        int i;
        int len_first_part;
        int len_total = len_total; /* for gcc */
        char *copy = copy; /* for gcc */
#endif
        if (semop(log_semid, SMrdn, 2) == -1)
            error_exit("semop[SMrdn]");

        /* Copy the info, helps gcc to realize that it doesn't change */
        shbuf_size = shbuf->size;
        shbuf_tail = shbuf->tail;
        shbuf_data = shbuf->data; /* pointer! */

        if (DEBUG)
            printf("cur:%u tail:%u size:%u
",
                    cur, shbuf_tail, shbuf_size);

        if (!(follow & 1)) { /* not -f */
            /* if -F, "convert" it to -f, so that we don't
             * dump the entire buffer on each iteration
             */
            follow >>= 1;

            /* advance to oldest complete message */
            /* find NUL */
            cur += strlen(shbuf_data + cur);
            if (cur >= shbuf_size) { /* last byte in buffer? */
                cur = strnlen(shbuf_data, shbuf_tail);
                if (cur == shbuf_tail)
                    goto unlock; /* no complete messages */
            }
            /* advance to first byte of the message */
            cur++;
            if (cur >= shbuf_size) /* last byte in buffer? */
                cur = 0;
        } else { /* -f */
            if (cur == shbuf_tail) {
                sem_up(log_semid);
                fflush_all();
                sleep(1); /* TODO: replace me with a sleep_on */
                continue;
            }
        }

        /* Read from cur to tail */
#if ENABLE_FEATURE_LOGREAD_REDUCED_LOCKING
        len_first_part = len_total = shbuf_tail - cur;
        if (len_total < 0) {
            /* message wraps: */
            /* [SECOND PART.........FIRST PART] */
            /*  ^data      ^tail    ^cur      ^size */
            len_total += shbuf_size;
        }
        copy = xmalloc(len_total + 1);
        if (len_first_part < 0) {
            /* message wraps (see above) */
            len_first_part = shbuf_size - cur;
            memcpy(copy + len_first_part, shbuf_data, shbuf_tail);
        }
        memcpy(copy, shbuf_data + cur, len_first_part);
        copy[len_total] = '';
        cur = shbuf_tail;
#else
        while (cur != shbuf_tail) {
            fputs(shbuf_data + cur, stdout);
            cur += strlen(shbuf_data + cur) + 1;
            if (cur >= shbuf_size)
                cur = 0;
        }
#endif
 unlock:
        /* release the lock on the log chain */
        sem_up(log_semid);

#if ENABLE_FEATURE_LOGREAD_REDUCED_LOCKING
        for (i = 0; i < len_total; i += strlen(copy + i) + 1) {
            fputs(copy + i, stdout);
        }
        free(copy);
#endif
        fflush_all();
    } while (follow);

    /* shmdt(shbuf); - on Linux, shmdt is not mandatory on exit */

    fflush_stdout_and_exit(EXIT_SUCCESS);
}

6. 小结

上述讲到了syslog机制的各种相关内容，kernel相关的有syslog系统调用、klogctl()、klogd；syslogd相关的有syslog()、syslogd、syslog.conf等。

在使用中有两个地方可以配置syslog行为：一是启动syslogd/klogd时设置选项；一是配置syslog.conf文件，rule项是越少越好。

在应用中调用syslog()输出日志。

参考文档：《linux日志：syslogd和klogd及syslog》、《syslogd以及syslog.conf文件解读说明》