Nginx配置Awstats分析Nginx日志笔记

1、修改Nginx日志格式：

log_format json '$remote_addr - $remote_user [$time_local] "$request" '
             '$status $body_bytes_sent "$http_referer" '
             '"$http_user_agent" "$http_x_forwarded_for"';
             
access_log /data/nginx_logs/access.log json;

2、Nginx日志切割（shell脚本，略）

3、安装GeoIP库

yum -y install GeoIP GeoIP-devel perl-Geo-IP

4、安装Awstats

tar xvf awstats-7.4.tar.gz
mv awstats-7.4 /usr/local/awstats
cd /usr/local/
chown root:root -R awstats/
chmod +x /usr/local/awstats/tools/*.pl
chmod +x /usr/local/awstats/wwwroot/cgi-bin/*.pl

5、运行脚本生成配置

cd /usr/local/awstats/tools/
./awstats_configure.pl

脚本交互1：

Config file path ('none' to skip web server setup):
因为在此我们使用的是nginx，所以填写none

脚本交互2：

Do you want me to build a new AWStats config/profile  
file (required if first install) [y/N]

输入Y创建一个新的统计配置文件。

脚本交互3：

Your web site, virtual server or profile name:  
> www.test.com 
在这输入自己的网站域名

脚本交互4：

Directory path to store config file(s) (Enter for default):  
> 
使用默认配置，生成配置文件

后面的直接按回车就可以

6、修改上面生成的配置文件/etc/awstats/awstats.app.mir.6wtx.com.conf

LogFile="/data/nginx_logs/cut_logs/%YYYY-24%MM-24%DD-24/
DirData="/data/awstats" 
LoadPlugin="geoip GEOIP_STANDARD /usr/share/GeoIP/GeoIP.dat"
LoadPlugin="geoip_city_maxmind GEOIP_STANDARD /usr/share/GeoIP/GeoLiteCity.dat"

7、生成数据文件并配置Nginx，此处应该有两种办法：

一种是用fastcgi调用perl分析数据文件

一种是直接生成静态文件nginx解析

第一种办法：

7.1.1、安装FCGI和FCGI::ProcManager

cpan>install FCGI
cpan>install FCGI::ProcManager

7.1.2、创建fastcgi来执行perl：/usr/local/nginx/sbin/fcgi

#!/usr/bin/perl
use FCGI;
#perl -MCPAN -e 'install FCGI'
use Socket;
use POSIX qw(setsid);
#use Fcntl;
require 'syscall.ph';
&daemonize;
#this keeps the program alive or something after exec'ing perl scripts
END() { } BEGIN() { }
*CORE::GLOBAL::exit = sub { die "fakeexit
rc=".shift()."
"; };
eval q{exit};
if ($@) {
        exit unless $@ =~ /^fakeexit/;
};
&main;
sub daemonize() {
    chdir '/'                 or die "Can't chdir to /: $!";
    defined(my $pid = fork)   or die "Can't fork: $!";
    exit if $pid;
    setsid                    or die "Can't start a new session: $!";
    umask 0;
}
sub main {
        #$socket = FCGI::OpenSocket( "127.0.0.1:8999", 10 );
        $socket = FCGI::OpenSocket( "/usr/local/nginx/fastcgi_temp/perl_cgi-dispatch.sock", 10 );
#use UNIX sockets - user running this script must have w access to the 'nginx' folder!!
        $request = FCGI::Request( *STDIN, *STDOUT, *STDERR, \%req_params, $socket );
        if ($request) { request_loop()};
            FCGI::CloseSocket( $socket );
}
sub request_loop {
        while( $request->Accept() >= 0 ) {
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      
           #processing any STDIN input from WebServer (for CGI-POST actions)
           $stdin_passthrough ='';
           $req_len = 0 + $req_params{'CONTENT_LENGTH'};
           if (($req_params{'REQUEST_METHOD'} eq 'POST') && ($req_len != 0) ){
                my $bytes_read = 0;
                while ($bytes_read < $req_len) {
                        my $data = '';
                        my $bytes = read(STDIN, $data, ($req_len - $bytes_read));
                        last if ($bytes == 0 || !defined($bytes));
                        $stdin_passthrough .= $data;
                        $bytes_read += $bytes;
                }
            }
            #running the cgi app
            if ( (-x $req_params{SCRIPT_FILENAME}) && #can I execute this?
                 (-s $req_params{SCRIPT_FILENAME}) && #Is this file empty?
                 (-r $req_params{SCRIPT_FILENAME})     #can I read this file?
            ){
                pipe(CHILD_RD, PARENT_WR);
                my $pid = open(KID_TO_READ, "-|");
                unless(defined($pid)) {
                        print("Content-type: text/plain

");
                        print "Error: CGI app returned no output - Executing $req_params
{SCRIPT_FILENAME} failed !
";
                        next;
                }
                if ($pid > 0) {
                        close(CHILD_RD);
                        print PARENT_WR $stdin_passthrough;
                        close(PARENT_WR);
                        while(my $s = <KID_TO_READ>) { print $s; }
                        close KID_TO_READ;
                        waitpid($pid, 0);
                } else {
                        foreach $key ( keys %req_params){
                           $ENV{$key} = $req_params{$key};
                        }
                        # cd to the script's local directory
                        if ($req_params{SCRIPT_FILENAME} =~ /^(.*)/[^/]+$/) {
                                chdir $1;
                        }
                        close(PARENT_WR);
                        close(STDIN);
                        #fcntl(CHILD_RD, F_DUPFD, 0);
                        syscall(&SYS_dup2, fileno(CHILD_RD), 0);
                        #open(STDIN, "<&CHILD_RD");
                        exec($req_params{SCRIPT_FILENAME});
                        die("exec failed");
                }
            }
            else {
                print("Content-type: text/plain

");
                print "Error: No such CGI app - $req_params{SCRIPT_FILENAME} may not exist or is
not executable by this process.
";
            }
        }
}

7.1.3、授权：

chmod 755 /usr/local/nginx/sbin/fcgi

7.1.4、启动：

perl /usr/local/nginx/sbin/fcgi >/dev/null 2>&1

7.1.5、授权socker让Nginx调用：

chown www:www /usr/local/nginx/fastcgi_temp/perl_cgi-dispatch.sock

7.1.6、创建/usr/local/nginx/conf/fastcgi_params1

               fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
               fastcgi_param QUERY_STRING     $query_string;
               fastcgi_param REQUEST_METHOD   $request_method;
               fastcgi_param CONTENT_TYPE     $content_type;
               fastcgi_param CONTENT_LENGTH   $content_length;
               fastcgi_param GATEWAY_INTERFACE CGI/1.1;
               fastcgi_param SERVER_SOFTWARE    nginx;
               fastcgi_param SCRIPT_NAME        $fastcgi_script_name;
               fastcgi_param REQUEST_URI        $request_uri;
               fastcgi_param DOCUMENT_URI       $document_uri;
               fastcgi_param DOCUMENT_ROOT      $document_root;
               fastcgi_param SERVER_PROTOCOL    $server_protocol;
               fastcgi_param REMOTE_ADDR        $remote_addr;
               fastcgi_param REMOTE_PORT        $remote_port;
               fastcgi_param SERVER_ADDR        $server_addr;
               fastcgi_param SERVER_PORT        $server_port;
               fastcgi_param SERVER_NAME        $server_name;
               fastcgi_read_timeout 60;

7.1.7、Nginx增加虚拟主机：

server {
                listen      33333;
                server_name  xxxxxxxxxxx;
 
                 location / {
                        root   /data/awstats;
                        index  index.html index.htm;
                }
 
 
                location ~* ^/cgi-bin/.*.pl$ {
                        root /usr/local/awstats/wwwroot;
                        fastcgi_pass unix:/usr/local/nginx/fastcgi_temp/perl_cgi-dispatch.sock;
                        fastcgi_index index.pl;
                        include  fastcgi_params1;
                        charset gb2312;
                }
                location ~ ^/icon/ {        # 图标目录
                        root   /usr/local/awstats/wwwroot;
                        index  index.html;
                        access_log off;
                        error_log off;
                }
        }

最后重启Nginx

7.1.8、生成awstats数据文件：

/usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=www.test.com

7.1.9、打开URL查看效果：http://xxxxxxxxx:33333/cgi-bin/awstats.pl?config=www.test.com

 

第二种办法：

7.2.1、配置Nginx，不需要调用perl

server {
                listen       44444;
                server_name  xxxxxxxxxxx;
                root   /data/awstats;
                index  index.html;
                access_log off;
                error_log off;
                charset gb2312;

                location ~ ^/icon/ {        # 图标目录
                        root   /usr/local/awstats/wwwroot;
                        index  index.html;
                        access_log off;
                        error_log off;
                }
        }

重启Nginx

7.2.2、生成awstats静态数据文件：

/usr/local/awstats/tools/awstats_buildstaticpages.pl -update -config=www.test.com -lang=cn -dir=/data/awstats -awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl

7.2.3、 打开URL查看效果：http://xxxxxxxxxx:44444/awstats.www.test.com.html

8、添加定时任务：

5 0 * * * /usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=www.sunsky.com >/dev/null 2>&1
10 0 * * * /usr/local/awstats/tools/awstats_buildstaticpages.pl -update -config=www.test.com -lang=cn -dir=/data/awstats -awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl >/dev/null 2>&1

本文出自 “运维笔记” 博客，请务必保留此出处http://lihuipeng.blog.51cto.com/3064864/1764467