awstats分析nginx日志

1.首先修改nginx日志格式为combined格式：
#    log_format  combined
#                          '$remote_addr - $remote_user [$time_local] '
#                          '"$request" $status $apache_bytes_sent '
#                          '"$http_referer" "$http_user_agent"';
#    log_format  main
#                      '$remote_addr - $remote_user [$time_local] "$request" '
#                      '$status $body_bytes_sent "$http_referer" '
#                      '"$http_user_agent" "$http_x_forwarded_for"';

#    access_log  logs/access.log  main;
    access_log  logs/access.log combined ;

2 安装awstats
tar -zxvf awstats-7.0.tar.gz
mkdir -p /usr/local/awstats
mv ./awstats-7.0/*  /usr/local/awstats
cd  /usr/local/awstats/tools
[root@XKWB5510 tools]# perl awstats_configure.pl



需要注意的地方是：

Check for web server install  这项填none。
Define config file name to create  这项填配置文件名字,一般用域名。

其余默认即可。

3.修改建成的配置文件astats.www.nginx.log
LogFile="LogFile="/usr/local/awstats/tools/logresolvemerge.pl /usr/local/nginx/logs/231.pcstars_access.log /usr/local/nginx/logs/231.pcstars_access.log /usr/local/nginx/logs/231.pcstars_access.log /usr/local/nginx/logs/231.pcstars_access.log /usr/local/nginx/logs/231.pcstars_access.log|"

4 修改完awstats的配置文件以后要更新

perl /usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=www.nginx.log
或 

[root@XKWB5510 logs]# /usr/local/awstats/tools/awstats_updateall.pl now
Running '"/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -update -config=www.nginx.log -configdir="/etc/awstats"' to update config www.nginx.log
Create/Update database for config "/etc/awstats/awstats.www.nginx.log.conf" by AWStats version 7.0 (build 1.971)
From data in log file "/usr/local/awstats/tools/logresolvemerge.pl /usr/local/nginx/logs/231.pcstars_access.log /usr/local/nginx/logs/231.pcstars_access.log /usr/local/nginx/logs/231.pcstars_access.log /usr/local/nginx/logs/231.pcstars_access.log /usr/local/nginx/logs/231.pcstars_access.log|"...
Phase 1 : First bypass old records, searching new record...
Direct access to last remembered record has fallen on another record.
So searching new records from beginning of log file...
Jumped lines in file: 0
Parsed lines in file: 48864
 Found 0 dropped records,
 Found 0 comments,
 Found 0 blank records,
 Found 0 corrupted records,
 Found 48864 old records,
 Found 0 new qualified records.

5 执行成功以后，就可以利用 awstats 的工具将统计的结果生成静态文件，访问web网页的时候，就是读取的这个静态文件
[root@XKWB5510 awstats]# perl /usr/local/awstats/tools/awstats_buildstaticpages.pl -update -config=www.nginx.log -lang=cn -dir=/data/webroot/awstats/  -awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl 
Launch update process : "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -update -configdir=
Build main page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output
Build alldomains page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=alldomains
Build allhosts page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=allhosts
Build lasthosts page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=lasthosts
Build unknownip page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=unknownip
Build allrobots page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=allrobots
Build lastrobots page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=lastrobots
Build session page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=session
Build urldetail page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=urldetail
Build urlentry page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=urlentry
Build urlexit page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=urlexit
Build osdetail page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=osdetail
Build unknownos page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=unknownos
Build browserdetail page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=browserdetail
Build unknownbrowser page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=unknownbrowser
Build refererse page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=refererse
Build refererpages page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=refererpages
Build keyphrases page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=keyphrases
Build keywords page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=keywords
Build errors404 page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.nginx.log -staticlinks -lang=cn -output=errors404
20 files built.
Main HTML page is 'awstats.www.nginx.log.html'.

根据提示可以看到主要生成的静态页面为 awstats.www.nginx.log.html
我们访问web的时候的链接是：http://IP/awstats/awstats.www.nginx.log.html

6.以下是测试时报错：

]# perl /usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=.www.nginx.log
Error: AWStats database directory defined in config file by 'DirData' parameter (/var/lib/awstats) does not exist or is not writable.
Setup ('/etc/awstats/awstats..www.nginx.log.conf' file, web server or permissions) may be wrong.
Check config file, permissions and AWStats documentation (in 'docs' directory).
解决：建一个awstats用于记录数据的目录
[root@test1 awstats]# mkdir /var/lib/awstats

另一种常见报错就是日志格式问题：
[root@XKWB5510 logs]# /usr/local/awstats/tools/awstats_updateall.pl now
Running '"/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -update -config=www.nginx.log -configdir="/etc/awstats"' to update config www.nginx.log
Create/Update database for config "/etc/awstats/awstats.www.nginx.log.conf" by AWStats version 7.0 (build 1.971)
From data in log file "/usr/local/awstats/tools/logresolvemerge.pl /usr/local/nginx/logs/*.pcstars_access.log|"...
Phase 1 : First bypass old records, searching new record...
Direct access to last remembered record has fallen on another record.
So searching new records from beginning of log file...
AWStats did not find any valid log lines that match your LogFormat parameter, in the 50th first non commented lines read of your log.
Your log file /usr/local/awstats/tools/logresolvemerge.pl /usr/local/nginx/logs/*.pcstars_access.log| must have a bad format or LogFormat parameter setup does not match this format.
Your AWStats LogFormat parameter is:
1
This means each line in your web server log file need to have "combined log format" like this:
111.22.33.44 - - [10/Jan/2001:02:14:14 +0200] "GET / HTTP/1.1" 200 1234 "http://www.fromserver.com/from.htm" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)"
And this is an example of records AWStats found in your log file (the record number 50 in your log):
59.36.213.132 - - [14/Sep/2011:00:00:17 +0800] GET /lm/3008-p610.html HTTP/1.1 302 5 -Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) -
Setup ('/etc/awstats/awstats.www.nginx.log.conf' file, web server or permissions) may be wrong.
Check config file, permissions and AWStats documentation (in 'docs' directory).

 
7  进行nginx日志分割，这个在这里不做解释

8 将下列写入计划任务，执行时间根据实际生产需求
/usr/local/awstats/tools/awstats_updateall.pl now   
perl /usr/local/awstats/tools/awstats_buildstaticpages.pl -update -config=www.nginx.log -lang=cn -dir=/data/webroot/awstats/  -awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl

最后浏览http://IP/awstats/awstats.www.nginx.log.html