Goaccess日志分析

最新推荐文章于 2024-08-04 17:45:00 发布

IT黑旋风

最新推荐文章于 2024-08-04 17:45:00 发布

阅读量7.3k

点赞数 1

分类专栏：日志分析工具文章标签： linux nginx 日志分析

本文链接：https://blog.csdn.net/u010391029/article/details/51002028

版权

日志分析工具专栏收录该内容

1 篇文章 0 订阅

订阅专栏

最近网站部署了zabbix监控平台2.4但是还是感觉zabbix的日志分析还是有点美中不足，这里又接触到了goacces感觉这个开源工具使用非常爽，再次记录一下他的相关介绍以及使用方法

至于安装这里我们可以采用官方yum源进行安装，如果找不到的可以给我留言，到时候发给您。

我这里web服务器为nginx。

首先先来介绍一下goacces，他的功能主要有

统计概况，流量消耗等
访客排名
动态Web请求
静态web请求，如图片、样式表、脚本等。
来路域名
404 错误
操作系统
浏览器和搜索引擎
主机、DNS和ip地址
HTTP 响应代码
引荐网站
键盘布局
自定义显示
支持超大日志（分析速度很快）

用法介绍

GoAccess的基本语法如下：

goaccess [ -b ][ -s ][ -e IP_ADDRESS][ - a ] <-f log_file >

参数说明：

-f – 日志文件名
-b – 开启流量统计，如果希望加快分析速度不建议使用该参数
-s – 开启HTTP响应代码统计
-a – 开启用户代理统计
-e – 开启指定IP地址统计，默认禁用
-p -指定goacces用户配置文件路径

可以直接输入goaccess -f +日志路径进入控制台，操作不多说，百度一大片。 IT网，http://www.it.net.cn

开始日志分析并导出为html文件：

1	[root@Rootop ~]# goaccess -f /home/wwwlogs/www.rootop.org.log --log-format='%h %^[%d:%^] "%r" %s %b "%R" "%u"' --date-format='%d/%b/%Y' -d -a > test.html

也可以通过指定配置文件方式来分析。就不需要在命令中体现日志格式：
首先下载配置文件：

1	[root@Rootop ~]# wget -O .goaccessrc https://raw.githubusercontent.com/allinurl/goaccess/master/config/goaccess.conf --no-check-certificate

编辑.goaccess文件，把16和27行的前面注释去掉。(NCSA Combined Log Format)

date-format %d/%b/%Y
log-format %h %^[%d:%t %^] "%r" %s %b "%R" "%u"
time-format %H:%M:%S

T网，http://www.it.net.cn

[root@Rootop ~]# goaccess -f /home/wwwlogs/www.rootop.org.log -d -a -p ~/.goaccessrc > test2.html IT网，http://www.it.net.cn

需要注意这里的日志格式，每个nginx日志格式配置可能不同，goaccess指定的日志格式也不一定一样，需要对照下面的参数和nginx日志格式来指定命令中日志的格式。否则日志
分析的时候会报错。需要对nginx的日志格式有一定了解。 Linux学习，http:// linux.it.net.cn

goaccess时间和日志格式支持的参数：
date_format
The date_format variable followed by a space, specifies the log format date containing any combination of regular characters and special format specifiers. They all
begin with a percentage (%) sign. See http://linux.die.net/man/3/strftime

Note that there is no need to use time specifiers since they are not used by GoAccess. It’s recommended to use only date specifiers, i.e., %Y-%m-%d. IT网，http://www.it.net.cn

log_format
The log_format variable followed by a space or \t , specifies the log format string. Linux学习，http:// linux.it.net.cn

%d date field matching the date_format variable.

%h host (the client IP address, either IPv4 or IPv6) IT网，http://www.it.net.cn

%r The request line from the client. This requires specific delimiters around the request (as single quotes, double quotes, or anything else) to be parsable. If not, we
have to use a combination of special format specifiers as %m %U %H.

%m The request method.

%U The URL path requested (including any query string).

%H The request protocol.

%s The status code that the server sends back to the client. IT网，http://www.it.net.cn

%b The size of the object returned to the client.

%R The “Referrer” HTTP request header.

%u The user-agent HTTP request header. Linux学习，http:// linux.it.net.cn

%D The time taken to serve the request, in microseconds.

%T The time taken to serve the request, in seconds or milliseconds. Note: %D will take priority over %T if both are used.

%^ Ignore this field. Linux学习，http:// linux.it.net.cn

PS:
之前配置文件错误，分析nginx日志报错，经过求助软件作者解决，并对goaccess有更深入的了解。赞一下软件作者。
[root@Rootop ~]# goaccess -d -f /home/wwwlogs/www.rootop.org.log -a -p ~/.goaccesssrc > test.html
Parsing… [373,734] [373,734/s]
GoAccess – version 0.8.5 – Nov 20 2014 16:48:39

Fatal error has occurred
Error occured at: goaccess.c – main – 832
Nothing valid to process. IT网，http://www.it.net.cn

~/.goaccesssrc文件内容：
date_format %d/%b/%Y
log_format %^:%^ %h %^[%d:%^] “%r” %s %b

nginx日志:
58.251.136.61 – - [20/Nov/2014:17:29:21 +0800] “GET /wp-content/themes/g-white/js/all.js HTTP/1.1″200 1292 “http://www.rootop.org/pages/890.html”"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.63 Safari/537.36″ -

三，详解日志文件

你可以按向下的箭头滚动页面，你会发现以下区域，它们是按请求排序的。这里提及的目录顺序可能会根据你的发行版或者你所选的安装方式（从源和库）不同而不同：

1，每天唯一访客（来自同样IP、同一日期和同一浏览器的请求被认为是是唯一访问）

 
 <code>
 1 - Unique visitors per day - Including spiders                                                                          Total: 5/5
 Hits having the same IP, date and agent are a unique visit.
 
  1 12.50%      0.0  B 14/May/2015 |||||||||||||||||||||||||||||||||||||||||||||||||
  1 12.50%      0.0  B 13/May/2015 |||||||||||||||||||||||||||||||||||||||||||||||||
  2 25.00%      0.0  B 12/May/2015 |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
  2 25.00%      0.0  B 11/May/2015 |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
  2 25.00%      0.0  B 10/May/2015 |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
</code>

2，请求的文件（网页URL）

 
 <code>
2 - Requested files (Pages-URL)                                                                                        Total: 46/46
 Top Requested Files sorted by hits - [time served] [protocol] [method]
 
  70 13.18%      0.0  B HTTP/1.1 GET /
  41 7.72%       0.0  B HTTP/1.1 GET /wp-content/themes/yusi1.0/style.css?ver=1.0
  29 5.46%       0.0  B HTTP/1.1 GET /wp-content/themes/yusi1.0/timthumb.php?src=http://vbird.dic.ksu.edu.tw/linux_server/0110network_
  27 5.08%       0.0  B HTTP/1.1 GET /wp-content/themes/yusi1.0/js/jquery.js?ver=1.0
  25 4.71%       0.0  B HTTP/1.1 GET /wp-content/themes/yusi1.0/timthumb.php?src=http://192.168.2.101/wp-content/uploads/2015/04/dis
  23 4.33%       0.0  B HTTP/1.1 GET /wp-content/themes/yusi1.0/timthumb.php?src=http://192.168.2.101/wp-content/themes/yusi1.0/img/pi
  22 4.14%       0.0  B HTTP/1.1 GET /wp-content/themes/yusi1.0/timthumb.php?src=http://192.168.2.101/wp-content/themes/yusi1.0/img/pi
</code>

3，请求的静态文件（例如，.png文件，.js文件等等）

 
 <code>
 3 - Requested static files (e.g., png, js, css, etc.)                                                                    Total: 7/7
 Top Static Files sorted by hits - [time served] [protocol] [method]
 
  4 0.75%      0.0  B HTTP/1.1 GET /wp-content/uploads/2015/04/dis﻾H4﻾I-300x162.jpg
  4 0.75%      0.0  B HTTP/1.1 GET /wp-content/themes/yusi1.0/js/html5.js
  4 0.75%      0.0  B HTTP/1.1 GET /wp-content/uploads/2015/04/dis﻾H1﻾I-300x162.jpg
  4 0.75%      0.0  B HTTP/1.1 GET /wp-content/uploads/2015/04/dis﻾H2﻾I-300x162.jpg
  4 0.75%      0.0  B HTTP/1.1 GET /wp-content/themes/yusi1.0/share.css
  3 0.56%      0.0  B HTTP/1.1 GET /wp-content/themes/yusi1.0/img/default.png
  3 0.56%      0.0  B HTTP/1.1 GET /wp-content/uploads/2015/04/dis﻾H5﻾I-300x162.jpg
</code>

4，HTTP 404 未找到的响应代码

 
 <code>
 4 - HTTP 404 Not Found URLs                                                                                              Total: 5/5
 Top 404 Not Found URLs sorted by hits - [time served] [protocol] [method]
 
  14 2.64%      0.0  B HTTP/1.1 GET /
  8  1.51%      0.0  B HTTP/1.1 GET /favicon.ico
  6  1.13%      0.0  B HTTP/1.1 GET /forum.php
  1  0.19%      0.0  B HTTP/1.1 GET /awstates/
  1  0.19%      0.0  B HTTP/1.1 GET /awstates
</code>

5，主机地址（客户端ip地址）

 
 <code>
5 - Hosts                                                                                                                Total: 8/8
 Top Hosts sorted by hits - [bandwidth] [time served]
 
  418 78.72%      0.0  B 192.168.2.168 |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
  64  12.05%      0.0  B 192.168.2.119 ||||||||||||||
  34  6.40%       0.0  B 192.168.2.101 |||||||
  6   1.13%       0.0  B 192.168.0.210 |
  5   0.94%       0.0  B 192.168.0.70  |
  2   0.38%       0.0  B 192.168.0.71  |
  1   0.19%       0.0  B 192.168.0.211 |
</code>

6，操作系统

 
 <code>
 6 - Operating Systems                                                                                                    Total: 1/1
 Top Operating Systems sorted by unique visitors
 
  8 100.00% Unknown ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
</code>

7，浏览器

 
 <code>
7 - Browsers                                                                                                             Total: 1/1
 Top Browsers sorted by unique visitors
 
  8 100.00% Unknown ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
</code>

8，引荐网址

 
 <code>
 8 - Referrers URLs                                                                                                       Total: 0/0
 Top Requested Referrers sorted by hits
</code>

9，参考网站

 
 <code>
 9 - Referring Sites                                                                                                      Total: 0/0
 Top Referring Sites sorted by hits
</code>

,10，来自谷歌搜索引擎的前几位的关键字

 
 <code>
 10 - Keyphrases from Google's search engine                                                                              Total: 0/0
 Top Keyphrases sorted by hits
</code>

11，HTTP 状态代码

 
 <code>
11 - HTTP Status Codes                                                                                                   Total: 7/7
 Top HTTP Status Codes sorted by hits
 
  294 55.37% 4xx Client Error
  123 23.16% 2xx Success
  91  17.14% 3xx Redirection
  23  4.33%  5xx Server Error
</code>