goaccess的安装和使用

2019独角兽企业重金招聘Python工程师标准>>>

一.简介
1.goaccess用于分析apache和nginx日志的强大工具

2.官网:
https://goaccess.io

二.安装
1.下载goaccess的安装包
wget http://tar.goaccess.io/goaccess-1.1.1.tar.gz

2.如果需要启用geoip,则1.1的版本,需要geoip.1.4.7的版本,编译安装geoip.1.4.7

cd /usr/local/src
wget http://geolite.maxmind.com/download/geoip/api/c/GeoIP-1.4.7.tar.gz
wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz

tar xzvf GeoIP-1.4.7.tar.gz
cd GeoIP-1.4.7
./configure && make && make install
mv GeoIP.dat.gz /usr/local/share/GeoIP/

3.编译安装goaccess

tar -xzvf goaccess-0.8.1.tar.gz
cd goaccess-0.8.1/
tar -xzvf goaccess-1.1.1.tar.gz 
cd goaccess-1.1.1/
./configure --enable-geoip --enable-utf8 
make
make install

如果出现
goaccess: error while loading shared libraries: libGeoIP.so.1
错误

ln -s /usr/local/lib/libGeoIP.so* /lib64/

三.使用
1.简单使用
goaccess -f access.log

2.使用指定的配置文件
wget -O .goaccessrc https://raw.githubusercontent.com/allinurl/goaccess/master/config/goaccess.conf --no-check-certificate

vi .goaccessrc
选择和编辑合适的date-format,log-format,time-format

goaccess -f access.log -d -a -p ~/.goaccessrc

 

输出到文件便于web查看

goaccess -f access.log -d -a -p ~/.goaccessrc > log.html

 

更多详细的使用:https://goaccess.io/man

 

Goaccess解析nginx日志备忘

参考

http://nginx.org/en/docs/http/ngx_http_log_module.html?&_ga=1.92028562.949762386.1481787781#log_format
https://www.goaccess.io/man#custom-log

展开

goaccess默认的日志格式与nginx的默认格式一致

%h %^[%d:%t %^] "%r" %s %b "%R" "%u"

log_format combined '$remote_addr - $remote_user [$time_local] '
                    '"$request" $status $body_bytes_sent '
                    '"$http_referer" "$http_user_agent"';

自定义日志格式

现在增加一个响应时间, 删除$remote_user, 并把$body_bytes_sent修改为$bytes_sent,

修改nginx.conf

log_format myfmt '$remote_addr [$time_local] '
                    '"$request" $status $bytes_sent '
                    '"$http_referer" "$http_user_agent" $request_time';
access_log logs/access.log myfmt;

修改goaccess 配置文件

# vim /usr/local/etc/goaccess.conf 
time-format %H:%M:%S
date-format %d/%b/%Y
log-format %h [%d:%t %^] "%r" %s %b "%R" "%u" %T

执行示例

cat logs/access.log | goaccess -o ~/static/report.html

goaccess -f logs/access.log -o ~/static/report.html --real-time-html

变量说明

1.$remote_addr 与$http_x_forwarded_for 用以记录客户端的ip地址；
 2.$remote_user ：用来记录客户端用户名称；
 3.$time_local ： 用来记录访问时间与时区；
 4.$request ： 用来记录请求的url与http协议；
 5.$status ： 用来记录请求状态；成功是200，
 6.$body_bytes_sent ：记录发送给客户端文件主体内容大小；
 7.$http_referer ：用来记录从那个页面链接访问过来的；
 8.$http_user_agent ：记录客户端浏览器的相关信息；

nginx日志相关

Syntax: log_format name [escape=default|json] string ...;
Default:
log_format combined "...";
Context: http
Specifies log format.

The escape parameter (1.11.8) allows setting json or default characters escaping in variables, by default, default escaping is used.

The log format can contain common variables, and variables that exist only at the time of a log write:

$bytes_sent
the number of bytes sent to a client
$connection
connection serial number
$connection_requests
the current number of requests made through a connection (1.1.18)
$msec
time in seconds with a milliseconds resolution at the time of the log write
$pipe
“p” if request was pipelined, “.” otherwise
$request_length
request length (including request line, header, and request body)
$request_time
request processing time in seconds with a milliseconds resolution; time elapsed between the first bytes were read from the client and the log write after the last bytes were sent to the client
$status
response status
$time_iso8601
local time in the ISO 8601 standard format
$time_local
local time in the Common Log Format
In the modern nginx versions variables $status (1.3.2, 1.2.2), $bytes_sent (1.3.8, 1.2.5), $connection (1.3.8, 1.2.5), $connection_requests (1.3.8, 1.2.5), $msec (1.3.9, 1.2.6), $request_time (1.3.9, 1.2.6), $pipe (1.3.12, 1.2.7), $request_length (1.3.12, 1.2.7), $time_iso8601 (1.3.12, 1.2.7), and $time_local (1.3.12, 1.2.7) are also available as common variables.
Header lines sent to a client have the prefix “sent_http_”, for example, $sent_http_content_range.

The configuration always includes the predefined “combined” format:

log_format combined '$remote_addr - $remote_user [$time_local] '
'"$request" $status $body_bytes_sent '
'"$http_referer" "$http_user_agent"';

转载于:https://my.oschina.net/mickelfeng/blog/1922660