生产环境不重新编译nginx添加lua插件模块收集resp日志

最近为了查app问题，生产环境上app不打开日志，所以只能靠服务器查接口问题，服务器上的接口有nginx的request请求日志。在nginx上配置了

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent $request_body "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for" $request_time $upstream_response_time $upstream_addr $upstream_status';

但是没有resp信息，要显示resp信息需要写脚本，网上的方案都是编译nginx添加进去lua模块，但是目前nginx已经在使用不想重新编译，最后在Google上找到了解决办法。就是下载安装nginx版本，查看已经安装的参数使用nginx -V查看。然后重新编译好后，不进行make install，把相应模块拷贝到已经安装目录。具体步骤如下

nginx -v

nginx version: nginx/1.16.1
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-39) (GCC) 
built with OpenSSL 1.0.2k-fips  26 Jan 2017
TLS SNI support enabled
configure arguments: --prefix=/usr/share/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --http-client-body-temp-path=/var/lib/nginx/tmp/client_body --http-proxy-temp-path=/var/lib/nginx/tmp/proxy --http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi --http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi --http-scgi-temp-path=/var/lib/nginx/tmp/scgi --pid-path=/run/nginx.pid --lock-path=/run/lock/subsys/nginx --user=nginx --group=nginx --with-file-aio --with-ipv6 --with-http_ssl_module --with-http_v2_module --with-http_realip_module --with-stream_ssl_preread_module --with-http_addition_module --with-http_xslt_module=dynamic --with-http_image_filter_module=dynamic --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_random_index_module --with-http_secure_link_module --with-http_degradation_module --with-http_slice_module --with-http_stub_status_module --with-http_perl_module=dynamic --with-http_auth_request_module --with-mail=dynamic --with-mail_ssl_module --with-pcre --with-pcre-jit --with-stream=dynamic --with-stream_ssl_module --with-google_perftools_module --with-debug --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic' --with-ld-opt='-Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,-E'

下载nginx源码 http://nginx.org/en/download.html

下载luajit，官网地址https://luajit.org/

下载lua-nginx-module https://github.com/openresty/lua-nginx-module/tags?after=v0.10.12

下载ngx_devel_kit https://github.com/vision5/ngx_devel_kit/archive/v0.3.1.tar.gz

tar -zxvf 解压后执行 make install PREFIX=/usr/local/luajit

导出luajit环境变量编辑/etc/profile

export LUAJIT_LIB=/usr/local/luajit/lib 
export LUAJIT_INC=/usr/local/luajit/include/luajit-2.0 
export LD_LIBRARY_PATH=/usr/local/lib/:$LD_LIBRARY_PATH
 

./configure --add-module=/opt/ngx_devel_kit --add-module=/opt/lua-nginx-module --prefix=/usr/share/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --http-client-body-temp-path=/var/lib/nginx/tmp/client_body --http-proxy-temp-path=/var/lib/nginx/tmp/proxy --http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi --http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi --http-scgi-temp-path=/var/lib/nginx/tmp/scgi --pid-path=/run/nginx.pid --lock-path=/run/lock/subsys/nginx --user=nginx --group=nginx --with-file-aio --with-ipv6 --with-http_ssl_module --with-http_v2_module --with-http_realip_module --with-stream_ssl_preread_module --with-http_addition_module --with-http_xslt_module=dynamic --with-http_image_filter_module=dynamic --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_random_index_module --with-http_secure_link_module --with-http_degradation_module --with-http_slice_module --with-http_stub_status_module --with-http_perl_module=dynamic --with-http_auth_request_module --with-mail=dynamic --with-mail_ssl_module --with-pcre --with-pcre-jit --with-stream=dynamic --with-stream_ssl_module --with-google_perftools_module --with-debug --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic' --with-ld-opt='-Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,-E'

这个就是使用yum instal的命令输出，会遇到不少错误

1、/usr/lib/rpm/redhat/redhat-hardened

yum install redhat-rpm-config

2、遇到the HTTP XSLT module requires the libxml2/libxslt安装

yum -y install libxml2 libxml2-dev
yum -y install libxslt-devel

3、遇到the HTTP image filter module requires the GD library

yum install gd-devel
4、遇到perl module ExtUtils::Embed is required

yum -y install perl-devel perl-ExtUtils-Embed

5、遇到error: the Google perftools module requires the Google perftools

yum install gperftools

6、遇到error: the GeoIP module requires the GeoIP library

yum -y install GeoIP GeoIP-devel GeoIP-data

 

7、检查通过执行make，但不要执行make install

原来nginx先停掉，systemctl stop nginx

原来先备份 cp /usr/sbin/nginx /usr/sbin/nginx.bak

启动如果遇到nginx lua 找不到libluajit-5.1.so.2，使用ldd /usr/sbin/nginx命令查看链接

发现libluajit-5.1.so.2 => not found，这个可以通过手动加入vim /etc/ld.so.conf.d/libc.conf

输入/usr/local/luajit/lib /usr/local/lib 执行ldconfig生效

8、错误no file '/usr/local/luajit/lib/lua/5.1/resty/core.so'
no file '/usr/local/lib/lua/5.1/loadall.so'
no file './resty.so'
no file '/usr/local/lib/lua/5.1/resty.so'
no file '/usr/local/luajit/lib/lua/5.1/resty.so'
no file '/usr/local/lib/lua/5.1/loadall.so') in /etc/nginx/nginx.conf:128

因为最新版本lua-nginx-module依赖了openresty库，目前如果不需要openresty可以在github选取比较早的版本，我是因为

Luajit2.0.5是2017左右，所以也选取在2017年左右的版本，后面再次编译后就可以启动了。

接下来需要在nginx 添加脚本获取返回内容，在nginx配置文件里配置

1、日志文件格式添加 ‘$resp_body’

2、添加脚本代码

            lua_need_request_body on;
            body_filter_by_lua ' 
              local resp_body = string.sub(ngx.arg[1],1,1000)
              ngx.ctx.buffered = (ngx.ctx.buffered or "") .. resp_body
              if ngx.arg[2] then
                 ngx.var.resp_body = ngx.ctx.buffered
              end
            ';

 

返回内容是以16进制格式进行utf-8编码过的，需要解码。简单点在网上有在线编解码工具。也可以使用vscode写个pythond脚本

str1='"{\x22result\x22:0,\x22message\x22:\x22\xE8\xAF\xB7\xE6\xB1\x82\xE6\x88\x90\xE5\x8A\x9F\x22,\x22data\x22:0}"'

print(str1.encode('raw_unicode_escape').decode('utf-8'))