如何使用prometheus监控nginx

[toc]

需求场景

使用prometheus监控nginx,图表展示使用grafana。

分析

nginx我们使用tengine(https://github.com/alibaba/tengine)

nginx prometheus模块使用nginx-module-vts(https://github.com/vozlt/nginx-module-vts)

exportor使用nginx-vts-exporter(https://github.com/hnlq715/nginx-vts-exporter)

(其实这个exportor也可以不用,因为默认的vts已经提供了prometheus的metrics地址,这个exportor只是为了生成这个地址)

nginx以及插件部署

下载文件
git clone https://github.com/alibaba/tengine
git clone https://github.com/vozlt/nginx-module-vts
yum -y install zlib* library*
yum -y install openssl openssl-devel
---------
[root@10-17-41-57 software]# ls
nginx-module-vts tengine
构建tengine,带上vts
[root@10-17-41-57 software]# cd tengine
[root@10-17-41-57 tengine]# ./configure --add-module=/root/software/nginx-module-vts --prefix=/usr/local/nginx --user=nginx --group=nginx --with-stream --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_v2_module --with-http_ssl_module
[root@10-17-41-57 tengine]# make
[root@10-17-41-57 tengine]# make install
[root@10-17-41-57 tengine]# useradd www
配置nginx

nginx.conf中的http块中加入下面代码:

# 开启基础监控
vhost_traffic_status_zone;
# 开启状态码过滤
vhost_traffic_status_filter on;
# 开启此功能,在Nginx配置有多个server_name的情况下,会根据不同的server_name进行流量的统计,否则默认会把流量全部计算到第一个server_name上
vhost_traffic_status_filter_by_host on;

新增nginx-vts-status.conf文件:

server {        
listen 3389;

location /mt_status {
vhost_traffic_status_display;
vhost_traffic_status_display_format html;
}

}

启动nginx:

[root@10-17-41-57 software]# /usr/local/nginx/sbin/nginx -t
[root@10-17-41-57 software]# /usr/local/nginx/sbin/nginx
查看原始nginx输出数据
[root@10-17-41-57 software]# curl https://localhost:3389/mt_status/format/prometheus

# HELP nginx_vts_start_time_seconds Nginx start time
# TYPE nginx_vts_start_time_seconds gauge
nginx_vts_start_time_seconds 1600401217.372
# HELP nginx_vts_main_connections Nginx connections
# TYPE nginx_vts_main_connections gauge
nginx_vts_main_connections{status="accepted"} 201
nginx_vts_main_connections{status="active"} 98
nginx_vts_main_connections{status="handled"} 201
nginx_vts_main_connections{status="reading"} 0
nginx_vts_main_connections{status="requests"} 10
nginx_vts_main_connections{status="waiting"} 97
nginx_vts_main_connections{status="writing"} 1
加一个测试的nginx配置文件
server {
listen 80;
server_name mtlab-nginx-test.meitu.com;

proxy_ignore_client_abort on;
# 开启详细状态码统计
vhost_traffic_status_filter_by_set_key $status $server_name;
# 开启uri统计
vhost_traffic_status_filter_by_set_key $uri uris::$server_name;

location / {
return 200;
}

}

配置好上述的参数,我们可以在输出看到一些更加详细的信息

png1
备注(nginx.conf文件)
user  www www;

worker_processes auto;

error_log /usr/local/nginx/logs/nginx_error.log error;
pid /usr/local/nginx/nginx.pid;

worker_rlimit_nofile 65535;

events
{
use epoll;
worker_connections 65535;
}

http {
include mime.types;
default_type application/octet-stream;
vhost_traffic_status_zone;


log_format api '$time_iso8601 $remote_addr - $host "$request" $status - $request_time $http_x_real_ip "$http_x_forwarded_for" $content_length $request_length $sent_http_content_length $body_bytes_sent $http_cdn "$http_referer" "$http_user_agent" $upstream_addr $upstream_status $upstream_response_time $sent_http_request_id';

log_format json '$time_iso8601 $remote_addr - $host "$request" $status - $request_time $http_x_real_ip "$http_x_forwarded_for" $content_length $request_length $sent_http_content_length $body_bytes_sent $http_cdn "$http_referer" "$http_user_agent" $upstream_addr $upstream_status $upstream_response_time $sent_http_request_id';

log_format mtlog '$time_iso8601 $remote_addr $server_addr $host "$request" $status - $request_time $http_x_real_ip "$http_x_forwarded_for" $content_length $request_length $sent_http_content_length $body_bytes_sent $http_cdn "$http_referer" "$http_user_agent" $upstream_addr $upstream_status $upstream_response_time $sent_http_request_id $http_request_id';

log_format toamtlog '$time_iso8601 $server_addr $host "$request" $status - $request_time $http_x_real_ip "$http_x_forwarded_for" $content_length $request_length $sent_http_content_length $body_bytes_sent $http_cdn "$http_referer" "$http_user_agent" $upstream_addr $upstream_status $upstream_response_time $sent_http_request_id $http_request_id';

log_format mtlog_noarg '$time_iso8601 $remote_addr $server_addr $host "$request_method $server_protocol" $status - $request_time $http_x_real_ip "$http_x_forwarded_for" $content_length $request_length $sent_http_content_length $body_bytes_sent $http_cdn "$http_referer" "$http_user_agent" $upstream_addr $upstream_status $upstream_response_time $sent_http_request_id $http_request_id';

log_format post '$time_iso8601 $remote_addr $server_addr $host "$request" $status - $request_time $http_x_real_ip "$http_x_forwarded_for" $content_length $request_length $sent_http_content_length $body_bytes_sent $http_cdn "$http_referer" "$http_user_agent" $upstream_addr $upstream_status $upstream_response_time $sent_http_request_id $request_body';

server_tokens off;
# charset gb2312;

server_names_hash_bucket_size 256;
client_header_buffer_size 32k;
large_client_header_buffers 4 32k;
client_max_body_size 100m;
client_body_buffer_size 10m;

sendfile on;
tcp_nopush off;

keepalive_timeout 600;
keepalive_requests 10000;

tcp_nodelay on;

#proxy_connect_timeout 20s;
#proxy_send_timeout 90s;
#proxy_read_timeout 90s;
proxy_connect_timeout 600s;
proxy_send_timeout 600s;
proxy_read_timeout 600s;
proxy_buffer_size 64k;
proxy_buffers 4 64k;
proxy_busy_buffers_size 128k;
proxy_temp_file_write_size 128k;
# proxy_buffering off;
# proxy_request_buffering off;

gzip on;
gzip_min_length 1k;
gzip_buffers 4 16k;
gzip_http_version 1.0;
gzip_comp_level 4;
gzip_types text/plain application/x-javascript text/css application/xml text/xml application/json application/javascript;
gzip_vary on;


ssi on;
ssi_silent_errors on;
ssi_types text/shtml;
ssi_last_modified on;

underscores_in_headers on;



# The following is a sneaky way to do "set $the_real_ip $remote_addr"
# Needed because using set is not allowed outside server blocks.

# We can't use $proxy_add_x_forwarded_for because the realip module
# replaces the remote_addr too soon
map $http_x_forwarded_for $full_x_forwarded_for {

default "$http_x_forwarded_for, $remote_addr";

}

# ws.live.meitu.com
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}

include upstream/*.conf;
include vhosts/*.conf;

limit_req_status 403;
limit_req_zone $remote_addr zone=mpstatplayvideo:100m rate=20r/s;
}

额外补充

如果你的nginx机器是多台,或者是动态的,那么这个是不能使用域名负载均衡来代理的,因为prometheus每次只能抓一条,即每次都只会获取一台机器的数据。

所以针对这个情况,两种方式:

1.如果你是容器化部署,那么他本身就有服务发现的机制,无所谓。

2.如果你是物理部署的,要嘛你写死prometheus的target配置,要嘛研究一下prometheus动态发现的原理。

一、prometheus简介 Prometheus是一个开源的系统监控和告警系统,现在已经加入到CNCF基金会,成为继k8s之后第二个在CNCF维护管理的项目,在kubernetes容器管理系统中,通常会搭配prometheus进行监控prometheus支持多种exporter采集数据,还支持通过pushgateway进行数据上报,Prometheus再性能上可支撑上万台规模的集群。 二、prometheus架构图 三、prometheus组件介绍 1.Prometheus Server: 用于收集和存储时间序列数据。 2.Client Library: 客户端库,检测应用程序代码,当Prometheus抓取实例的HTTP端点时,客户端库会将所有跟踪的metrics指标的当前状态发送到prometheus server端。 3.Exporters: prometheus支持多种exporter,通过exporter可以采集metrics数据,然后发送到prometheus server端 4.Alertmanager: 从 Prometheus server 端接收到 alerts 后,会进行去重,分组,并路由到相应的接收方,发出报警,常见的接收方式有:电子邮件,微信,钉钉, slack等。 5.Grafana:监控仪表盘 6.pushgateway: 各个目标主机可上报数据到pushgatewy,然后prometheus server统一从pushgateway拉取数据。 四、课程亮点 五、效果图展示 六、讲师简介 先超(lucky):高级运维工程师、资深DevOps工程师,在互联网上市公司拥有多年一线运维经验,主导过亿级pv项目的架构设计和运维工作 主要研究方向: 1.云计算方向:容器 (kubernetes、docker),虚拟化(kvm、Vmware vSphere),微服务(istio),PaaS(openshift),IaaS(openstack)等2.系统/运维方向:linux系统下的常用组件(nginx,tomcat,elasticsearch,zookeeper,kafka等),DevOps(Jenkins+gitlab+sonarqube+nexus+k8s),CI/CD,监控(zabbix、prometheus、falcon)等 七、课程大纲
Prometheus可以监控nginx的流量。通过使用Prometheus的插件或者导出器可以收集nginx监控数据。其中,可以使用nginx-vts-exporter来收集Nginx监控数据,并将其通过HTTP暴露给Prometheus进行消费。nginx-vts-exporter是一个简单的服务器,它会抓取Nginx的vts(虚拟主机统计)统计数据,并通过HTTP接口提供给Prometheus。默认情况下,nginx-vts-exporter监听的端口号是9913。 Prometheus具有多维数据模型,可以很好地对nginx的指标进行收集和存储,例如流量。你可以自定义Prometheus的数据标签,以便针对不同的机房和项目对nginx进行监控。这样,你可以根据需要对各个维度的流量进行可视化展示,也可以使用Grafana等工具来生成图像展示和监控报告。<span class="em">1</span><span class="em">2</span><span class="em">3</span> #### 引用[.reference_title] - *1* *2* [用PrometheusNginx监控](https://blog.csdn.net/weixin_41709748/article/details/88063548)[target="_blank" data-report-click={"spm":"1018.2226.3001.9630","extra":{"utm_source":"vip_chatgpt_common_search_pc_result","utm_medium":"distribute.pc_search_result.none-task-cask-2~all~insert_cask~default-1-null.142^v93^chatsearchT3_1"}}] [.reference_item style="max-width: 50%"] - *3* [使用prometheus监控nginx](https://blog.csdn.net/Mob_com/article/details/89312413)[target="_blank" data-report-click={"spm":"1018.2226.3001.9630","extra":{"utm_source":"vip_chatgpt_common_search_pc_result","utm_medium":"distribute.pc_search_result.none-task-cask-2~all~insert_cask~default-1-null.142^v93^chatsearchT3_1"}}] [.reference_item style="max-width: 50%"] [ .reference_list ]
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值