《Centos7——elk+lnmp+zabbix+grafana来分析日志》

最新推荐文章于 2024-05-12 08:21:42 发布

逻辑思维菜鸟

最新推荐文章于 2024-05-12 08:21:42 发布

阅读量987

点赞数 2

分类专栏：架构项目文章标签： centos mysql nginx elasticsearch php

本文链接：https://blog.csdn.net/weixin_45842014/article/details/110171113

版权

架构项目专栏收录该内容

5 篇文章 1 订阅

订阅专栏

elk+lnmp+zabbix+grafana来分析日志

环境准备
3台Centos7
192.168.234.130 jdk+elasticsearch+kibana
192.168.234.131 jdk+logstash+zabbix-agent+lnmp
192.168.234.132 zabbix-server+grafana

在这里插入图片描述

一、 Elasticsearch部署(192.168.234.130)

1. 上传安装包

elasticsearch包

链接: https://pan.baidu.com/s/1x0VD4N_p4mC7kTKdHUGTrg 提取码: 84jw 复制这段内容后打开百度网盘手机App，操作更方便哦

jdk包

链接: https://pan.baidu.com/s/1OK2v74y6Bg7-jE47dCF8nA 提取码: ndrn 复制这段内容后打开百度网盘手机App，操作更方便哦

kibana包

链接: https://pan.baidu.com/s/1lsmMI9R3j7YZ16AcbKlEIw 提取码: kmht 复制这段内容后打开百度网盘手机App，操作更方便哦

2. 安装elasticsearch+kibana

[root@localhost ~]# systemctl stop firewalld;setenforce 0

[root@localhost ~]# rpm -ivh elasticsearch-7.3.2-x86_64.rpm kibana-7.3.2-x86_64.rpm

3. 修改es和kibana配置文件

[root@localhost ~]# vim /etc/elasticsearch/elasticsearch.yml
[root@localhost ~]# cat /etc/elasticsearch/elasticsearch.yml |egrep -v '^#|^$'
cluster.name: my-application #集群的名字
node.name: node-1 #节点的名字，在集群中是唯一的
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 0.0.0.0 #监听的IP地址
http.port: 9200 #监听的端口
#发现的集群的其他elasticsearch节点
discovery.seed_hosts: ["192.168.234.130""] 
cluster.initial_master_nodes: ["192.168.234.130"]
#集群中有几个Elasticsearch 运行时，集群恢复正常
gateway.recover_after_nodes: 2

在这里插入图片描述

[root@localhost ~]# vim /etc/kibana/kibana.yml
[root@localhost ~]# cat /etc/kibana/kibana.yml |egrep -v '^$|^#'
server.port: 5601 #监听端口
server.host: "0.0.0.0" #监听IP
elasticsearch.hosts: ["http://192.168.234.130:9200"] #elasticsearch服务器地址
i18n.locale: "zh-CN" #设置页面语言为中文

在这里插入图片描述

4. 启动服务

启动Elasticsearch 并设置为开机自启

[root@localhost ~]# systemctl start elasticsearch kibana

[root@localhost ~]# systemctl enable elasticsearch kibana
Created symlink from /etc/systemd/system/multi-user.target.wants/elasticsearch.service to /usr/lib/systemd/system/elasticsearch.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/kibana.service to /etc/systemd/system/kibana.service.

[root@localhost ~]# netstat -lptnu|grep java
tcp6       0      0 :::9200                 :::*                    LISTEN      32844/java
tcp6       0      0 :::9300                 :::*                    LISTEN      32844/java

[root@localhost ~]# curl 192.168.234.130:9200
{
  "name" : "node-1",
  "cluster_name" : "my-application",
  "cluster_uuid" : "Ij96w1J5QyqIU1XpLCrmYA",
  "version" : {
    "number" : "7.3.2",
    "build_flavor" : "default",
    "build_type" : "rpm",
    "build_hash" : "1c1faf1",
    "build_date" : "2019-09-06T14:40:30.409026Z",
    "build_snapshot" : false,
    "lucene_version" : "8.1.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

4. 访问页面进行可视化展示

在这里插入图片描述

二、 lnmp+logstash+zabbix-agent部署(192.168.234.131)

lnmp操作

1. 安装nginx依赖

[root@localhost ~]# systemctl stop firewalld;setenforce 0

[root@localhost ~]# yum -y install php php-mysql php-gd php-fpm mariadb mariadb-server gcc gcc-c++ pcre-devel zlib-devel zabbix-agent

2. 安装nginx

[root@localhost ~]# tar xzf nginx-1.12.2.tar.gz

[root@localhost ~]# cd nginx-1.12.2

[root@localhost nginx-1.12.2]# ./configure && make && make install

3. 修改nginx配置文件

[root@localhost ~]# vim /usr/local/nginx/conf/nginx.conf

在这里插入图片描述

4. 启动服务

[root@localhost ~]# /usr/local/nginx/sbin/nginx

[root@localhost ~]# systemctl start php-fpm mariadb

[root@localhost ~]# systemctl enable php-fpm mariadb
Created symlink from /etc/systemd/system/multi-user.target.wants/php-fpm.service to /usr/lib/systemd/system/php-fpm.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/mariadb.service to /usr/lib/systemd/system/mariadb.service.

[root@localhost ~]# mysql
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 3
Server version: 5.5.68-MariaDB MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> create database ds character set utf8 collate utf8_bin;
Query OK, 1 row affected (0.00 sec)

MariaDB [(none)]> exit
Bye

5. 上线电商

上传电商包到/usr/localnginx/html

[root@localhost ~]# cd /usr/local/nginx/html/

[root@localhost html]# yum -y install unzip

[root@localhost html]# unzip tinyshopV2.5_data.zip

[root@localhost html]# chmod -R 777 .*

6. 访问网页(192.168.234.131)

在这里插入图片描述

logstash操作

logstash包

链接: https://pan.baidu.com/s/1C-UALUt_9PAYu9fKPmpySA 提取码: fg2s 复制这段内容后打开百度网盘手机App，操作更方便哦

jdk包

链接: https://pan.baidu.com/s/1OK2v74y6Bg7-jE47dCF8nA 提取码: ndrn 复制这段内容后打开百度网盘手机App，操作更方便哦

1. 安装logstash

[root@localhost ~]# rpm -ivh jdk-8u131-linux-x64_.rpm 

[root@localhost ~]# rpm -ivh logstash-7.3.2.rpm

2. 采集日志数据

nginx日志

[root@localhost ~]# cd /etc/logstash/conf.d/
[root@localhost ~]# vim nginx.conf
input { #收集的日志信息
        file { #收集的方式是文件形式
                path => "/usr/local/nginx/logs/access.log" #日志的绝对路径
                type => "nginx-log" #日志的类型，只是标签的含义，可以自定义名字
                start_position => "beginning" #从日志文件的最开始收集日志
        }
}
output { #日志的输出
        elasticsearch { #日志输出给elasticsearch
                hosts => "192.168.234.130:9200" #elasticsearch的IP和端口
                index => "nginx-log-%{+YYYY.MM.dd}" #日志的索引和日期后缀
        }
}

系统日志

[root@localhost ~]# cd /etc/logstash/conf.d/
[root@localhost ~]# vim nginx.conf
input {
        file {
                path => "/var/log/messages"
                type => "system-log"
                start_position => "beginning"
        }
}
output {
        elasticsearch {
                hosts => "192.168.234.130:9200"
                index => "nginx-log-%{+YYYY.MM.dd}"
        }
}

[root@localhost conf.d]# chmod +r /var/log/messages

[root@localhost ~]# cd /etc/logstash/
[root@localhost logstash]# vim pipelines.yml
- pipeline.id: nginx
  path.config: "/etc/logstash/conf.d/nginx.conf"
- pipeline.id: system
  path.config: "/etc/logstash/conf.d/system.conf"

在这里插入图片描述

3. 启动服务

[root@localhost ~]# systemctl start logstash

[root@localhost ~]# systemctl enable logstash
Created symlink from /etc/systemd/system/multi-user.target.wants/logstash.service to /etc/systemd/system/logstash.service.

[root@localhost ~]# netstat -lptnu|grep 9600
tcp6       0      0 127.0.0.1:9600          :::*                    LISTEN      77808/java

[root@localhost ~]# curl -X GET http://192.168.234.130:9200/_cat/indices?v #查看索引
health status index                uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   .kibana_task_manager q-WKgG_wQMCxtA7fDZQKjg   1   0          2            0     45.5kb         45.5kb
yellow open   nginx-log-2020.11.26 LTDk2PFbTvi98s989Dnrkw   1   1        964            0    364.9kb        364.9kb
yellow open   system-2020.11.26    c8IsWtvORpCCLlUa3I8KPQ   1   1         35            0     23.7kb         23.7kb
green  open   .kibana_1            q3wdSka5RsirL6-Bhm0UKA   1   0          2            0     11.3kb         11.3kb

在这里插入图片描述

zabbix-agent操作

zabbix的yum源

链接: https://pan.baidu.com/s/1p3hA10as_YpuJrxhdoCYlg 提取码: ks4m 复制这段内容后打开百度网盘手机App，操作更方便哦

上传zabbix_Aliyun.repo 到 /etc/yum.repos.d/

[root@localhost ~]# yum clean all
[root@localhost ~]# yum repolist

1. 安装zabbix-agent

[root@localhost ~]# yum -y install zabbix-agent

2. 修改配置文件

[root@localhost ~]# vim /etc/zabbix/zabbix_agentd.conf

[root@localhost ~]# cat /etc/zabbix/zabbix_agentd.conf |grep '^Server'
Server=192.168.234.132
ServerActive=192.168.234.132

在这里插入图片描述

3. 启动服务

[root@localhost ~]# systemctl start zabbix-agent

[root@localhost ~]# systemctl enable zabbix-agent
Created symlink from /etc/systemd/system/multi-user.target.wants/zabbix-agent.service to /usr/lib/systemd/system/zabbix-agent.service.

收集nginx的pv、uv

1. 编写脚本

[root@localhost ~]# cd /etc/zabbix/

[root@localhost zabbix]# mkdir scripts

[root@localhost zabbix]# cd scripts/

[root@localhost scripts]# vim nginx_page.sh
#!/bin/bash
logfile="/usr/local/nginx/logs/access.log"

pv=$(cat $logfile | wc -l)
uv=$(awk '{print $1}' $logfile | sort | uniq |wc -l)

case $1 in
pv)
        echo $pv
;;
uv)
        echo $uv
;;
*)
        echo "./nginx_page.sh (pv|uv)"
esac

[root@localhost scripts]# chmod +x nginx_page.sh

[root@localhost scripts]# vim /etc/zabbix/zabbix_agentd.conf
UnsafeUserParameters=1  #开启用户自定义监控项
UserParameter=nginx[*],/etc/zabbix/scripts/page.sh $1  #定义的键值的名字：check_system

在这里插入图片描述

2. 重启zabbix-agent

[root@localhost scripts]# systemctl restart zabbix-agent

3. zabbix监控mysql库或表空间

三种方式(任意一种都可以)
方法一

[root@localhost ~]# vim /etc/my.cnf
[root@localhost ~]# cat /etc/my.cnf|egrep 'client|^user'
[client]
user=root

在这里插入图片描述
方法二

[root@localhost ~]# vim /etc/zabbix/zabbix_agentd.d/userparameter_mysql.conf

在这里插入图片描述
方法三
给zabbix授予权限

[root@localhost ~]# mysql
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 889
Server version: 5.5.68-MariaDB MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> grant select on *.* to zabbix@'localhost';
Query OK, 0 rows affected (0.04 sec)

三、 zabbix-server+grafana部署(192.168.234.132)

zabbix-server操作

1. 准备zabbix的yum源

zabbix的yum源

链接: https://pan.baidu.com/s/1p3hA10as_YpuJrxhdoCYlg 提取码: ks4m 复制这段内容后打开百度网盘手机App，操作更方便哦

上传zabbix_Aliyun.repo 到 /etc/yum.repos.d/

[root@localhost ~]# yum clean all
[root@localhost ~]# yum repolist

2. 安装zabbix-server

[root@localhost ~]# systemctl stop firewalld;setenforce 0

[root@localhost ~]# yum -y install zabbix-server-mysql zabbix-web-mysql zabbix-agent mariadb mariadb-server

3. 启动mariadb、并创建数据库

[root@localhost ~]# systemctl start mariadb
[root@localhost ~]# mysql
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 2
Server version: 5.5.68-MariaDB MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> create database zabbix character set utf8 collate utf8_bin;
Query OK, 1 row affected (0.00 sec)

MariaDB [(none)]> grant all on zabbix.* to zabbix@'localhost' identified by 'zabbix';
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> flush privileges;
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> exit
Bye

4. 导入初始的数据

[root@localhost ~]# zcat /usr/share/doc/zabbix-server-mysql-4.2.8/create.sql.gz |mysql zabbix

5. 修改配置文件

[root@localhost ~]# vim /etc/zabbix/zabbix_server.conf
[root@localhost ~]# cat /etc/zabbix/zabbix_server.conf |egrep  '^DB'
DBHost=localhost
DBName=zabbix
DBUser=zabbix
DBPassword=zabbix

在这里插入图片描述

6. 修改时区

[root@localhost ~]# vim /etc/httpd/conf.d/zabbix.conf
[root@localhost ~]# cat /etc/httpd/conf.d/zabbix.conf|grep Asia/Shanghai
        php_value date.timezone Asia/Shanghai

在这里插入图片描述

7. 中文乱码

[root@localhost ~]# cd /usr/share/zabbix/assets/fonts/

[root@localhost fonts]# ls
graphfont.ttf  simkai.ttf

[root@localhost fonts]# mv simkai.ttf graphfont.ttf
mv：是否覆盖"graphfont.ttf"？ yes

7. 启动服务

[root@localhost ~]#  systemctl start httpd zabbix-server zabbix-agent

[root@localhost ~]#  systemctl enable mariadb httpd zabbix-server zabbix-agent


[root@localhost ~]# netstat -lptnu|grep 10050
tcp        0      0 0.0.0.0:10050           0.0.0.0:*               LISTEN      22053/zabbix_agentd
tcp6       0      0 :::10050                :::*                    LISTEN      22053/zabbix_agentd

访问网页(192.168.234.132/zabbix)

在这里插入图片描述

8. 安装数据采集工具

[root@localhost ~]# yum -y install zabbix-get

在zabbix-server服务器上测试连接zabbix-agent成功

[root@localhost ~]# zabbix_get -s 192.168.234.131 -k agent.ping #这里的命令是在zabbix-server服务器上执行、IP是被监控主机的，返回值为1测试成功
1

[root@localhost ~]# zabbix_get -s 192.168.234.131 -k nginx[pv]
80

[root@localhost ~]# zabbix_get -s 192.168.234.131 -k nginx[uv]
1

[root@localhost ~]# zabbix_get -s 192.168.234.131 -k mysql.status[Uptime]
11217

zabbix自动发现

在这里插入图片描述

添加nginx（pv|uv）监控项

在这里插入图片描述

创建图形

在这里插入图片描述

zabbix监控mysql库或表空间

查看mysql数据库数据大小

[root@localhost ~]# zabbix_get -s 192.168.234.131 -k mysql.size[mysql,,data]
555022

查看mysql数据库user表数据大小

[root@localhost ~]# zabbix_get -s 192.168.234.131 -k mysql.size[mysql,user,data]
324

1. 创建监控项

在这里插入图片描述

2. 创建图形

在这里插入图片描述

查看mysql的qps

在这里插入图片描述

grafana操作

grafana包

链接: https://pan.baidu.com/s/1v6adr7RfDXnwRQrxEwPsFg 提取码: hvv6 复制这段内容后打开百度网盘手机App，操作更方便哦

zabbix插件包

链接: https://pan.baidu.com/s/1XfJWlgE-FtKxyJz1EaiI3g 提取码: gc3h 复制这段内容后打开百度网盘手机App，操作更方便哦

1. 安装grafana

[root@localhost ~]# yum -y install grafana-7.3.3-1.x86_64.rpm

2. 启动服务

[root@localhost ~]# systemctl start grafana-server

[root@localhost ~]# netstat -lptnu|grep 3000
tcp6       0      0 :::3000                 :::*                    LISTEN      81025/grafana-serve

3. 上传zabbix插件

[root@localhost ~]# mv alexanderzobnin-zabbix-app-4.0.2.zip /var/lib/grafana/plugins/

[root@localhost ~]# cd /var/lib/grafana/plugins/

[root@localhost plugins]# yum -y install unzip

[root@localhost plugins]# unzip alexanderzobnin-zabbix-app-4.0.2.zip

[root@localhost plugins]# systemctl restart grafana-server

4. 访问网页

在这里插入图片描述

收集nginx日志

在这里插入图片描述

展示网站PVUV

在这里插入图片描述

展示mysql库user表大小

在这里插入图片描述

逻辑思维菜鸟

关注

2
点赞
踩
4

收藏

觉得还不错? 一键收藏
1
评论
《Centos7——elk+lnmp+zabbix+grafana来分析日志》

目录elk+lnmp+zabbix+grafana来分析日志一、 Elasticsearch部署(192.168.234.130)1. 上传安装包2. 安装elasticsearch+kibana3. 修改es和kibana配置文件4. 启动服务4. 访问页面进行可视化展示二、 lnmp+logstash+zabbix-agent部署(192.168.234.131)lnmp操作1. 安装nginx依赖2. 安装nginx3. 修改nginx配置文件4. 启动服务5. 上线电商6. 访问网页(192.168
复制链接

扫一扫