ELK6.2.4搭建并监控tomcat日志

ElasticSearch安装

1.java 运行环境
yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel -y
2.获取es源码
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.2.4-linux-x86_64.tar.gz
3.修改配置文件
vi config/elasticsearch.yml
path.data: /usr/local/elasticsearch/data
path.logs:/usr/local/elasticsearch/logs
network.host: 0.0.0.0
http.cors.enabled: true
http.cors.allow-origin: "*"
4.修改打开最大文件数
vi /etc/security/limits.conf
* soft nofile 65536
* hard nofile 131072
* soft nproc 2048
* hard nproc 4096
5.修改打开最大线程数
vi /etc/security/limits.d/*.conf
* soft nproc 4096
6. 修改系统虚拟内存大小
vi /etc/sysctl.conf
vm.max_map_count=655360
fs.file-max=655360
7.重启生效配置
sysctl -p
reboot
8.新建es用户
adduser devops
passwd devops
chown -R devops /usr/local/elk/elasticsearch-6.2.4
9.后台启动
./bin/elasticsearch  -d

10.查看情况
lsof -i:9200

ElasticSearch插件安装

1.nodejs安装
wget https://nodejs.org/dist/v12.18.1/node-v12.18.1-linux-x64.tar.xz
tar xf node-v12.18.1-linux-x64.tar.xz //解压 
cd node-v12.18.1-linux-x64
然后 vim /etc/profile，
在最下面添加 export PATH=$PATH: 后面跟上 node 下 bin 目录的路径
export PATH=$PATH:/root/node-v12.18.1-linux-x64/bin
立即生效
source /etc/profile
[root@localhost ~]# node -v
v12.18.1
2.淘宝源安装
npm install -g cnpm --registry=https://registry.npm.taobao.org
cnpm install
3.插件源码下载并启动
wget https://github.com/mobz/es-head/archive/master.zip
在es-head 目录下
cnpm install
cnpm run start
ip:9100
4.后台运行
nohup cnpm run start &> run.log &

Logstash安装

1.获取源码并解压
wget https://artifacts.elastic.co/downloads/logstash/logstash-6.2.4-linux-x86_64.tar.gz
tar -xvf ogstash-6.2.4-linux-x86_64.tar.gz
2.编辑数据文件采集tomcat日志
vi config/test.conf
input {
	file {
		path => ["/usr/local/elk/apache-tomcat-8.5.9/logs/localhost_access.*"]
		type => "tomcat_log"
		start_position => "beginning"
		stat_interval => "2"
		codec => json
	}
	file {
		path => ["/usr/local/elk/apache-tomcat-8.5.9/logs/catalina.out"]
                type => "tomcat_catalina"
                start_position => "beginning"
                stat_interval => "2"
                codec => plain{

              charset => "UTF-8"

        }

	}
}
filter {
  if [path] =~ "access" {
    mutate { replace => { "type" => "tomcat catalina.out" } }
    grok {
      match => { "message" => "%{COMBINEDAPACHELOG}" }
    }
  }
  date {
    match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
  }
}

output {
if [type] == "tomcat_log" {

    elasticsearch {

      hosts => ["192.168.203.9:9200"]

      index => "tomcat-pc-%{+YYYY.MM.dd}"

      }

 }

  if [type]  == "tomcat_catalina" {

      elasticsearch {

        hosts => ["192.168.203.9:9200"]

        index => "catalina-%{+YYYY.MM.dd}"

      }

	 }
}

3.后台启动
启动 nohup ./bin/logstash -f config/test.conf &> run.log &

kibana安装

1.获取源码并解压
wget https://artifacts.elastic.co/downloads/kibana/kibana-6.2.4-linux-x86_64.tar.gz
tar -xvf  kibana-6.2.4-linux-x86_64.tar.gz
2.配置文件修改
vi kibana.yml
server.host: "192.168.203.9"
elasticsearch.url: "http://192.168.203.9:9200"
3.后台启动
启动 nohup ./bin/kibana -H 0.0.0.0 &> run.log &
4.访问地址
ip:5601

5.kibana web 配置
打开web 页面
配置index
打开Dicover 页面