1.安装jdk8
linux下使用wget下载jdk8:
进到目录/usr/local/software
wget --no-cookies --no-check-certificate --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/8u181-b13/96a7b8442fe848ef90c96a2fad6ed6d1/jdk-8u181-linux-x64.tar.gz"
解压文件:
tar zxvf jdk-8u181-linux-x64.tar.gz
mv jdk-8u181-linux-x64.tar.gz jdk8
增加环境变量,编辑对呀的文件
vim /etc/profile
#加入
export JAVA_HOME=/usr/local/software/jdk8
export JAVA_BIN=/usr/local/software/jdk8
export PATH=$PATH:$JAVA_HOME/bin
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export JAVA_HOME JAVA_BIN PATH CLASSPATH
#启动文件
source /etc/profile
#验证java是否安装成功
java -version
2.安装ELK
1.参考网站:https://www.elastic.co/downloads
2.通过wget命令下载 Elasticsearch/Logstash/Kibaber
1.下载elasticsearch
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.4.0.tar.gz
2.下载logstash
wget https://artifacts.elastic.co/downloads/logstash/logstash-6.4.0.tar.gz
3.下载Kibana
wget https://artifacts.elastic.co/downloads/kibana/kibana-6.4.0-linux-x86_64.tar.gz
解压
tar -zxvf elasticsearch-6.4.0.tar.gz
tar -zxvf logstash-6.4.0.tar.gz
tar -zxvf kibana-6.4.0-linux-x86_64.tar.gz
3.配置并启动Elasticsearch
配置es出现相关问题处理:
1、问题一
Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000c5330000, 986513408, 0) failed; error='Cannot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 986513408 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /usr/local/software/temp/elasticsearch-6.2.2/hs_err_pid1912.log
解决:内存不够,购买阿里云的机器可以动态增加内存
2、问题二
[root@iZwz95j86y235aroi85ht0Z bin]# ./elasticsearch
解决:用非root用户
添加用户:useradd -m 用户名 然后设置密码 passwd 用户名
3、问题三
./elasticsearch
Exception in thread "main" java.nio.file.AccessDeniedException: /usr/local/software/temp/elasticsearch-6.4.0/config/jvm.options
解决:权限不够 chmod 777 -R 当前es目录
常见配置问题资料:https://www.jianshu.com/p/c5d6ec0f35e0
使用nohub启动
nohup ./bin/elasticsearch &
在配置文件中增加http外网访问
bootstrap.memory_lock: false
:#增加centos 无法访问
bootstrap.system_call_filter: false
#增加外网访问
http.host: 0.0.0.0
4.配置logstash
input {
beats {
port => 5044
}
}
filter {
grok {
match => { "message" => ["%{IPORHOST:[nginx][access][remote_ip]} - %{DATA:[nginx][access][user_name]} \[%{HTTPDATE:[nginx][access][time]}\] \"%{WORD:[nginx][access][method]} %{DATA:[nginx][access][url]} HTTP/%{NUMBER:[nginx][access][http_version]}\" %{NUMBER:[nginx][access][response_code]} %{NUMBER:[nginx][access][body_sent][bytes]} \"%{DATA:[nginx][access][referrer]}\" \"%{DATA:[nginx][access][agent]}\""] }
remove_field => "message"
}
mutate {
add_field => { "read_timestamp" => "%{@timestamp}" }
}
date {
match => [ "[nginx][access][time]", "dd/MMM/YYYY:H:m:s Z" ]
remove_field => "[nginx][access][time]"
}
useragent {
source => "[nginx][access][agent]"
target => "[nginx][access][user_agent]"
remove_field => "[nginx][access][agent]"
}
geoip {
source => "[nginx][access][remote_ip]"
target => "[geoip]"
add_field => ["[geoip][coordinates]", "%{[geoip][longitude]}"]
add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}" ]
}
mutate {
convert => [ "[geoip][coordinates]", "float" ]
}
}
output {
elasticsearch {
hosts => ["http://localhost:9200"]
index => "logstash-%{[@metadata][beat]}-%{+YYYY.MM.dd}"
}
}
启动logstart
./bin/logstash -f config/file-beats.conf
5.配置kibana
修改kibana.yml
server.host="0.0.0.0"
启动kibana
6.下载filebeat-6.3.2
- 下载地址:https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.3.2-linux-x86_64.tar.gz
- 解压文件tar -zxvf filebeat-6.3.2-linux-x86_64.tar.gz
- 修改配置文件
vi filebeat.yml 文件内容如下: #------------input部分---------- filebeat.prospectors: - type: log paths: - /local/nas/docker/nginx/logs/access.log tags: ["nginx-accesslog"] document_type: nginx-access #注意:filebeat在6版本里面,document_type字段好像不起作用 - type: log paths: - /local/nas/docker/nginx/logs/error.log tags: ["nginx-errorlog"] document_type: nginx-error #-------------output部分,将输出到Elasticsearch注释掉,开启输出到logstash---- output.logstash: hosts: ["172.17.227.15:5044"]
- 启动filebeat
nohup ./filebeat &