手动配置ETK过程

2019独角兽企业重金招聘Python工程师标准>>>

1：安装elasticsearch

到官网上下载最新的elasticsearch-5.4.0:
cd /home/soft
mkdir elasticsearch
cd elasticsearch 
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.4.0.tar.gz
tar xzvf elasticsearch-5.4.0.tar.gz
cd elasticsearch-5.4.0
./bin/elasticsearch
此时如果服务器的内存大于2g的话，则不会报内存heap size [268435456] not equal to maximum heap size [2147483648]的问题，如果报了，则需要修改：
vi ./config/jvm.options 
修改内存：
#-Xms2g
#-Xmx2g
-Xms256m
-Xmx256m

再次启动，如果报以下错误：
max file descriptors [65535] for elasticsearch process is too low, increase to at least [65536]
则编辑一下文件：
vi  /etc/security/limits.conf
在文件末尾，新增如下内容：
* soft nofile 65536
* hard nofile 131072
* soft nproc 2048
* hard nproc 4096

elasticsearch-5.4.0在配置外网可以访问的时候，已经将配置：
network.host: xxx.xxx.xxx.xxx
http.port: 9200
启动还是报错，报错信息如下：
 [2017-05-12T09:49:32,544][INFO ][o.e.b.BootstrapChecks    ] [node-1] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
ERROR: bootstrap checks failed
system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk

需要修改配置文件：
vi /home/soft/elastic/elasticsearch-5.4.0/config/elasticsearch.yml添加以下代码：
discovery.zen.ping.unicast.hosts: ["xxx.xxx.xxx.xxx"]
discovery.zen.minimum_master_nodes: 3
bootstrap.system_call_filter: false
启动服务：
cd /home/soft/elasticsearch
nohup ./elasticsearch-5.4.0/bin/elasticsearch &

2：安装logstash
到官网上下载最新的logstash5.4.0,
cd /home/soft
mkdir logstash
cd logstash
wget https://artifacts.elastic.co/downloads/logstash/logstash-5.4.0.tar.gz
tar xzvf logstash-5.4.0.tar.gz
cd logstash-5.4.0
mkdir customer_conf
vi logstash_nginx.conf
输入以下内容，配置简单的读取nginx日志文件：
input {
        file {
                type => "nginx access log"
                path => ["/home/wwwlogs/access.log"]  #nginx日志路径
        }
}
filter {
  #Only matched data are send to output.
}
output {
    elasticsearch {
        hosts => ["127.0.0.1:9200"]
        index => "logstash-nginx-access-%{+YYYY.MM.dd}"
    }
    stdout {codec => rubydebug}
}

如果服务器内存不够或者报内存溢出的话，需要调低内存配置，编辑一下配置文件：
vi /home/soft/logstash/logstash-5.4.0/config/jvm.options 
修改以下内容：
#-Xms256m
#-Xmx1g
-Xms100m
-Xmx200m

然后启动服务：
cd /home/soft/logstash
nohup ./logstash-5.4.0/bin/logstash -f logstash-5.4.0input_data_conf/logstash_agent.conf &

3：安装kibana
cd /home/soft
mkdir kibana
cd kibana
wget https://artifacts.elastic.co/downloads/kibana/kibana-5.4.0-linux-x86_64.tar.gz
tar xzvf kibana-5.4.0-linux-x86_64.tar.gz

编辑kibana配置文件
vi  kibana-5.4.0-linux-x86_64/config/kibana.yml 
配置以下内容：
server.host = xxx.xxx.xxx.xxx(IP地址)
elasticsearch.url: "http://localhost:9200"

启动服务：
cd /home/soft/kibana
nohup ./kibana-5.4.0-linux-x86_64/bin/kibana &

通过浏览器访问:xxx.xxx.xxx:5601即可

转载于:https://my.oschina.net/u/435872/blog/898165