ELK安装文档
参考文档:https://www.cnblogs.com/yuhuLin/p/7018858.html
机器配置
* ELK-SERVER 192.168.232.10 4G内存 4核CPU 50G存储
* ELK-CLIENT 192.168.232.11 2G内存 2核CPU 20G存储
* VMware workstation 10
一、给设备配静态IP、网关及DNS(server,client)
1. # vi /etc/sysconfig/network-script/ifcf-enoxxxx
HWADDR=00:0C:29:B9:8E:xx
TYPE=Ethernet
BOOTPROTO=static
NAME=eno16xxxx
UUID=897e213a-2271-4855-b57b-a0f50fxxxxx
ONBOOT=yes
IPADDR=192.168.232.10
NETMASK=255.255.255.0
GATEWAY=192.168.232.2
DNS1=192.168.232.2
2. 修改网关
# vi /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=ELK-SERVER
GATEWAY=192.168.232.2
3. 修改DNS
# vi /etc/resolv.conf
nameserver 192.168.232.2
二、修改hostname,修改网络hosts(server,client),这里以server为例
1. 对于centos7 修改hostname需要改动两个文件,一个是 /etc/hostaname,另一个是/etc/sysconfig/network
1.1 # vi /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=ELK-SERVER
GATEWAY=192.168.232.2
1.2
# vi /etc/hostnam
ELK-SERVER
2. 修改网络hosts,使得server和client通过hostname互ping (server,client)
2.1 # vi /etc/hosts
192.168.232.10 ELK-SERVER
192.168.232.11 ELK-CLIENT
三、 安装JDK(server,client)
1. 将提前下载好的jdk-8u151-linux-x64.rpm 上传到/usr/local
2. rpm方式安装,JDK默认安装在/usr/java中
# rpm -ivh jdk-8u151-linux-x64.rpm
验证:# java -version
java version “1.8.0_151”
Java(TM) SE Runtime Environment (build 1.8.0_151-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.151-b12, mixed mode)
3. 配置JAVA环境变量 # vi + /etc/profile
JAVA_HOME=/usr/java/jdk1.8.0_151
JRE_HOME=/usr/java/jdk1.8.0_151/jre
PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin
CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JRE_HOME/lib
export JAVA_HOME JRE_HOME PATH CLASSPATH
立即生效环境变量:
# source /etc/profile
验证:
# echo $JAVA_HOME
/usr/java/jdk1.8.0_151
以上部分为设备的基础配置,接下来我们开始安装ELK各个部件
四、rpm安装Elasticsearch(server)
1. 将rpm包上传到server的/usr/local/elasticsearch
2. 下载MD5校验文件到/usr/local/elasticsearch
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.1.1.rpm.sha512
如果执行这一步,发现没有wget命令,# yum install -y wget
安装一下就好
3. rpm 安装
3.1 # shasum -a 512 -c elasticsearch-6.1.1.rpm.sha512
3.2 # rpm -ivh elasticsearch-6.1.1.rpm
3.3 查看rpm把elasticsearch安装到了哪里
# whereis elasticsearch
elasticsearch: /etc/elasticsearch /usr/local/elasticsearch /usr/share/elasticsearch
注:一般/etc/elasticsearch的是配置文件, /usr/local/elasticsearch是刚才上传的rpm包,/usr/share/elasticsearch才是安装的目录
3.4 设置elasticsearch开机启动
# /bin/systemctl daemon-reload
# /bin/systemctl enable elasticsearch.service
ln -s '/usr/lib/systemd/system/elasticsearch.service' '/etc/systemd/system/multi-user.target.wants/elasticsearch.service'
3.5 配置elasticsearch
# vi /etc/elasticsearch/elasticsearch.yml
找到配置文件中的cluster.name,打开该配置并设置集群名称
cluster.name: demon
找到配置文件中的node.name,打开该配置并设置节点名称
node.name: elk-1
修改logs日志的路径
path.logs: /var/log/elasticsearch/
配置内存使用用交换分区
bootstrap.memory_lock: true
监听的网络地址
network.host: 0.0.0.0
开启监听的端口
http.port: 9200
增加新的参数,这样head插件可以访问es (5.x版本,如果没有可以自己手动加)
http.cors.enabled: true
http.cors.allow-origin: "*"
注意:每一个配置项,冒号后边一定要留个空格!
3.6 创建elk用户和elk用户组(因为root不能启动elasticsearch服务)
# useradd -g elk elk
# passwd elk
3.7 把/usr/share/elasticsearch所属权限改成elk用户
# chown elk elasticsearch/
# chgrp elk elasticsearch/
3.8 切换到elk用户,启动elasticsearch服务
# cd /usr/share/elasticsearch/bin
# sh elasticsearch
./elasticsearch-env: line 70: /etc/sysconfig/elasticsearch: Permission denied
报没有权限,用root用户把/etc/sysconfig/elasticsearch权限赋予elk用户,再起尝试启动
# chown -R elk elasticsearch
# chgrp -R elk elasticsearch
接着报Exception in thread "main" org.elasticsearch.bootstrap.BootstrapException: java.nio.file.AccessDeniedException: /etc/elasticsearch
再把/etc/elasticsearch赋权给elk,再次尝试启动
# chown -R elk /etc/elasticsearch/
# chgrp -R elk /etc/elasticsearch/
接着报:.AccessDeniedException: /var/log/elasticsearch
同理赋予权限,再次尝试启动
接着报:AccessDeniedException: /var/lib/elasticsearch
同理赋予权限
接着报:max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
修改几个参数:
# vi /etc/security/limits.conf
````
在末尾追加以下内容(elk为启动用户,当然也可以指定为*)
elk soft nofile 65536
elk hard nofile 65536
elk soft nproc 4096
elk hard nproc 4096
elk soft memlock unlimited
elk hard memlock unlimited
```
继续再修改一个参数
# vim /etc/security/limits.d/90-nproc.conf
```
将里面的1024改为2048(ES最少要求为2048)
* soft nproc 2048
```
重启服务器后重新启动elasticsearch服务,无报错。
检验:curl http://192.168.232.10:9200
{
"name" : "elk-1",
"cluster_name" : "demon",
"cluster_uuid" : "EmITSZ46Rn6-3sxLsLgwXQ",
"version" : {
"number" : "6.1.1",
"build_hash" : "bd92e7f",
"build_date" : "2017-12-17T20:23:25.338Z",
"build_snapshot" : false,
"lucene_version" : "7.1.0",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}
3.9 关闭selinux,防火墙放通,主机的浏览器能访问虚机的9200端口
3.9.1 关闭selinux
# vi /etc/selinux/config
SELINUX=disable
3.9.2 防火墙放通9200端口
# firewall-cmd --zone=public --add-port=9200/tcp --permanent
# firewall-cmd --reload
# firewall-cmd --list-ports 查看开放了哪些端口
4. elasticsearch安装完毕,接下来要安装一个小插件elasticsearch-head(切换到elk用户)
4.1 使用git安装
# git clone git://github.com/mobz/elasticsearch-head.git
4.2 安装nodejs(因为要用到npm命令,npm是集成在nodejs中)
4.2.1 官网查看nodejs的最新版本 https://nodejs.org/dist/
# node-v9.4.0-linux-x64.tar.gz
4.2.2 解压
# tar -zxvf node-v9.4.0-linux-x64.tar.gz
4.2.3 配置环境变量,保存退出,使生效
# vi + /etc/profile
#set for nodejs
export NODE_HOME=/usr/share/nodejs/node
export PATH=$NODE_HOME/bin:$PATH
# source /etc/profile
验证:
# node -v
v9.4.0
# npm -v
5.6.0
4.3 安装head
# cd /usr/share/elasticsearch-head/elasticsearch-head
# npm install
此处有漫长的等待,会有报错,但不貌似影响启动
# npm runstart
4.4 防火墙放通9100端口
# firewall-cmd --zone=public --add-port=9100/tcp --permanent
# firewall-cmd --reload
# firewall-cmd --list-ports
4.5 主机浏览器访问虚机,http://192.168.232.10:9100
4.6 连接到本地的elasticsearch
五、安装logstash(server,client)
1. 下载yum源的密钥认证:
# rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
2. 添加yum源
# vi /etc/yum.repo.d/logstash.repo
[logstash-6.x]
name=Elastic repository for 6.x packages
baseurl=https://artifacts.elastic.co/packages/6.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
# yum install logstash -y
安装目录在 /etc/share/logstash
3. 创建软连接,每次执行命令的时候不用写安装路径
# ln -s /usr/share/logstash/bin/logstash /bin/
4. 设置开机启动
# systemctl enable logstash
5. 测试:logstash -e 'input { stdin { } } output { stdout {} }'
六、 安装Kibana(server)
1. 把rpm包上传到/usr/local/ kibana-6.1.1-x86_64.rpm
2. # rpm --install kibana-6.1.1-x86_64.rpm
3. 修改配置文件
# vi /etc/kibana/kibana.yml
server.port: 5601
server.host: "0.0.0.0"
kibana.index: ".kibana"
elasticsearch.url: http://192.168.232.10:5601
4. 防火墙放开5601端口
# firewall-cmd --zone=public --add-port=5601/tcp --permanent
# firewall-cmd --reload
# firewall-cmd --list-ports
5. 主机浏览器访问虚机,http://192.168.232.10:5601
以上
#################################################
第一次搭建ELK系统,若有纰漏之处,请大神们多多指教。