ELK分布式日志系统搭建一步。es集群搭建

1.首先准备三台机器，我直接在在虚拟机中创建了三台服务器。

192.168.44.126 master-node
192.168.44.128 data-node1
192.168.44.129 data-node2

2.环境准备:JDK1.8 

es 7.2.0版本，运行时环境需求

使用root 用户创建 目录 mkdir  /software

创建用户es

把目录/software赋权限

chown -R es.es /software

3.参数修改

3.1  vim  /etc/security/limits.conf
es soft nofile 65536
es hard nofile 65536

退出用户重新登录，使配置生效

重新 ulimit -Hn  查看硬限制 会发现数值改成65536

3.2  vim /etc/sysctl.conf

vm.max_map_count=655360

注意：这个值会根据启动报错适当修改，每个服务器的配置不同，根据具体情况修改适当的参数即可。

3.3
vim /etc/security/limits.d/90-nproc.conf 

找到如下内容：

soft nproc 1024

修改为

soft nproc 4096

修改完后，执行sysctl -p

4.上传es压缩包到三台服务器

     修改配置文件/software/elasticsearch-7.2.0/config/elasticsearch.yml

第一台：192.168.44.126（管理节点）
cluster.name: master-node
node.name: master
node.master: true
node.data: false
network.host: 0.0.0.0
discovery.seed_hosts: ["192.168.44.126", "192.168.44.128", "192.168.44.129"]
注：在末尾最后一行添加如下：
bootstrap.system_call_filter: false

不然会报以下错误:java.lang.UnsupportedOperationException: seccomp unavailable: CONFIG_SECCOMP not compiled into kernel, CONFIG_SECCOMP and CONFIG_SECCOMP_FILTER are needed

第二台：192.168.44.128（数据节点1）
cluster.name: master-node
node.name: data-node1
node.master: false
node.data: true
network.host: 0.0.0.0
discovery.seed_hosts: ["192.168.44.126", "192.168.44.128", "192.168.44.129"]
注：在末尾最后一行添加如下：
bootstrap.system_call_filter: false

不然会报以下错误:java.lang.UnsupportedOperationException: seccomp unavailable: CONFIG_SECCOMP not compiled into kernel, CONFIG_SECCOMP and CONFIG_SECCOMP_FILTER are needed

第三台：192.168.44.129（数据节点2）
cluster.name: master-node
node.name: data-node2
node.master: false
node.data: true
network.host: 0.0.0.0
discovery.seed_hosts: ["192.168.44.126", "192.168.44.128", "192.168.44.129"]
注：在末尾最后一行添加如下：
bootstrap.system_call_filter: false

不然会报以下错误:java.lang.UnsupportedOperationException: seccomp unavailable: CONFIG_SECCOMP not compiled into kernel, CONFIG_SECCOMP and CONFIG_SECCOMP_FILTER are needed

5.依次启动管理节点和数据节点

使用nohup启动 脚本es.sh

#/bash/bin
echo "start ..."
nohup /software/elasticsearch-7.2.0/bin/elasticsearch >> /software/elasticsearch-7.2.0/bin/es.log 2>&1 &
echo "sucessfully"
三台一样。

chmod +x es.sh

6.成功查看

使用192.168.44.126：9200，192.168.44.128：9200，192.168.44.129：9200在浏览器执行查询

192.168.44.126：9200

{
  "name" : "master",
  "cluster_name" : "master-node",
  "cluster_uuid" : "_na_",
  "version" : {
    "number" : "7.2.0",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "508c38a",
    "build_date" : "2019-06-20T15:54:18.811730Z",
    "build_snapshot" : false,
    "lucene_version" : "8.0.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

192.168.44.128：9200

{
  "name" : "data-node1",
  "cluster_name" : "master-node",
  "cluster_uuid" : "_na_",
  "version" : {
    "number" : "7.2.0",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "508c38a",
    "build_date" : "2019-06-20T15:54:18.811730Z",
    "build_snapshot" : false,
    "lucene_version" : "8.0.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

192.168.44.129：9200

{
  "name" : "data-node2",
  "cluster_name" : "master-node",
  "cluster_uuid" : "_na_",
  "version" : {
    "number" : "7.2.0",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "508c38a",
    "build_date" : "2019-06-20T15:54:18.811730Z",
    "build_snapshot" : false,
    "lucene_version" : "8.0.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

6.服务检查

[root@master-node ~]# netstat -lntp |grep java
tcp        0      0 :::9200                     :::*                        LISTEN      4167/java           
tcp        0      0 :::9300                     :::*                        LISTEN      4167/java 

[root@data-node1 ~]# netstat -lntp |grep java
tcp        0      0 :::9200                     :::*                        LISTEN      3160/java           
tcp        0      0 :::9300                     :::*                        LISTEN      3160/java 

[root@data-node2 ~]# netstat -lntp |grep java
tcp        0      0 :::9200                     :::*                        LISTEN      2758/java           
tcp        0      0 :::9300                     :::*                        LISTEN      2758/java