ES与 Kibana安装部署

ES与 Kibana安装部署

一、ElasticSearch安装部署

1.1 安装包下载

1）ElasticSearch官网： https://www.elastic.co/cn/downloads/elasticsearch

1.2 ElasticSearch安装

1.2.1 解压安装ElasticSearch

1）解压elasticsearch-6.6.0.tar.gz到/opt/module目录下

[hadoop@hadoop102 software]$ tar -zxvf elasticsearch-6.6.0.tar.gz -C /opt/module/
[hadoop@hadoop102 module]$

2）在/opt/module/elasticsearch路径下创建data文件夹

[hadoop@hadoop102 elasticsearch]$ mkdir data

3）修改配置文件/opt/module/elasticsearch/config/elasticsearch.yml

[hadoop@hadoop102 config]$ pwd
/opt/module/elasticsearch/config
[hadoop@hadoop102 config]$ vi elasticsearch.yml

#-----------------------Cluster-----------------------
cluster.name: my-application
#-----------------------Node-----------------------
node.name: node-102
#-----------------------Paths-----------------------
path.data: /opt/module/elasticsearch/data
path.logs: /opt/module/elasticsearch/logs
#-----------------------Memory-----------------------
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
#-----------------------Network-----------------------
network.host: 192.168.6.102 
#-----------------------Discovery-----------------------
discovery.zen.ping.unicast.hosts: ["192.168.1.102"]

（1）cluster.name

如果要配置集群需要两个节点上的elasticsearch配置的cluster.name相同，都启动可以自动组成集群，这里如果不改cluster.name则默认是cluster.name=my-application，

（2）nodename随意取但是集群内的各节点不能相同

（3）修改后的每行前面不能有空格，修改后的“：”后面必须有一个空格

4）修改配置文件/opt/module/elasticsearch/config/jvm.options

修改
-Xms1g
-Xmx1g
为
-Xms256m
-Xmx256m

分发至hadoop103以及hadoop104，分发之后修改：

[hadoop@hadoop102 module]$ xsync elasticsearch/

node.name: node-103
network.host: 192.168.1.103

node.name: node-104
network.host: 192.168.1.104

1.2.2 配置linux系统环境

（参考：http://blog.csdn.net/satiling/article/details/59697916）

1）切换到root用户，编辑limits.conf 添加类似如下内容

[root@hadoop102 elasticsearch]# vi /etc/security/limits.conf
添加如下内容:
* soft nofile 65536
* hard nofile 131072
* soft nproc 2048
* hard nproc 4096
注意：“*” 不要省略掉

2）切换到root用户，进入limits.d目录下修改配置文件。（CentOS7.x不用改）

[root@hadoop102 elasticsearch]# vi /etc/security/limits.d/90-nproc.conf
修改如下内容：
* soft nproc 1024
#修改为
* soft nproc 4096

3）切换到root用户修改配置sysctl.conf

[root@hadoop102 elasticsearch]# vi /etc/sysctl.conf
添加下面配置：
vm.max_map_count=655360
并执行命令：
[root@hadoop102 elasticsearch]# sysctl -p
以上修改的Linux配置需要分发至其他节点
[root@hadoop102 elasticsearch]# xsync /etc/security/limits.conf

[root@hadoop102 elasticsearch]# xsync /etc/security/limits.d/90-nproc.conf  #(Centos7.X不需要)

[root@hadoop102 elasticsearch]# xsync /etc/sysctl.conf

然后，重新启动Linux，必须重启！！！
4）启动Elasticsearch

[hadoop@hadoop102 elasticsearch]$ bin/elasticsearch

5）测试elasticsearch

[hadoop@hadoop102 elasticsearch]$ curl http://hadoop102:9200
{
  "name" : "node-102",
  "cluster_name" : "my-application",
  "cluster_uuid" : "w9UFspEFR4-F5gLrTYMHIw",
  "version" : {
    "number" : "6.6.0",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "a9861f4",
    "build_date" : "2019-01-24T11:27:09.439740Z",
    "build_snapshot" : false,
    "lucene_version" : "7.6.0",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

或者在浏览器中访问http://hadoop102:9200

8）停止集群

kill -9 进程号

9）群起脚本

[hadoop@hadoop102 bin]$ vi es.sh
#!/bin/bash
es_home=/opt/module/elasticsearch
case $1  in
 "start") {
  for i in hadoop102 hadoop103 hadoop104
  do
    echo "==============$i=============="
ssh $i  "source /etc/profile;${es_home}/bin/elasticsearch >/dev/null 2>&1 &"
sleep 4s;
  done

};;
"stop") {
  for i in hadoop102 hadoop103 hadoop104
  do
    echo "==============$i=============="
    ssh $i "ps -ef|grep $es_home |grep -v grep|awk '{print \$2}'|xargs kill" >/dev/null 2>&1
  done

};;
esac

1.3 ElasticSearch操作工具

1.3.1 浏览器

查看集群节点：浏览器中输入hadoop102:9200/_cat/nodes

二、Kibana

Kibana的安装
将kibana压缩包上传到虚拟机指定目录

[hadoop@hadoop102 software]$ tar -zxvf kibana-6.6.0-linux-x86_64.tar.gz -C /opt/module/
[hadoop@hadoop102 module]$ mv kibana-6.6.0-linux-x86_64/ kibana/
修改相关配置，连接Elasticsearch
[hadoop@hadoop102 kibana]$ vi config/kibana.yml

# Kibana is served by a back end server. This setting specifies the port to use.
server.port: 5601
# Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.
# The default is 'localhost', which usually means remote machines will not be able to connect.
# To allow connections from remote users, set this parameter to a non-loopback address.
server.host: "192.168.6.102"
... ...
... ...
# The URL of the Elasticsearch instance to use for all your queries.
elasticsearch.hosts: "http://192.168.6.102:9200"

启动Kibana

[hadoop@hadoop102 kibana]$ bin/kibana

浏览器访问hadoop102:5601

修改之前的ES启动脚本为：

#!/bin/bash
es_home=/opt/module/elasticsearch
kibana_home=/opt/module/kibana
case $1  in
 "start") {
  for i in hadoop102 hadoop103 hadoop104
  do
    echo "==============$i=============="
ssh $i  "source /etc/profile;${es_home}/bin/elasticsearch >/dev/null 2>&1 &"
sleep 4s;
  done
  sleep 2s;
  nohup ${kibana_home}/bin/kibana > kibana.log 2>&1 &
};;
"stop") {
  ps -ef | grep ${kibana_home} | grep -v grep | awk '{print $2}'| xargs kill
  for i in hadoop102 hadoop103 hadoop104
  do
    echo "==============$i=============="
    ssh $i "ps -ef|grep $es_home |grep -v grep|awk '{print \$2}'|xargs kill" >/dev/null 2>&1
  done
};;
esac

Kibana的Dev Tools
在操作窗口输入 GET _cat/nodes