基于HA高可用搭建Hadoop-3.2.1实战搭建之ES-7.7.1+Kibana-7.7.1集群部署

本文链接：https://blog.csdn.net/colby_chenlun/article/details/107107620

本文详细介绍了在CentOS8环境下，如何搭建一个基于HA的Hadoop-3.2.1集群，并结合elasticsearch-7.7.1和kibana-7.7.1进行高可用配置。内容涵盖环境准备、Elasticsearch集群安装、配置修改、启动脚本编写以及Kibana的安装和测试。

摘要由CSDN通过智能技术生成

一、安装环境

操作系统 CentOS8

elasticsearch-7.7.1

kibana-7.7.1

机器：hadoop102、hadoop103、hadoop104、hadoop105、hadoop106

二、安装elasticsearch-7.7.1

1、上传文件elasticsearch-7.7.1-linux-x86_64.tar.gz、kibana-7.7.1-linux-x86_64.tar.gz到/opt/software目录

2、解压到/opt/module

[deploy@hadoop102 module]$ ls -tlr
总用量 24
drwxr-xr-x. 13 deploy deploy  211 2月   3 03:47 spark-2.4.5-bin-hadoop2.7
lrwxrwxrwx.  1 deploy deploy   12 5月  30 23:57 jdk-default -> jdk1.8.0_171
drwxr-xr-x.  8 deploy deploy 4096 5月  30 23:59 jdk1.8.0_171
lrwxrwxrwx.  1 deploy deploy   26 5月  31 00:32 zookeeper-default -> apache-zookeeper-3.6.1-bin
drwxr-xr-x.  8 deploy deploy  159 5月  31 04:31 apache-zookeeper-3.6.1-bin
drwxr-xr-x.  4 deploy deploy   43 5月  31 09:12 job_history
lrwxrwxrwx.  1 deploy deploy   34 6月   2 01:15 spark-default -> spark-3.0.0-preview2-bin-hadoop3.2
drwxr-xr-x.  6 deploy deploy   99 6月  11 12:01 maven
drwxr-xr-x. 11 deploy deploy  195 6月  13 23:42 hadoop-2.10.0
drwxr-xr-x. 10 deploy deploy  184 6月  14 00:17 apache-hive-3.1.2-bin
drwxr-xr-x. 10 deploy deploy  184 6月  14 00:21 apache-hive-2.3.7-bin
drwxr-xr-x. 11 deploy deploy  173 6月  14 01:10 hadoop-2.7.2
drwxr-xr-x.  3 deploy deploy   18 6月  14 02:01 hive
drwxr-xr-x.  5 deploy deploy 4096 6月  14 02:11 tez
lrwxrwxrwx.  1 deploy deploy   12 6月  14 02:26 hadoop-default -> hadoop-3.2.1
lrwxrwxrwx.  1 deploy deploy   21 6月  14 06:06 hive-default -> apache-hive-3.1.2-bin
drwxr-xr-x. 11 deploy deploy  173 6月  14 06:32 hadoop-3.2.1
-rw-rw-r--.  1 deploy deploy  265 6月  14 06:46 TestDFSIO_results.log
drwxr-xr-x. 14 deploy deploy  224 6月  14 09:46 spark-3.0.0-preview2-bin-hadoop3.2
-rw-------.  1 deploy deploy 8534 6月  14 12:31 nohup.out
drwxrwxr-x. 13 deploy deploy  266 6月  14 20:16 kibana-7.7.1-linux-x86_64
lrwxrwxrwx.  1 deploy deploy   19 6月  14 20:19 elasticsearch-default -> elasticsearch-7.7.1
lrwxrwxrwx.  1 deploy deploy   25 6月  14 20:19 kibana-default -> kibana-7.7.1-linux-x86_64
drwxr-xr-x. 10 deploy deploy  167 6月  15 05:05 elasticsearch-7.7.1

3、切换到/opt/module/elasticsearch-7.7.1/config目录，修改elasticsearch.yml文件

修改yml配置的注意事项:

每行必须顶格，不能有空格

“：”后面必须有一个空格

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production