elasticsearch集群搭建

最新推荐文章于 2024-06-27 00:24:54 发布
最新推荐文章于 2024-06-27 00:24:54 发布
阅读量1.2k 收藏 1
点赞数
分类专栏： bigdata 文章标签： elasticsearch 搜索引擎 big data
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/rustwei/article/details/121905305
版权

文章目录

elasticsearch.yml

1 先把无安全集群搭建出来

1.1

其中一个节点配置为

cluster.name: technical-support-sit
node.name: sit-node-128-75
path.data: /data/elastic/es/data
path.logs: /data/elastic/es/log
network.host: 10.201.128.75 # 本机IP
discovery.seed_hosts: ["sit-201-128-10-12-20-03", "sit-201-128-10-12-20-04", "sit-201-128-10-12-20-05"] # hostname
cluster.initial_master_nodes: ["node129-6","node129-5"] #  Bootstrap the cluster using an initial set of master-eligible nodes. node名字

1.2 kibana 配置

server.host: "10.201.128.75"
# The Kibana server's name.  This is used for display purposes.
server.name: "sit-128-75"
# The URLs of the Elasticsearch instances to use for all your queries.
elasticsearch.hosts: ["http://sit-201-128-10-12-20-03:9200"]

分别启动后,访问http://10.201.128.75:5601/,右下角点
在这里插入图片描述
了解完,需要先配置最低安全配置,最后再一层一层累加.
这里注意到
在这里插入图片描述
集群模式一定要先配置TLS再做如下操作.

配置完启动集群,执行

 ./bin/elasticsearch-setup-passwords auto

配置kibana

elasticsearch.username: "kibana_system"

执行

./bin/kibana-keystore create
./bin/kibana-keystore add elasticsearch.password
./bin/kibana

此时日志会一直INFO打印

...
[2021-12-29T20:16:24,494][INFO ][o.e.x.s.a.RealmsAuthenticator] [sit-node-128-77] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
[2021-12-29T20:16:34,503][INFO ][o.e.x.s.a.RealmsAuthenticator] [sit-node-128-77] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
[2021-12-29T20:16:44,471][INFO ][o.e.x.s.a.RealmsAuthenticator] [sit-node-128-77] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
[2021-12-29T20:16:54,518][INFO ][o.e.x.s.a.RealmsAuthenticator] [sit-node-128-77] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
...

Set up basic security for the Elastic Stack plus secured HTTPS trafficedit

配置完之后INFO会一直打印

...
[2021-12-29T20:45:14,479][WARN ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [sit-node-128-77] received plaintext http traffic on an https channel, closing connection Netty4HttpChannel{localAddress=/10.201.128.77:9200, remoteAddress=/10.201.128.77:47767}
[2021-12-29T20:45:18,142][WARN ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [sit-node-128-77] received plaintext http traffic on an https channel, closing connection Netty4HttpChannel{localAddress=/10.201.128.77:9200, remoteAddress=/10.201.128.75:22507}
[2021-12-29T20:45:18,367][WARN ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [sit-node-128-77] received plaintext http traffic on an https channel, closing connection Netty4HttpChannel{localAddress=/10.201.128.77:9200, remoteAddress=/10.201.128.76:49091}
...

7.16.2版本ES对证书支持正则,在生成证书时也说也可以用正则,而Kibana组件所有的证书不支持,
在执行

./bin/elasticsearch-certutil http

This command generates a .zip file that contains certificates and keys to use with Elasticsearch and Kibana. Each folder contains a README.txt explaining how to use these files.

  1. When asked if you want to generate a CSR, enter n.

  2. When asked if you want to use an existing CA, enter y.

  3. Enter the path to your CA. This is the absolute path to the elastic-stack-ca.p12 file that you generated for your cluster.

  4. Enter the password for your CA.

  5. Enter an expiration value for your certificate. You can enter the validity period in years, months, or days. For example, enter 90D for 90 days.

  6. When asked if you want to generate one certificate per node, enter y.

     Each certificate will have its own private key, and will be issued for a specific hostname or IP address.

  7. When prompted, enter the name of the first node in your cluster. Use the same node name that you used when generating node certificates.

  8. Enter all hostnames used to connect to your first node. These hostnames will be added as DNS names in the Subject Alternative Name (SAN) field in your certificate.

     List every hostname and variant used to connect to your cluster over HTTPS.

  9. Enter the IP addresses that clients can use to connect to your node.

  10. Repeat these steps for each additional node in your cluster.

也就是在6 中,不要看命令提示说可以输入通配符,这里一定不要输入,ES中是可以通过通配符访问,但到kibana中认证时就不支持了

elastic@SIT-201-128-10-12-20-03 es]$ ./bin/elasticsearch-certutil http

## Elasticsearch HTTP Certificate Utility

The 'http' command guides you through the process of generating certificates
for use on the HTTP (Rest) interface for Elasticsearch.

This tool will ask you a number of questions in order to generate the right
set of files for your needs.

## Do you wish to generate a Certificate Signing Request (CSR)?

A CSR is used when you want your certificate to be created by an existing
Certificate Authority (CA) that you do not control (that is, you don't have
access to the keys for that CA). 

If you are in a corporate environment with a central security team, then you
may have an existing Corporate CA that can generate your certificate for you.
Infrastructure within your organisation may already be configured to trust this
CA, so it may be easier for clients to connect to Elasticsearch if you use a
CSR and send that request to the team that controls your CA.

If you choose not to generate a CSR, this tool will generate a new certificate
for you. That certificate will be signed by a CA under your control. This is a
quick and easy way to secure your cluster with TLS, but you will need to
configure all your clients to trust that custom CA.

Generate a CSR? [y/N]N

## Do you have an existing Certificate Authority (CA) key-pair that you wish to use to sign your certificate?

If you have an existing CA certificate and key, then you can use that CA to
sign your new http certificate. This allows you to use the same CA across
multiple Elasticsearch clusters which can make it easier to configure clients,
and may be easier for you to manage.

If you do not have an existing CA, one will be generated for you.

Use an existing CA? [y/N]y

## What is the path to your CA?

Please enter the full pathname to the Certificate Authority that you wish to
use for signing your new http certificate. This can be in PKCS#12 (.p12), JKS
(.jks) or PEM (.crt, .key, .pem) format.
CA Path: /opt/elastic/es/es/elastic-stack-ca.p12
Reading a PKCS12 keystore requires a password.
It is possible for the keystore's password to be blank,
in which case you can simply press <ENTER> at the prompt
Password for elastic-stack-ca.p12:

## How long should your certificates be valid?

Every certificate has an expiry date. When the expiry date is reached clients
will stop trusting your certificate and TLS connections will fail.

Best practice suggests that you should either:
(a) set this to a short duration (90 - 120 days) and have automatic processes
to generate a new certificate before the old one expires, or
(b) set it to a longer duration (3 - 5 years) and then perform a manual update
a few months before it expires.

You may enter the validity period in years (e.g. 3Y), months (e.g. 18M), or days (e.g. 90D)

For how long should your certificate be valid? [5y] 10Y

## Do you wish to generate one certificate per node?

If you have multiple nodes in your cluster, then you may choose to generate a
separate certificate for each of these nodes. Each certificate will have its
own private key, and will be issued for a specific hostname or IP address.

Alternatively, you may wish to generate a single certificate that is valid
across all the hostnames or addresses in your cluster.

If all of your nodes will be accessed through a single domain
(e.g. node01.es.example.com, node02.es.example.com, etc) then you may find it
simpler to generate one certificate with a wildcard hostname (*.es.example.com)
and use that across all of your nodes.

However, if you do not have a common domain name, and you expect to add
additional nodes to your cluster in the future, then you should generate a
certificate per node so that you can more easily generate new certificates when
you provision new nodes.

Generate a certificate per node? [y/N]y

## What is the name of node #1?

This name will be used as part of the certificate file name, and as a
descriptive name within the certificate.

You can use any descriptive name that you like, but we recommend using the name
of the Elasticsearch node.

node #1 name: sit-node-128-75

## Which hostnames will be used to connect to sit-node-128-75?

These hostnames will be added as "DNS" names in the "Subject Alternative Name"
(SAN) field in your certificate.

You should list every hostname and variant that people will use to connect to
your cluster over http.
Do not list IP addresses here, you will be asked to enter them later.

If you wish to use a wildcard certificate (for example *.es.example.com) you
can enter that here.

Enter all the hostnames that you need, one per line.
When you are done, press <ENTER> once more to move on to the next step.

sit-201-128-10-12-20-03
sit-201-128-10-12-20-04
sit-201-128-10-12-20-05

You entered the following hostnames.

 - sit-201-128-10-12-20-03
 - sit-201-128-10-12-20-04
 - sit-201-128-10-12-20-05

Is this correct [Y/n]Y

## Which IP addresses will be used to connect to sit-node-128-75?

If your clients will ever connect to your nodes by numeric IP address, then you
can list these as valid IP "Subject Alternative Name" (SAN) fields in your
certificate.

If you do not have fixed IP addresses, or not wish to support direct IP access
to your cluster then you can just press <ENTER> to skip this step.

Enter all the IP addresses that you need, one per line.
When you are done, press <ENTER> once more to move on to the next step.


You did not enter any IP addresses.

Is this correct [Y/n]Y

## Other certificate options

The generated certificate will have the following additional configuration
values. These values have been selected based on a combination of the
information you have provided above and secure defaults. You should not need to
change these values unless you have specific requirements.

Key Name: sit-node-128-75
Subject DN: CN=sit-node-128-75
Key Size: 2048

Do you wish to change any of these options? [y/N]N
Generate additional certificates? [Y/n]Y

## What is the name of node #2?

This name will be used as part of the certificate file name, and as a
descriptive name within the certificate.

You can use any descriptive name that you like, but we recommend using the name
of the Elasticsearch node.

node #2 name: sit-node-128-76

## Which hostnames will be used to connect to sit-node-128-76?

These hostnames will be added as "DNS" names in the "Subject Alternative Name"
(SAN) field in your certificate.

You should list every hostname and variant that people will use to connect to
your cluster over http.
Do not list IP addresses here, you will be asked to enter them later.

If you wish to use a wildcard certificate (for example *.es.example.com) you
can enter that here.

Enter all the hostnames that you need, one per line.
When you are done, press <ENTER> once more to move on to the next step.

sit-201-128-10-12-20-03
sit-201-128-10-12-20-04
sit-201-128-10-12-20-05

You entered the following hostnames.

 - sit-201-128-10-12-20-03
 - sit-201-128-10-12-20-04
 - sit-201-128-10-12-20-05

Is this correct [Y/n]Y

## Which IP addresses will be used to connect to sit-node-128-76?

If your clients will ever connect to your nodes by numeric IP address, then you
can list these as valid IP "Subject Alternative Name" (SAN) fields in your
certificate.

If you do not have fixed IP addresses, or not wish to support direct IP access
to your cluster then you can just press <ENTER> to skip this step.

Enter all the IP addresses that you need, one per line.
When you are done, press <ENTER> once more to move on to the next step.


You did not enter any IP addresses.

Is this correct [Y/n]Y

## Other certificate options

The generated certificate will have the following additional configuration
values. These values have been selected based on a combination of the
information you have provided above and secure defaults. You should not need to
change these values unless you have specific requirements.

Key Name: sit-node-128-76
Subject DN: CN=sit-node-128-76
Key Size: 2048

Do you wish to change any of these options? [y/N]N
Generate additional certificates? [Y/n]Y

## What is the name of node #3?

This name will be used as part of the certificate file name, and as a
descriptive name within the certificate.

You can use any descriptive name that you like, but we recommend using the name
of the Elasticsearch node.

node #3 name: sit-node-128-77

## Which hostnames will be used to connect to sit-node-128-77?

These hostnames will be added as "DNS" names in the "Subject Alternative Name"
(SAN) field in your certificate.

You should list every hostname and variant that people will use to connect to
your cluster over http.
Do not list IP addresses here, you will be asked to enter them later.

If you wish to use a wildcard certificate (for example *.es.example.com) you
can enter that here.

Enter all the hostnames that you need, one per line.
When you are done, press <ENTER> once more to move on to the next step.

sit-201-128-10-12-20-03
sit-201-128-10-12-20-04
sit-201-128-10-12-20-05

You entered the following hostnames.

 - sit-201-128-10-12-20-03
 - sit-201-128-10-12-20-04
 - sit-201-128-10-12-20-05

Is this correct [Y/n]Y

## Which IP addresses will be used to connect to sit-node-128-77?

If your clients will ever connect to your nodes by numeric IP address, then you
can list these as valid IP "Subject Alternative Name" (SAN) fields in your
certificate.

If you do not have fixed IP addresses, or not wish to support direct IP access
to your cluster then you can just press <ENTER> to skip this step.

Enter all the IP addresses that you need, one per line.
When you are done, press <ENTER> once more to move on to the next step.


You did not enter any IP addresses.

Is this correct [Y/n]Y

## Other certificate options

The generated certificate will have the following additional configuration
values. These values have been selected based on a combination of the
information you have provided above and secure defaults. You should not need to
change these values unless you have specific requirements.

Key Name: sit-node-128-77
Subject DN: CN=sit-node-128-77
Key Size: 2048

Do you wish to change any of these options? [y/N]N
Generate additional certificates? [Y/n]N

## What password do you want for your private key(s)?

Your private key(s) will be stored in a PKCS#12 keystore file named "http.p12".
This type of keystore is always password protected, but it is possible to use a
blank password.

If you wish to use a blank password, simply press <enter> at the prompt below.
Provide a password for the "http.p12" file:  [<ENTER> for none]
Repeat password to confirm: 

## Where should we save the generated files?

A number of files will be generated including your private key(s),
public certificate(s), and sample configuration options for Elastic Stack products.

These files will be included in a single zip archive.

What filename should be used for the output zip file? [/opt/elastic/es/elasticsearch-7.16.2/elasticsearch-ssl-http.zip] 

Zip file written to /opt/elastic/es/elasticsearch-7.16.2/elasticsearch-ssl-http.zip

这样操作完之后,日志仍会不断输出INFO

...
[2021-12-30T15:26:26,823][WARN ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [sit-node-128-75] received plaintext http traffic on an https channel, closing connection Netty4HttpChannel{localAddress=/10.201.128.75:9200, remoteAddress=/10.201.128.77:42414}
[2021-12-30T15:26:35,077][WARN ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [sit-node-128-75] received plaintext http traffic on an https channel, closing connection Netty4HttpChannel{localAddress=/10.201.128.75:9200, remoteAddress=/10.201.128.75:33978}
[2021-12-30T15:27:06,154][WARN ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [sit-node-128-75] received plaintext http traffic on an https channel, closing connection Netty4HttpChannel{localAddress=/10.201.128.75:9200, remoteAddress=/10.201.128.76:64747}
[2021-12-30T15:27:06,955][WARN ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [sit-node-128-75] received plaintext http traffic on an https channel, closing connection Netty4HttpChannel{localAddress=/10.201.128.75:9200, remoteAddress=/10.201.128.75:33984}
...





# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
# discovery.seed_hosts: ["host1", "host2"]
discovery.seed_hosts: ["10.201.128.75","10.201.128.76","10.201.128.77"] # 需要IP

#
 Bootstrap the cluster using an initial set of master-eligible nodes:
cluster.initial_master_nodes:
        - 10.201.128.75
#        - 10.201.128.76
#        - 10.201.128.77
# 注意:只在初启动群集时有用,如果配置3个node,那么初始化时一定要都启动.

discovery.seed_hosts 也可通过如下做配置

# discovery.seed_hosts: ["10.201.128.75","10.201.128.76","10.201.128.77"]
discovery.seed_providers: file

然后在创建文件 $ES_PATH_CONF/unicast_hosts.txt. 只要unicast_hosts.txt改变,就会传到ES.
unicast_hosts.txt

10.10.10.5
10.10.10.6:9305
10.10.10.5:10005
# an IPv6 address
[2001:0db8:85a3:0000:0000:8a2e:0370:7334]:9301

这时host name是可以代替IP,被 DNS解析. 这时列出所有可以选举master表节点,但要注意如何三个节点,项链宕2个master,则第三个可能不能转换为master.

大怀特
关注
  • 0
    点赞
  • 踩
  • 1
    收藏
    觉得还不错? 一键收藏
  • 打赏
    打赏
  • 0
    评论
专栏目录
Windows10下安装Elasticsearch8.1.1过程遇到的问题
qq_45273678的博客
03-31 3535
Elasticsearch8.1.1安装过程遇到的问题，localhos：9200t页面无法正常运作， received plaintext http traffic on an https channel, closing connection elasticsearch默认用户名elastic，密码第一次运行elasticsearch.bat文件时会自动生成
Java配置14-elasticsearch重置用户密码
JustDI0209的博客
01-17 5319
1.现状 因为某些原因，重启了elasticsearch，发现报错 Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic] 也不知道什么原因，导致验证elastic用户失败 2.方法 进入elasticsearch安装目录的bin文件夹下，执行命令 ./elasticsearch-setup-passwords interactive ..
elastic 账户认证 401 问题？
u010069194的专栏
07-26 754
出现：Kibana server is not ready yet访问异常？elasticsearch 7.6.2 重置密码；同时测试 kibana 5601 登陆？本地 验证 elastic 账户。同样的 访问 kibana。检查 kibana 配置；如果 重置依然出现问题？
Elasticsearch环境搭建|ES单机|ES单节点模式启动|ES集群搭建|ES集群环境搭建
最新发布
浣花御劍
06-27 2523
ES单节点模式 | ES集群环境搭建
ES 认证
一条大河
11-15 3332
ES默认是不需要认证的，可以直接访问，但是这样不安全； 需要为其添加认证； 认证设置方式取自：Elasticsearch 7.8 开启用户名密码认证 - Kevin_zheng - 博客园 在elasticsearch.yml 末尾添加如下配置: # 配置X-Pack http.cors.enabled: true http.cors.allow-origin: "*" http.cors.allow-headers: Authorization xpack.security.enabled: tr
ELK: 给Elasticsearch, Kibanan设置访问的账号和密码
louis_lee7812的专栏
09-24 1万+
实现elasticsearch和kibana的账号密码登录
ElasticSearch集群搭建.doc
12-15
在本文中，我们将深入探讨如何搭建Elasticsearch集群以及索引分片的相关概念。Elasticsearch是一个分布式的、实时的搜索和分析引擎，它能够处理大量数据并提供高效的检索服务。在多节点环境下构建Elasticsearch集群...
elasticsearch集群搭建步骤6.2.2.docx
06-02
"Elasticsearch 集群...搭建 Elasticsearch 集群需要遵循严格的步骤，包括安装准备、JDK 安装、ES 安装及配置、集群搭建、集群优化和集群维护等步骤。只有遵循这些步骤，我们才能搭建一个高效的 Elasticsearch 集群。
ES集群搭建详细说明
12-12
### ES集群搭建详细说明 #### 一、概述 在分布式计算环境中，Elasticsearch (ES) 是一个非常流行的搜索引擎和分析引擎，适用于多种场景的数据存储、检索与分析。本篇文章将详细介绍如何搭建一个稳定的ES集群，并...
详解ElasticSearch6.4.0集群搭建
08-26
"详解ElasticSearch 6.4.0 集群搭建" ElasticSearch 是一个基于 Lucene 的搜索服务器，提供了分布式搜索引擎的功能。下面详解了 ElasticSearch 6.4.0 集群搭建的过程。 软件及环境准备 为了搭建 ElasticSearch ...
Eclipse中Android SDK Manager无法打开-附件资源
03-02
Eclipse中Android SDK Manager无法打开-附件资源
Elasticsearch 的用户认证和授权
互联网知识分享
07-28 1711
是一个开源的分布式搜索和分析引擎，它能够存储、搜索和分析海量的结构化和非结构化数据。）允许根据用户角色对用户进行授权。可以创建角色，并将用户分配到角色中，然后授予角色对资源的访问权限。，它使用内部用户数据库来进行用户认证和授权。是一种常用的企业级目录服务，用于集中管理用户和权限信息。进行用户认证和授权，以限制用户访问和操作索引的权限。中，用户认证和授权是独立的过程。）对用户进行授权，限制用户访问和操作资源的权限。中的数据安全，限制用户对数据的访问和操作权限。用户认证和授权的概念和实现方法，包括。
ES内存溢出，报错：java.lang.OutOfMemoryError: Java heap space
lcm_linux的博客
01-13 5753
es集群挂掉，报错信息如下： [WARN ][o.e.h.n.Netty4HttpServerTransport] [capitalcentre-14] caught exception while handling client http traffic, closing connection [id: 0x2f997a9e, L:/10.21.3.24:9200 - R:/10.21.3.27...
ES集群新增节点无法加入集群 timed out while waiting for initial discovery state - timeout: 30s
寒夜
11-06 4019
ES : 7.5.0 ES集群新增节点无法加入集群 timed out while waiting for initial discovery state - timeout: 30s   ES集群需要增加专用的master节点，加入完master节点需要修改data节点的elasticsearch.yml为专用的data节点并重启。这里出现了一个问题，在data节点重启后无法加入cluster当中，自己形成了一个集群。 部分elasticsearch.yml配置如下 node.master: fal.
elasticsearch7.6.0启动报错问题
star的博客
04-08 1722
elasticsearch7.6.0启动报错： java.lang.IllegalStateException: Received message from unsupported version: [6.1.0] minimal compatible version is: [6.8.0] 问题范围划定： 1、java环境，我是java version "1.8.0_211"导致的该...
elasticsearch启动常见错误
11-22 1942
root用户启动es出现如下错误 [root@bigdata124 ~]# cd /opt/module/elasticsearch-5.6.2/ [root@bigdata124 elasticsearch-5.6.2]# ./bin/elasticsearch [2019-11-21T18:02:25,703][WARN ][o.e.b.ElasticsearchUncaughtExcep...
centos7中配置elasticsearch集群和离线安装x-pack
热门推荐
山鬼谣的专栏
04-05 1万+
环境操作系统:win7 虚拟机：centos7 elasticsearch：5.2.2步骤前提条件由于我是自己学习用，笔记本的内存有限，所以想在一台虚拟机中配置集群。 我虚拟机的情况是，已经安装好了elasticsearch、kibana、x-pack.网上的一般做法是把已经安装好得elasticsearch目录再复制一遍，而我呢！由于保存了安装包，所以又重新解压了一份并且重命名为elasti
监控日志loging elastIcsearch 部署篇(二)
鱼的博客
08-01 627
ELK日志系统介绍 开源实时日志分析ELK平台能够完美的解决我们上述的问题，ELK由ElasticSearch、Logstash和Kiabana三个开源工具组成。官方网站：https://www.elastic.co/products Elasticsearch是个开源分布式搜索引擎，它的特点有：分布式，零配置，自动发现，索引自动分片，索引副本机制，restful风格接口，多数据源，自动搜索负载等 Logstash是一个完全开源的工具，他可以对你的日志进行收集、过滤，并将其存储供以后使用（如，搜
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

大怀特

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付：¥1
获取中
扫码支付

您的余额不足，请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值