ELK系列（一） ElasticSearch 8.9.2集群搭建

已于 2024-01-18 18:03:29 修改
阅读量1.2k 收藏 11
点赞数 10
分类专栏： ELK 文章标签： elk elasticsearch
于 2023-12-29 11:36:35 首次发布
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/BourneSu/article/details/135285691
版权

ELK系列传送门

ELK系列（一） ElasticSearch 8.9.2集群搭建

ELK系列（二）Kafka集群3.4.0搭建

基础环境配置

主机规划

主机名     内网地址      搭建程序

ELK8-1   192.168.0.1  elasticsearch+kafka+zookeeper
ELK8-2   192.168.0.2  elasticsearch+kafka+zookeeper
ELK8-3   192.168.0.3  elasticsearch+kafka+zookeeper
               192.168.0.4  kibana+logstash

环境配置

ELK8-1 ELK8-2 ELK8-3 kibana+logstash 统一配置

Base源及epel源配置 关闭防火墙
curl -o /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo
yum -y install vim wget lvm2 tcping
wget -O /etc/yum.repos.d/epel.repo https://mirrors.aliyun.com/repo/epel-7.repo
iptables -F
iptables -X
iptables -Z
systemctl stop firewalld

java环境配置
tar xf jdk-20_linux-x64_bin.tar.gz -C /usr/local/
vim /etc/profile.d/java.sh
JAVA_HOME=/usr/local/jdk-20.0.1
PATH=$PATH:$JAVA_HOME/bin
export JAVA_HOME PATH

source /etc/profile.d/java.sh
[root@ELK8-1 jdk-20.0.1]# java -version
java version "20.0.1" 2023-04-18
Java(TM) SE Runtime Environment (build 20.0.1+9-29)
Java HotSpot(TM) 64-Bit Server VM (build 20.0.1+9-29, mixed mode, sharing)

lvm搭建--数据目录/data

所需操作主机：ELK8-1 ELK8-2 ELK8-3

LVM挂载数据盘到/data目录
fdisk /dev/vdb
pvcreate /dev/vdb1
vgcreate vg1 /dev/vdb1
lvcreate -l 25599 -n lv1 vg1
mkfs.xfs /dev/vg1/lv1
mkdir /data
mount /dev/vg1/lv1 /data
mkdir /data/{elasticsearch,kafka,zookeeper}

ES集群搭建

安装yum源

所需操作主机：ELK8-1 ELK8-2 ELK8-3 logstash1

rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
vi /etc/yum.repos.d/elasticsearch.repo
[elasticsearch]
name=Elasticsearch repository for 8.x packages
baseurl=https://artifacts.elastic.co/packages/8.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=0
autorefresh=1
type=rpm-md

开启elastic-8.x版本下载
vi /etc/yum.repos.d/elastic.repo
[elastic-8.x]
name=Elastic repository for 8.x packages
baseurl=https://artifacts.elastic.co/packages/8.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

安装ES
yum install --enablerepo=elasticsearch elasticsearch

ELK8-1 ELK8-2 ELK8-3 安装后会得到超管用户elastic和密码请保存

ELK8-1  192.168.0.1  tIxjmD8nWabee6Rs0QRm
ELK8-2  192.168.0.2  1FL54xSW8N8sqP5o2RRH
ELK8-3  192.168.0.3  enxjhT8UFylO6eVqGEbp

系统优化设置

所需操作主机：ELK8-1 ELK8-2 ELK8-3

守护进程配置
sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable elasticsearch.service

jvm设置
sed -i 's/-Xms4g/-Xms4g/' /etc/elasticsearch/jvm.options
sed -i 's/-Xmx4g/-Xmx4g/' /etc/elasticsearch/jvm.options

data目录设置
mkdir -p /data/elasticsearch/data 
mkdir -p /data/elasticsearch/logs 
chown -R elasticsearch:elasticsearch /data/elasticsearch

所需操作主机：ELK8-1 ELK8-2 ELK8-3 logstash1

系统参数设置
vim /etc/security/limits.conf
* soft nofile 65536
* hard nofile 131072
* soft nproc 2048
* hard nproc 4096
echo "vm.max_map_count=262144" >> /etc/sysctl.conf
echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf
sysctl -p

ELK8-1启动ElasticSearch

ELK8-1 作为集群节点 ELK8-2及ELK8-3 加入ELK8-1所在集群

[root@ELK8-1 ~]# cat /etc/elasticsearch/elasticsearch.yml
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: ELK8  #集群名
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: ELK8-1  #节点名
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /data/elasticsearch/data  #数据目录
#
# Path to log files:
#
path.logs: /data/elasticsearch/logs  #日志目录
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# By default Elasticsearch is only accessible on localhost. Set a different
# address here to expose this node on the network:
#
network.host: 192.168.0.1  #内网地址
#
# By default Elasticsearch listens for HTTP traffic on the first free port it
# finds starting at 9200. Set a specific HTTP port here:
#
#http.port: 9200  #默认端口
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.seed_hosts: ["192.168.0.1"]  #可被发现节点
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
cluster.initial_master_nodes: ["192.168.0.1"]  #集群可成为master的节点
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Allow wildcard deletion of indices:
#
#action.destructive_requires_name: false

#----------------------- BEGIN SECURITY AUTO CONFIGURATION -----------------------
#
# The following settings, TLS certificates, and keys have been automatically
# generated to configure Elasticsearch security features on 11-09-2023 06:20:32
#
# --------------------------------------------------------------------------------

# Enable security features
xpack.security.enabled: true  #开启xpack

xpack.security.enrollment.enabled: true

# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:  #ssl证书配置 http.p12yum安装默认生成
  enabled: true
  keystore.path: certs/http.p12

# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl: #内网节点ssl配置 transport.p12安装默认生成
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
# Create a new cluster with the current node only
# Additional nodes can still join the cluster later
#cluster.initial_master_nodes: ["ELK8-1"]

# Allow HTTP API connections from anywhere
# Connections are encrypted and require user authentication
http.host: 0.0.0.0

# Allow other nodes to join the cluster from anywhere
# Connections are encrypted and mutually authenticated
#transport.host: 0.0.0.0  内网建议注释

#----------------------- END SECURITY AUTO CONFIGURATION -------------------------


启动ELK8-1
systemctl start elasticsearch
查看EL8-1状态及节点状态
curl -k -u elastic:tIxjmD8nWabee6Rs0QRm https://192.168.0.1:9200
curl -k -u elastic:tIxjmD8nWabee6Rs0QRm https://192.168.0.1:9200/_cat/nodes
192.168.0.1 84 98 3 0.09 0.07 0.09 cdfhilmrstw * ELK8-1 *号代表主节点

ELK8-2 ELK8-3加入集群

ELK8-1生成token

[root@ELK8-1 data]# /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s node  
eyJ2ZXIiOiI4LjkuM7RuSa27ilvkGSBwiL4NB1vRxnGNqGhjKKEeJ2DinW8diovGp0d2YYTqCIqS8iUTWozwo1iSZTbGDN0e9Uido7lhUhJN1edsc7ZcHI5THZ3In0=

ELK8-2 ELK8-3 通过token加入集群
/usr/share/elasticsearch/bin/elasticsearch-reconfigure-node --enrollment-token eyJ2ZXIiOiI4LjkuMiIsImFkciI6WyIxODMuMjUwLjE3OS43Mzo5MjAwIl0sImZnciI6ImM2OGM0YjM0OWUyYWFjMmViODhjY2NiMDhmMjRhMmFiNmY1MjMxYTllMDg4ODAzYTgwZWIwY2I1NzAxZDgxZGIiLCJrZXkiOiIwSDRLZzRvQmRCTXJvYkUwTEV4UzpqYmFCUFZVWlRBcWNiZjVCNHI5THZ3In0=

ELK8-2 ELK8-3 配置文件修改  ELK8-3配置参考ELK8-2
[root@elk8-2 ~]# cat /etc/elasticsearch/elasticsearch.yml
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: ELK8
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: ELK8-2
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /data/elasticsearch/data
#
# Path to log files:
#
path.logs: /data/elasticsearch/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# By default Elasticsearch is only accessible on localhost. Set a different
# address here to expose this node on the network:
#
network.host: 192.168.0.2
#
# By default Elasticsearch listens for HTTP traffic on the first free port it
# finds starting at 9200. Set a specific HTTP port here:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
#cluster.initial_master_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Allow wildcard deletion of indices:
#
#action.destructive_requires_name: false

#----------------------- BEGIN SECURITY AUTO CONFIGURATION -----------------------
#
# The following settings, TLS certificates, and keys have been automatically
# generated to configure Elasticsearch security features on 11-09-2023 07:04:53
#
# --------------------------------------------------------------------------------

# Enable security features
xpack.security.enabled: true

xpack.security.enrollment.enabled: true

# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12

# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
# Discover existing nodes in the cluster
discovery.seed_hosts: ["192.168.0.1:9300"] 

# Allow HTTP API connections from anywhere
# Connections are encrypted and require user authentication
http.host: 0.0.0.0

# Allow other nodes to join the cluster from anywhere
# Connections are encrypted and mutually authenticated
#transport.host: 0.0.0.0

ELK8-2 ELK8-3启动elasticsearch
systemctl start elasticsearch

ELK8-1观察节点状态 ELK8-2 ELK8-3已加入集群
[root@ELK8-1 logs]# curl -k -u elastic:tIxjmD8nWabee6Rs0QRm https://192.168.0.1:9200/_cat/nodes  
192.168.0.2  9 82 0 0.03 0.13 0.12 cdfhilmrstw - ELK8-2
192.168.0.1 19 81 0 0.03 0.02 0.05 cdfhilmrstw * ELK8-1
192.168.0.3  6 82 9 0.34 0.24 0.13 cdfhilmrstw - ELK8-3

ElasticSearch集群搭建完成

诉光海
关注
专栏目录
高可用elasticsearch集群搭建
lmj3732018的博客
11-23 1万+
Elasticsearch（简称为ES）是一个开源的分布式搜索和分析引擎，构建在Apache Lucene搜索引擎库之上。它提供了一个强大的全文搜索和实时分析功能，适用于大规模的数据存储、检索和分析。
ElasticSearch 8.6集群搭建
weixin_38333521的博客
02-12 1627
CentOS 8操作系统搭建ElasticSearch 8.6集群
8版本ELK集群部署
xhredeem的博客
12-30 1392
ELK集群搭建
ELK(八)ElasticSearch集群
weixin_43744059的博客
09-23 270
集群节点 ElasticSearch集群是由多个节点组成的,通过cluster.name设置集群名称,并且用于区分其他的集群,每个节点通过node.name指定节点的名称 在ElasticSearch中,节点的类型主要有4种 master节点 配置文件中node,master属性为true,就有资格被选为master节点 master节点用于控制整个集群的操作,比如创建或删除索引,管理其他非m...
银河麒麟 V10 上 Ansible 部署 Elasticsearch 8.9.2 的最佳实践
最新发布
linjifan的专栏
09-19 1076
使用 Ansible 自动化工具，实现在银河麒麟V10-SP3 x86架构版本上对 Elasticsearch 的一键式离线环境安装。
elasticsearch8.9.1集群搭建
haohaounique的博客
08-30 672
xpack.security.enabled: false 默认这个配置是开启的，配置文件中没有，启动后如果正常一般有一堆的安全配置属性也包括这个(xpack.security.enabled: true)，配置就麻烦很多，而且kibana登录时也要配置。elasticsearch文档：https://www.elastic.co/guide/en/elasticsearch/reference/8.9/targz.html。kbana在主节点上下载即可，其他机器可以不用(集群例外)
elasticsearch8.9.0集群安装指导（实战）
weixin_48865765的博客
08-23 814
elasticsearch8.9.0集群安装指导，实战。
elasticsearch集群搭建
ApexPredator的博客
07-04 1011
elasticsearch集群搭建
@ELK集群环境部署搭建
ଲ一笑奈@何
07-01 845
elk部署
ELK集群搭建(基础教程)
lqcbj的博客
01-30 3194
ELK集群搭建(基础教程) 目录： 机器准备 集群内各台机器安装Elasticsearch 安装部署Kafka（注：每个节点都配置，注意ip不同） 安装logstash工具 安装filebeat ELK收集Nginx的json日志 ELK收集Nginx正常日志和错误日志 ELK收集Tomcat日志 ELK收集docker日志 配置filebeat收集单个docker日志 modules日志收集 使用redis作为缓存收集日志 使用Kafka做缓存收集日志
ElasticSearch7.x集群搭建(es7)主从读写分离搭建教程
06-16
适用人群 开发从业人员，大数据从业人员...课程概述 ElasticSearch7.x集群？Elasticsearch-head7.x官网都没有的插件整合？cerebro插件酷炫暗黑插件神器？一次搞定 官网版本 主从读写 中文分词 Head插件 cerebro插件
Elasticsearch 3 节点集群搭建 (7.9.0)
DevOps 教练
08-27 1456
最近发布的 Elastic Stack 7.9 ，带来了很多新的特性。Elastic Agent 统一集成数据采集代理是一大亮点。另外还看增加了企业搜索、端点安全防护等组件。Ingest Manager 统一 Beat 配置管理功能让我们向 SaaS 风格的监控工具又迈进了一步。由代理端自行注册到后端，在后端统一纳管所有被管理服务器，将是一种以后非常通用的模式。这样做的好处是：将数据采集端点的配置工作量和复杂度降低到最低。Beats 的各种相关独立模块也在平行的发布，这种双轨模式也可以让用户更弹性的做出选择
Elastic Search 8.9：与 RRF 的混合搜索、更快的向量搜索和面向公众的搜索端点
Elastic 中国社区官方博客
07-30 2507
Elastic Search 8.9 引入了带有的混合搜索，以结合向量、关键字和语义技术以获得更好的结果。此版本还提高了向量搜索和摄取的性能，响应时间加快了 30% 以上。用户还可以通过新的 SharePoint Online 连接器获得更多摄取选项，其中包括文档级安全性。此外，8.9 还通过搜索应用程序测试版为你的 Elasticsearch® 索引带来了新的公共搜索端点。，这是唯一包含最新版本中所有新功能的托管 Elasticsearch 产品。你还可以。
elasticsearch8.9.1部署及配置基本安全性以及安全的 HTTPS 流量
qq_43700739的博客
09-13 2810
2、在任何单个节点上，从安装 Elasticsearch 的目录中，运行 Elasticsearch 用于生成证书签名请求 （CSR） 的 HTTP 证书工具。2、修改elasticsearch配置文件elasticsearch.yml(支持ipv4以及ipv6双栈)，文件权限修改成非root的应用程序账号。注：1、输出文件名称（默认）2、CA密码设置（可留空直接回车，生产环境建议设置）输入http.p12密码(配置了的节点都需要将密码存入安全设置中)注：1、输入CA密码。注：1、2：输入密码。
elk搭建
ApexPredator的博客
06-21 794
elk搭建
内网离线安装elasticsearch(8.9.2)、kibana(8.9.2)
略简略烦的博客
11-16 576
需要调试看信息的话，可在cmd窗口运行bat，就会打印输出信息了。在nssm.exe路径下，执行cmd窗口。
ELK的搭建—Elasticsearch-8.11.3的安装及集群的搭建
weixin_45760327的博客
01-08 3216
应用场景：信息检索，旅游网站，滴滴车离我多远。
ELK集群搭建(基础教程）
qq_43279371的博客
11-13 2562
测试创建topic（注：zookeeper保存信息为：topic创建的副本是3个，分区是3个，topic主体叫做Kafkatest）3、传入es的日志为json，那么显示在kibana的格式也是json，便于日志管理。配置主机id（注意：每台主机myid都不一样，每台都要配置）思路：1、将nginx中的日志以json格式记录。2、filebeat采的时候说明是json格式。2、filebeat采的时候说明是json格式。1、配置nginx的日志以json格式记录。2、修改filebeat的配置文件。
8.9 SpringBoot集成ElasticSearch之高亮
Steven的博客
01-05 479
1.condition开发 在项目目录“/src/main/java/com/example/es/condition”的EmployeeCondition类中实现SortProvider接口，具体代码如下。 @Data public class EmployeeCondition extends SampleEmployeeCondition implements RoutingProvider, ScoreFunctionProvider,SortProvider, HighlighterProvide
评论 1
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值