1 序言
Elasticsearch集群部署，只需要配置elasticsearch.yml配置文件，各个节点分别启动即可自动识别
2 环境
Window 7下创建三个虚拟机：centos6.7，jdk1.8，elasticsearch6.1.3
虚拟机配置如下：
| 类别 | 主机名 | ip | 内存 | 硬盘 | 主要目录 | 注释 |
| Master | master | 192.168.43.30 | 2G | 50G | /soft/elasticsearch-6.1.3/data | 索引目录 |
| /soft/elasticsearch-6.1.3/log | 日志目录 | |||||
| segments | slave1 | 192.168.43.31 | 2G | 50G | /soft/elasticsearch-6.1.3/data | 索引目录 |
| /soft/elasticsearch-6.1.3/log | 日志目录 | |||||
| segments | slave2 | 192.168.43.32 | 2G | 50G | /soft/elasticsearch-6.1.3/data | 索引目录 |
| /soft/elasticsearch-6.1.3/log | 日志目录 |
3 创建es用户
用户组：[root@localhost ~]# groupadd elasticsearch
用户：[root@localhost ~]# useradd es
密码：[root@localhost ~]# passwd es
用户添加到用户组：[root@localhost ~]# usermod -G elasticsearch es
设置用户sudo权限：
[root@localhost ~]# visudo
在root ALL=(ALL) ALL 一行下面
添加es用户 如下:
es ALL=(ALL) ALL
4 安装es
4.1 安装包处理
下载：https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.1.3.tar.gz
上传：分别上传到各节点/soft目录下
解压：tar -xvf /soft/elasticsearch-6.1.3.tar.gz
修改文件的所属用户:sudo chown -R es:elasticsearch /soft/elasticsearch-6.1.3
4.2 es配置
切换用户：su - es
进入安装包目录：cd /soft/elasticsearch-6.1.3
修改配置文件：vim config/elasticsearch.yml，具体内容如下：
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: my-es
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: node-1
#
# Add custom attributes to the node:
#
node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /soft/elasticsearch-6.1.3/data
#
# Path to log files:
#
path.logs: /soft/elasticsearch-6.1.3/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
bootstrap.memory_lock: false
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 192.168.43.30
#
# Set a custom port for HTTP:
#
http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
node.master: true
node.data: true
node.ingest: true
transport.tcp.port: 9300
discovery.zen.ping.unicast.hosts: ["192.168.43.30:9300","192.168.43.31:9300","192.168.43.32:9300"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):
#
discovery.zen.minimum_master_nodes: 2
#
# For more information, consult the zen discovery module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
http.cors.enabled: true
http.cors.allow-origin: "*"
bootstrap.system_call_filter: false
备注：另外两个节点配置：node.name: node-2、node.name: node-3；node.attr.rack: r2、r3；network.host: 192.168.43.31、network.host: 192.168.43.32，其它保持不变
4.3 elasticsearch.yml配置项说明
cluster.name:elasticsearch
配置es的集群名称，默认是elasticsearch，es会自动发现在同一网段下的es，如果在同一网段下有多个集群，就可以用这个属性来区分不同的集群。
node.name:”FranzKafka”
节点名，默认随机指定一个name列表中名字，该列表在es的jar包中config文件夹里name.txt文件中，其中有很多作者添加的有趣名字。
node.master:true
指定该节点是否有资格被选举成为node，默认是true，es是默认集群中的第一台机器为master，如果这台机挂了就会重新选举master。
node.data:true
指定该节点是否存储索引数据，默认为true。
index.number_of_shards:5
设置默认索引分片个数，默认为5片。
index.number_of_replicas:1
设置默认索引副本个数，默认为1个副本。
path.conf:/path/to/conf
设置配置文件的存储路径，默认是es根目录下的config文件夹。
path.data:/path/to/data
设置索引数据的存储路径，默认是es根目录下的data文件夹，可以设置多个存储路径，用逗号隔开，例：
path.data:/path/to/data1,/path/to/data2
path.work:/path/to/work
设置临时文件的存储路径，默认是es根目录下的work文件夹。
path.logs:/path/to/logs
设置日志文件的存储路径，默认是es根目录下的logs文件夹
path.plugins:/path/to/plugins
设置插件的存放路径，默认是es根目录下的plugins文件夹
bootstrap.mlockall:true
设置为true来锁住内存。因为当jvm开始swapping时es的效率会降低，所以要保证它不swap，可以把ES_MIN_MEM和ES_MAX_MEM两个环境变量设置成同一个值，并且保证机器有足够的内存分配给es。同时也要允许elasticsearch的进程可以锁住内存，linux下可以通过`ulimit-lunlimited`命令。
network.bind_host:192.168.0.1
设置绑定的ip地址，可以是ipv4或ipv6的，默认为0.0.0.0。network.publish_host:192.168.0.1
设置其它节点和该节点交互的ip地址，如果不设置它会自动判断，值必须是个真实的ip地址。
network.host:192.168.0.1
这个参数是用来同时设置bind_host和publish_host上面两个参数。
transport.tcp.port:9300
设置节点间交互的tcp端口，默认是9300。
transport.tcp.compress:true
设置是否压缩tcp传输时的数据，默认为false，不压缩。
http.port:9200
设置对外服务的http端口，默认为9200。
http.max_content_length:100mb
设置内容的最大容量，默认100mb
http.enabled:false
是否使用http协议对外提供服务，默认为true，开启。
gateway.type:local
gateway的类型，默认为local即为本地文件系统，可以设置为本地文件系统，分布式文件系统，hadoop的HDFS，和amazon的s3服务器，其它文件系统的设置方法下次再详细说。
gateway.recover_after_nodes:1
设置集群中N个节点启动时进行数据恢复，默认为1。
gateway.recover_after_time:5m
设置初始化数据恢复进程的超时时间，默认是5分钟。
gateway.expected_nodes:2
设置这个集群中节点的数量，默认为2，一旦这N个节点启动，就会立即进行数据恢复。
cluster.routing.allocation.node_initial_primaries_recoveries:4
初始化数据恢复时，并发恢复线程的个数，默认为4。
cluster.routing.allocation.node_concurrent_recoveries:2
添加删除节点或负载均衡时并发恢复线程的个数，默认为4。
indices.recovery.max_size_per_sec:0
设置数据恢复时限制的带宽，如入100mb，默认为0，即无限制。
indices.recovery.concurrent_streams:5
设置这个参数来限制从其它分片恢复数据时最大同时打开并发流的个数，默认为5。
discovery.zen.minimum_master_nodes:1
设置这个参数来保证集群中的节点可以知道其它N个有master资格的节点。默认为1，对于大的集群来说，可以设置大一点的值（2-4）
discovery.zen.ping.timeout:3s
设置集群中自动发现其它节点时ping连接超时时间，默认为3秒，对于比较差的网络环境可以高点的值来防止自动发现时出错。
discovery.zen.ping.multicast.enabled:false
设置是否打开多播发现节点，默认是true。
discovery.zen.ping.unicast.hosts:[“host1″,”host2:port”,”host3[portX-portY]”]
设置集群中master节点的初始列表，可以通过这些节点来自动发现新加入集群的节点
4.4 修改系统设置
1.切换回root用户
2.修改sysctl.conf：
vim /etc/sysctl.conf
在文件最后添加以下内容
vm.max_map_count=655360
保存后退出，再执行 sysctl -p 是设置立即生效
sysctl -p
3.修改limits.conf：
vim /etc/security/limits.conf
尾部追加内容如下
* soft nofile 65536
* hard nofile 65536
* soft nproc 131072
* hard nproc 131072
4.修改90-nproc.conf
vi /etc/security/limits.d/90-nproc.conf
修改如下内容：
* soft nproc 1024
#修改为
* soft nproc 4096
5.其余两个节点均需修改，完成后重启
6.其余两个节点均需修改
4.5 启动es
[root@bogon ~]# su - es
[es@bogon ~]$ cd /soft/elasticsearch-6.1.3
[es@bogon elasticsearch-6.1.3]$ ./bin/elasticsearch
4.6 查看启动是否成功
1.节点执行：“[root@bogon ~]# curl -X GET "192.168.43.30:9200/"”，结果如下：
{
"name" : "node-1",
"cluster_name" : "my-es",
"cluster_uuid" : "SzSwRqiqQbmA-LGe7Y2hpA",
"version" : {
"number" : "6.1.3",
"build_hash" : "af51318",
"build_date" : "2018-01-26T18:22:55.523Z",
"build_snapshot" : false,
"lucene_version" : "7.1.0",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}
2.节点执行：[root@bogon ~]# curl '192.168.43.30:9200/_cat/health?v'
4.7 关闭es
采用杀进程的方式
[root@bogon ~]# jps
3143 Jps
2988 Elasticsearch
[root@bogon ~]# kill -9 2988
5 安装elasticsearch-head插件
5.1 安装node
安装elasticsearch-head插件需要nodejs的支持，所以此处讲解一下安装nodejs步骤
下载：https://nodejs.org/en/download/
上传解压：[root@master soft]# tar -xvf node-v10.15.0-linux-x64
[root@master soft]# cd node-v10.15.0-linux-x64/bin/
[root@master bin]# pwd
/soft/node-v10.15.0-linux-x64/bin
添加环境变量：[root@master bin]# vim /etc/profile
[root@master bin]# source /etc/profile
查看node -v查看安装是否成功：[root@master bin]# node -v
5.2 安装elasticsearch-head插件
下载：https://github.com/mobz/elasticsearch-head
上传并解压：[root@master soft]# unzip elasticsearch-head-master.zip
安装：
[root@master soft]# cd elasticsearch-head-master
[root@master elasticsearch-head-master]# npm install
npm WARN deprecated coffee-script@1.10.0: CoffeeScript on NPM has moved to "coffeescript" (no hyphen)
npm WARN deprecated http2@3.3.7: Use the built-in module in node 9.0.0 or newer, instead
> phantomjs-prebuilt@2.1.16 install /soft/elasticsearch-head-master/node_modules/phantomjs-prebuilt
> node install.js
PhantomJS not found on PATH
Downloading https://github.com/Medium/phantomjs/releases/download/v2.1.1/phantomjs-2.1.1-linux-x86_64.tar.bz2
Saving to /tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2
Receiving...
[========================================] 100%
Received 22866K total.
Extracting tar contents (via spawned process)
Removing /soft/elasticsearch-head-master/node_modules/phantomjs-prebuilt/lib/phantom
Copying extracted folder /tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2-extract-1548727009699/phantomjs-2.1.1-linux-x86_64 -> /soft/elasticsearch-head-master/node_modules/phantomjs-prebuilt/lib/phantom
Phantom installation failed { [Error: EACCES: permission denied, link '/tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2-extract-1548727009699/phantomjs-2.1.1-linux-x86_64' -> '/soft/elasticsearch-head-master/node_modules/phantomjs-prebuilt/lib/phantom']
errno: -13,
code: 'EACCES',
syscall: 'link',
path:
'/tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2-extract-1548727009699/phantomjs-2.1.1-linux-x86_64',
dest:
'/soft/elasticsearch-head-master/node_modules/phantomjs-prebuilt/lib/phantom' } Error: EACCES: permission denied, link '/tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2-extract-1548727009699/phantomjs-2.1.1-linux-x86_64' -> '/soft/elasticsearch-head-master/node_modules/phantomjs-prebuilt/lib/phantom'
npm WARN elasticsearch-head@0.0.0 license should be a valid SPDX license expression
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.7 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.7: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! phantomjs-prebuilt@2.1.16 install: `node install.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the phantomjs-prebuilt@2.1.16 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2019-01-29T01_56_56_057Z-debug.log
执行该命名可能会出现以上错误，此时忽略phantomjs-prebuilt@2.1.16，执行命令如下：
[root@master elasticsearch-head-master]# npm install phantomjs-prebuilt@2.1.16 --ignore-scripts
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN elasticsearch-head@0.0.0 license should be a valid SPDX license expression
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.7 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.7: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
+ phantomjs-prebuilt@2.1.16
added 62 packages from 64 contributors, removed 4 packages and audited 1717 packages in 6.853s
found 23 vulnerabilities (16 low, 1 moderate, 6 high)
run `npm audit fix` to fix them, or `npm audit` for details
启动：[root@master elasticsearch-head-master]# npm run start
> elasticsearch-head@0.0.0 start /soft/elasticsearch-head-master
> grunt server
>> Local Npm module "grunt-contrib-jasmine" not found. Is it installed?
Running "connect:server" (connect) task
Waiting forever...
Started connect web server on http://192.168.43.30:9100:9100
查看：http://192.168.43.30:9100/
5.3 启动连接elasticsearch
切换到es启动elasticsearch集群：[es@master elasticsearch-6.1.3]# ./bin/elasticsearch
http://192.168.43.30:9100/，并连接到http://192.168.43.30:9200/，如图：
6 安装Kibana
6.1 定义
Kibana是Elasticsearch的开源数据可视化插件，为查看存储在ElasticSearch提供了友好的Web界面，并提供了条形图，线条和散点图，饼图和地图等分析工具
6.2 安装
下载：https://artifacts.elastic.co/downloads/kibana/kibana-6.1.3-linux-x86_64.tar.gz
上传并解压：[root@master soft]# tar -xvf kibana-6.1.3-linux-x86_64
配置：[root@master soft]# cd kibana-6.1.3-linux-x86_64/config/
[root@master config]# vim kibana.yml ，修改以下属性：
server.port: 5601
server.host: "192.168.43.30"
elasticsearch.url: "http://192.168.43.30:9200"
启动：先启动elasticsearch集群，再启动kibana
[root@master kibana-6.1.3-linux-x86_64]# ./bin/kibana
log [02:58:25.944] [info][status][plugin:kibana@6.1.3] Status changed from uninitialized to green - Ready
log [02:58:26.018] [info][status][plugin:elasticsearch@6.1.3] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [02:58:26.073] [info][status][plugin:console@6.1.3] Status changed from uninitialized to green - Ready
log [02:58:26.135] [info][status][plugin:metrics@6.1.3] Status changed from uninitialized to green - Ready
log [02:58:26.713] [info][status][plugin:timelion@6.1.3] Status changed from uninitialized to green - Ready
log [02:58:26.720] [info][listening] Server running at http://192.168.43.30:5601
log [02:58:26.836] [info][status][plugin:elasticsearch@6.1.3] Status changed from yellow to green - Ready
2139
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
