编写不易,转载请注明( http://shihlei.iteye.com/blog/2327971)!
概述
(一)简介
基于lucene 实现的近实时搜索服务,Restful 接口操作
(二)名词解释
Indics:索引库,相当于RDBMS的 数据库,整体控制分片(shard)和副本(Replica),一旦创建,分片不能改变。
Document type:索引类型
Document:索引记录,由唯一ID区分,ID决定Shard位置
Filed:一条索引记录中的组成字段,有类型概念,通过Mapping控制类型,是否分词,索引等
一 单机版安装
(一)环境
java 7 以上
环境:
CentOS Linux release 7.2.1511 (Core)
jre:1.8.0_101
版本:
elasticsearch-2.4.0.tar.gz
核心配置文件:
elasticsearch-2.4.0/config/elasticsearch.yml : yml协议的默认配置文件
elasticsearch-2.4.0/config/logging.xml :yml协议格式的日志配置文件
(二)安装
1)创建用户
groupadd es
useradd -r -g es -s /bin/bash es
chown -hR es:es /opt/elasticsearch-2.4.0
|
注:由于不能用Root用户启动,否则报java.lang.RuntimeException,所以创建用户
Exception in thread "main" java.lang.RuntimeException: don't run elasticsearch as root.
at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:94)
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:160)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:286)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
Refer to the log for complete error details.
|
2)解压缩
tar -xvf elasticsearch-2.4.0.tar.gz
|
3)启动
cd /opt/elasticsearch-2.4.0
bin/elasticsearch
|
日志:
[2016-09-22 08:45:21,503][WARN ][bootstrap ] unable to install syscall filter: seccomp unavailable: your kernel is buggy and you should upgrade
[2016-09-22 08:45:21,608][INFO ][node ] [Deacon Frost] version[2.4.0], pid[205], build[ce9f0c7/2016-08-29T09:14:17Z]
[2016-09-22 08:45:21,608][INFO ][node ] [Deacon Frost] initializing ...
[2016-09-22 08:45:21,977][INFO ][plugins ] [Deacon Frost] modules [reindex, lang-expression, lang-groovy], plugins [], sites []
[2016-09-22 08:45:21,994][INFO ][env ] [Deacon Frost] using [1] data paths, mounts [[/ (none)]], net usable_space [45gb], net total_space [59gb], spins? [possibly], types [aufs]
[2016-09-22 08:45:21,994][INFO ][env ] [Deacon Frost] heap size [998.4mb], compressed ordinary object pointers [unknown]
[2016-09-22 08:45:23,147][INFO ][node ] [Deacon Frost] initialized
[2016-09-22 08:45:23,148][INFO ][node ] [Deacon Frost] starting ...
[2016-09-22 08:45:23,251][INFO ][transport ] [Deacon Frost] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
[2016-09-22 08:45:23,255][INFO ][discovery ] [Deacon Frost] elasticsearch/6TSRp8FFQ3aXBPtkS4sROQ
[2016-09-22 08:45:26,316][INFO ][cluster.service ] [Deacon Frost] new_master {Deacon Frost}{6TSRp8FFQ3aXBPtkS4sROQ}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-join(elected_as_master, [0] joins received)
[2016-09-22 08:45:26,341][INFO ][http ] [Deacon Frost] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}
[2016-09-22 08:45:26,343][INFO ][node ] [Deacon Frost] started
[2016-09-22 08:45:26,359][INFO ][gateway ] [Deacon Frost] recovered [0] indices into cluster_state
|
注1 :后台启动方法:bin/elasticsearch -d
注2 : 这里绑定了两个端口
9200:默认的Http端口,用于Restful API的调用
9300:默认TCP端口,用户ES Node 之间的通讯,以及TCP 基本调用 Restful API , Java API TransportClient 需要连接该端口才能访问
注3:默认绑定127.0.0.1 本机访问,
远程访问需修改 ${ES_HOME}/config/elasticsearch.yml 的 network.host绑定指定的IP
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 0.0.0.0
|
(三)验证
浏览器:http://127.0.0.1:9200 返回
{
"name" : "Abomination",
"cluster_name" : "elasticsearch",
"version" : {
"number" : "2.4.0",
"build_hash" : "ce9f0c7394dee074091dd1bc4e9469251181fc55",
"build_timestamp" : "2016-08-29T09:14:17Z",
"build_snapshot" : false,
"lucene_version" : "5.5.2"
},
"tagline" : "You Know, for Search"
}
|
由节点名称等,说明安装成功
(四)简单操作
(1)插入:
$ curl -X PUT 'http://localhost:9200/indexdb/type/1' -d '
> {
> "name":"test",
> "desc":"test"
> }
> '
{"_index":"indexdb","_type":"type","_id":"1","_version":1,"_shards":{"total":2,"successful":1,"failed":0},"created":
|
(2)查询:
$ curl -XGET 'http://localhost:9200/indexdb/type/1'
{"_index":"indexdb","_type":"type","_id":"1","_version":1,"found":true,"_source":
{
"name":"test",
"desc":"test"
}
}
|
(五)关闭
<!--?xml version="1.0" encoding="UTF-8"?-->
for pid in `jps | grep Elasticsearch | awk '{print $1}'`
do
kill -9 $pid
done;
|
(六)常用插件安装
(1)Head:
方便对es进行各种操作的客户端(
https://github.com/mobz/elasticsearch-head)
bin/plugin install mobz/elasticsearch-head
|
访问:
http://127.0.0.1:9200/_plugin/head
|
![](https://i-blog.csdnimg.cn/blog_migrate/4ba4d22a94191f6ac0fd43d5f776e847.png)
(2)bigdesk:
监控es状态的插件(
https://github.com/lukas-vlcek/bigdesk)
bigdesk 的源码开发在 2015 年夏天就已经停止,所以默认是无法支持 Elasticsearch 2.x 的监控的
二 集群搭建
(一)概述
(1)ES Node 的角色:
Master Node : 通过选举出来,控制集群元数据和状态,压力较小,CRUD不需要通过 ( node.master=true)
Data Node : 拥有数据,并控制数据的CRUD,有查询压力 (node.data=true)
Client Node: 只能路由,转发响应的请求给Master Node 或者Data Node ,做聚合操作,可以分担部分Data Node 的压力,成本合适,可以创建。
Trib Node : 联合节点,控制到多个集群的访问
(2)架构
ES 的所有Node 节点是对等的,可以服务于客户端的查询,索引请求。结点通过选举成为Master,维持Cluster 状态,Master 所有的Node 周知。
(二)集群搭建
(1)架构规划
<!--?xml version="1.0" encoding="UTF-8"?-->
![](https://i-blog.csdnimg.cn/blog_migrate/43d3ce193f4dfee7e9968cf2104c116f.png)
说明:客户端通过 ClientNode 访问集群,ClientNode开放tcp,http访问,
所有DataNode置于后端,只开发TCP进行通讯
(2)目录规划
ES_HOME : /opt/elasticsearch-2.4.0
结点名称 | 配置文件目录 | 数据目录 | 日志目录 | pid目录 |
ClientNode | cluster/config/ClientNode | cluster/data/ClientNode | cluster/logs/ClientNode | cluster/ClientNode.pid |
DataNode1 | cluster/config/DataNode1 | cluster/data/DataNode1 | cluster/logs/DataNode1 | cluster/DataNode1.pid |
DataNode2 | cluster/config/DataNode2 | cluster/data/DataNode2 | cluster/logs/DataNode2 | cluster/DataNode2.pid |
DataNode3 | cluster/config/DataNode3 | cluster/data/DataNode3 | cluster/logs/DataNode3 | cluster/DataNode3.pid |
(3)端口规划
结点名称 | Http端口 | TCP端口 |
ClientNode | 9200 | 9300 |
DataNode1 | 9201 | 9301 |
DataNode2 | 9202 | 9302 |
DataNode2 | 9203 | 9303 |
(4)安装
步骤1:拷贝配置文件到约定目录:
cd /opt/elasticsearch-2.4.0
mkdir -p cluster/config/ClientNode && cp config/logging.yml cluster/config/ClientNode
mkdir -p cluster/config/DataNode1 && cp config/logging.yml cluster/config/DataNode1
mkdir -p cluster/config/DataNode2 && cp config/logging.yml cluster/config/DataNode2
mkdir -p cluster/config/DataNode3 && cp config/logging.yml cluster/config/DataNode3
|
步骤2:编辑个结点配置文件:
特别注意:ES的集群是通过自发现机制,需要cluster.name配置相同才会自动加入相同的集群,这里集群统一命名为ES_CLUSTER
下面的配置文件已经过删减,可以直接替换elasticsearch.yml
ClientNode:
ClientNode :node.master , node.data标志都为false,并开启http访问
vi cluster/config/ClientNode/elasticsearch.yml
# ---------------------------------- Cluster -----------------------------------
cluster.name: ES_CLUSTER
# ------------------------------------ Node ------------------------------------
node.name: ClientNode
http.enabled: true
node.master: false
node.data: false
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /opt/elasticsearch-2.4.0/cluster/data/ClientNode
#
# Path to log files:
#
path.logs: /opt/elasticsearch-2.4.0/cluster/logs/ClientNode
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 0.0.0.0
#
# Set a custom port for HTTP:
#
http.port: 9200
#
# Set a custom port for TCP:
transport.tcp.port: 9300
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.zen.ping.unicast.hosts: ["127.0.0.1:9301","127.0.0.1:9302","127.0.0.1:9303"]
|
DataNode1:
可选举成为Master 的DataNode : node.master,node.data都为true,由于客户端方位从ClientNode口出,http.enabled禁掉了下面同
vi cluster/config/DataNode1/elasticsearch.yml
# ---------------------------------- Cluster -----------------------------------
cluster.name: ES_CLUSTER
# ------------------------------------ Node ------------------------------------
node.name: DataNode1
http.enabled: false
node.master: true
node.data: true
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /opt/elasticsearch-2.4.0/cluster/data/DataNode1
#
# Path to log files:
#
path.logs: /opt/elasticsearch-2.4.0/cluster/logs/DataNode1
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 0.0.0.0
#
# Set a custom port for HTTP:
#
http.port: 9201
#
# Set a custom port for TCP:
transport.tcp.port: 9301
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.zen.ping.unicast.hosts: ["127.0.0.1:9301","127.0.0.1:9302","127.0.0.1:9303"]
|
DataNode2
vi cluster/config/DataNode2/elasticsearch.yml
# ---------------------------------- Cluster -----------------------------------
cluster.name: ES_CLUSTER
# ------------------------------------ Node ------------------------------------
node.name: DataNode2
http.enabled: false
node.master: true
node.data: true
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /opt/elasticsearch-2.4.0/cluster/data/DataNode2
#
# Path to log files:
#
path.logs: /opt/elasticsearch-2.4.0/cluster/logs/DataNode2
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 0.0.0.0
#
# Set a custom port for HTTP:
#
http.port: 9202
#
# Set a custom port for TCP:
transport.tcp.port: 9302
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.zen.ping.unicast.hosts: ["127.0.0.1:9301","127.0.0.1:9302","127.0.0.1:9303"]
|
DataNode3
vi cluster/config/DataNode3/elasticsearch.yml
# ---------------------------------- Cluster -----------------------------------
cluster.name: ES_CLUSTER
# ------------------------------------ Node ------------------------------------
node.name: DataNode3
http.enabled: false
node.master: true
node.data: true
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /opt/elasticsearch-2.4.0/cluster/data/DataNode3
#
# Path to log files:
#
path.logs: /opt/elasticsearch-2.4.0/cluster/logs/DataNode3
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 0.0.0.0
#
# Set a custom port for HTTP:
#
http.port: 9203
#
# Set a custom port for TCP:
transport.tcp.port: 9303
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
discovery.zen.ping.unicast.hosts: ["127.0.0.1:9301","127.0.0.1:9302","127.0.0.1:9303"]
|
特别注意:——坑:
transport.tcp.port
参考:http://knktc.com/2016/06/10/elasticsearch-multiple-instances/?utm_source=tuicool&utm_medium=referral
由于到了2.x版本之后,ES取消了默认的广播模式来发现master节点,需要使用该配置来指定发现master节点。这个配置在单机双实例的配置中需要特别注意下,因为习惯上我们配置时并未指定master节点的tcp端口,如果实例的transport.tcp.port配置为9301,那么实例启动后会认为discovery.zen.ping.unicast.hosts中指定的主机tcp端口也是9301,可能导致这些节点无法找到master节点。因此在该配置中需要指定master节点提供服务的tcp端口。
所以配置了
discovery.zen.ping.unicast.hosts: ["127.0.0.1:9301","127.0.0.1:9302","127.0.0.1:9303"]
|
(5)启动
启动命令为:
bin/elasticsearch -Des.path.conf=cluster/config/ClientNode -d -p cluster/ClientNode.pid
bin/elasticsearch -Des.path.conf=cluster/config/DataNode1 -d -p cluster/DataNode1.pid
bin/elasticsearch -Des.path.conf=cluster/config/DataNode2 -d -p cluster/DataNode2.pid
bin/elasticsearch -Des.path.conf=cluster/config/DataNode3 -d -p cluster/DataNode3.pid
|
(6)状态:
bash-4.2$ jps -lm
3815 org.elasticsearch.bootstrap.Elasticsearch start -Des.path.conf=cluster/config/DataNode1 -d -p cluster/DataNode1.pid
4152 sun.tools.jps.Jps -lm
4089 org.elasticsearch.bootstrap.Elasticsearch start -Des.path.conf=cluster/config/ClientNode -d -p cluster/ClientNode.pid
3836 org.elasticsearch.bootstrap.Elasticsearch start -Des.path.conf=cluster/config/DataNode2 -d -p cluster/DataNode2.pid
3885 org.elasticsearch.bootstrap.Elasticsearch start -Des.path.conf=cluster/config/DataNode3 -d -p cluster/DataNode3.pid
|
ClientNode
{ "name" : "ClientNode", "cluster_name" : "ES_CLUSTER", "version" : { "number" : "2.4.0", "build_hash" : "ce9f0c7394dee074091dd1bc4e9469251181fc55", "build_timestamp" : "2016-08-29T09:14:17Z", "build_snapshot" : false, "lucene_version" : "5.5.2" }, "tagline" : "You Know, for Search" } |
![](https://i-blog.csdnimg.cn/blog_migrate/031242da4af99cd7ddd175da85319086.png)
(7)停止
for pid in `jps | grep Elasticsearch | awk '{print $1}'`
do
kill -9 $pid
done;
|