文章目录
版本号 | 作者 | 版本号 | 备注 | |
---|---|---|---|---|
v1.0.1 | 若布与宫合 | 8416837 | UBUNTU 18 CENTOS 8 ES 7.5.2 Kibana 7.5.2 spring-data-elasticsearch 3.2.4 spring-boot 2.2.3 |
安装
Docker
# 创建网桥
docker network create somenetwork
# 在网桥跑es
docker run -d --name elasticsearch --net somenetwork -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" elasticsearch:7.5.2
# 在网桥跑kibana 注意版本与es相同
docker run -d --restart=always --net somenetwork --log-driver json-file --log-opt max-size=100m --log-opt max-file=2 --name xinyar-kibana -p 5601:5601 -v /home/app/elk/kibana.yml:/usr/share/kibana/config/kibana.yml kibana:7.5.2
# 网桥也可以如下传入
--network=container:elasticsearch
二者启动完毕↓
从Networks查看
ubuntu开启端口
- 访问es↓
- 访问kibana
ip:5601/app/kibana#/dev_tools/console?_g=()
- 执行自带查询,查询成功
docker compose
- docker-compose.yml
version: '3.7'
services:
elasticsearch:
image: elasticsearch
container_name: elasticsearch
hostname: elasticsearch
environment:
- cluster.name=docker-cluster
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- esdata1:/usr/share/elasticsearch/data
ports:
- 9200:9200
networks:
- esnet
elasticsearch2:
image: elasticsearch
container_name: elasticsearch2
hostname: elasticsearch2
environment:
- cluster.name=docker-cluster
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- "discovery.zen.ping.unicast.hosts=[elasticsearch,elasticsearch2,elasticsearch3]"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- esdata2:/usr/share/elasticsearch/data
networks:
- esnet
elasticsearch3:
image: elasticsearch
container_name: elasticsearch3
hostname: elasticsearch3
environment:
- cluster.name=docker-cluster
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- "discovery.zen.ping.unicast.hosts=[elasticsearch,elasticsearch2,elasticsearch3]"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- esdata3:/usr/share/elasticsearch/data
networks:
- esnet
kibana:
image: kibana
container_name: kibana
ports:
- 5601:5601
environment:
- ELASTICSEARCH_URL=http://elasticsearch:9200
networks:
- esnet
volumes:
esdata1:
driver: local
esdata2:
driver: local
esdata3:
driver: local
networks:
esnet:
CENTOS 8
安装包安装
# 下载略
sudo tar xvzf elasticsearch-7.6.1-linux-x86_64.tar.gz -C /home/app/ElasticSearch
cd /home/app/ElasticSearch/elasticsearch-7.6.1/
# 启动命令
bin/elasticsearch
# 提示:future versions of Elasticsearch will require Java 11; your Java version from [/usr/lib/jvm/jdk1.8.0_241/jre] does not meet this requirement - 将来的版本要求java11
# 平时使用jdk8
# 报错:不能使用root java.lang.RuntimeException: can not run elasticsearch as root
# 切换用户
su lilei
bin/elasticsearch
# 报错:Exception in thread "main" java.nio.file.AccessDeniedException: /home/app/ElasticSearch/elasticsearch-7.6.1/config/jvm.options 说明权限不够,命令前加上sudo # 若配置了权限,就不需要sudo
/home/app/ElasticSearch/elasticsearch-7.6.1/bin/elasticsearch
# 继续报错:java.lang.RuntimeException: can not run elasticsearch as root 没关系,解决就是。说明账户lilei也是root角色呀。
# 添加新的用户组
sudo groupadd elsearch;sudo useradd elsearch -g elsearch;sudo passwd elsearch
# 要求输入新的密码
# 输入lileiel作为密码
# passwd: all authentication tokens updated successfully.
# 继续执行。报错:elsearch is not in the sudoers file. This incident will be reported 当前使用的账户elsearch不属于sudoer
# 添加sudoer
su root
echo 'elsearch ALL=(ALL) ALL' >> /etc/sudoers
su elsearch # 切换回去
# 错错错,当然不对了,上文把elsearch变成了root用户!
# 正解见下文:
su root
sudo chown -R elsearch:elsearch elasticsearch-7.6.1/
# 将权限赋给elsearch。不要加sudo
# 执行jps,显示:8277 Elasticsearch 启动成功
- 访问
curl -X GET http://localhost:9200/
{
"name" : "localhost.localdomain",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "vdJlG_JuSreD3wZROjqorw",
"version" : {
"number" : "7.6.1", // es版本号
"build_flavor" : "default",
"build_type" : "tar",
"build_hash" : "aa751e09be0a5072e8570670309b1f12348f023b",
"build_date" : "2020-02-29T00:15:25.529771Z",
"build_snapshot" : false,
"lucene_version" : "8.4.0", // lucene版本号
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search" // 宣传词
}
- 配置网络访问,首先确保防火墙开启了端口或者防火墙关闭
vim /home/app/ElasticSearch/elasticsearch-7.6.1/config/elasticsearch.yml
# 配置ip和端口
启动,报错:max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535] 可能一直报错,需要增加配置,直到不报错
su root
sudo vim /etc/security/limits.conf
添加内容:
#切换到root用户修改
sudo vim /etc/security/limits.conf
# 更改linux的最大文件描述限制要求
* soft nofile 262144
* hard nofile 262144
es soft nofile 262144
es hard nofile 262144
* soft nproc 2048
* hard nproc 4096
es soft nproc 2048
es hard nproc 4096
#更改linux的锁内存限制要求
es soft memlock unlimited
es hard memlock unlimited
#然后 logout 重新登录 (7.6.1好像不重登录,不执行下述命令也行)
ulimit -Hn
262144
继续修改:sudo vim /etc/security/limits.d/90-nproc.conf
内容:
# 使用root
# vim /etc/security/limits.d/90-nproc.conf
* soft nproc 2048
root soft nproc 2048
再修改:sudo vim /etc/security/limits.conf
内容:
es soft nproc 4096
es hard nproc 4096
再修改:sudo vim /etc/sysctl.conf
内容:
#使用root
# vim /etc/sysctl.conf
#添加下面配置:
vm.max_map_count=655360
#更改linux禁用swapping,添加或修改如下:
vm.swappiness = 1
#并执行命令:
sysctl -p
继续修改:sudo vim config/elasticsearch.yml
增加:cluster.initial_master_nodes: ["node-1"]
- 上述修改了那么多,终于启动成功了:
ps:与底层同时跑,🐂🍺啊,——在中央仓库看到lucene的最新版本是8.4.1
- 查看jdk与es版本对应
demo
命令行
用命令行建索引并查询
- 索引
curl -XPUT 'http://192.168.191.6:9200/twitter/_doc/1?pretty' -H 'Content-Type: application/json' -d '
{
"user": "kimchy",
"post_date": "2009-11-15T13:12:00",
"message": "Trying out Elasticsearch, so far so good?"
}'
索引结果:
{
"_index" : "twitter",
"_type" : "_doc",
"_id" : "1",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1, // 成功
"failed" : 0
},
"_seq_no" : 0,
"_primary_term" : 1
}
- 查看
curl -XGET 'http://192.168.191.6:9200/twitter/_doc/1?pretty=true'
# 或者
curl -XGET 'http://192.168.191.6:9200/twitter/_search?pretty=true' -H 'Content-Type: application/json' -d '
{
"query" : {
"match" : { "user": "kimchy" }
}
}'
结果:
{
"_index" : "twitter",
"_type" : "_doc",
"_id" : "1",
"_version" : 1,
"_seq_no" : 0,
"_primary_term" : 1,
"found" : true,
"_source" : {
"user" : "kimchy",
"post_date" : "2009-11-15T13:12:00",
"message" : "Trying out Elasticsearch, so far so good?"
}
}
- 搜索
搜索twitter目录下的文档
curl -XGET 'http://192.168.191.6:9200/twitter/_search?q=user:kimchy&pretty=true'
响应:
{
"took" : 136,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 1,
"relation" : "eq"
},
"max_score" : 0.2876821,
"hits" : [
{
"_index" : "twitter",
"_type" : "_doc",
"_id" : "1",
"_score" : 0.2876821,
"_source" : {
"user" : "kimchy",
"post_date" : "2009-11-15T13:12:00",
"message" : "Trying out Elasticsearch, so far so good?"
}
}
]
}
}
图形界面
分词器
中文
IK
ik,起先做solr的时候也用的这个,记得当时还有其它收费分词器。
历史版本下载
安装kibana
- 脚本
sudo tar xvzf kibana-7.6.1-linux-x86_64.tar.gz -C /home/app/Kibana/
cd /home/app/Kibana/kibana-7.6.1-linux-x86_64
# 目录授权
sudo chown -R elsearch:elsearch kibana-7.6.1-linux-x86_64/
# 安装sense
# 切换非root账户
su elsearch
sh bin/kibana plugin --install elastic/sense # 5+以后不需要安装.在界面点Dev Tools
# 启动kibana 默认5601
# run
# run es
/home/app/apache-elasticsearch/elasticsearch-2.4.4/bin/elasticsearch
# run k 可能有点慢 【注意】第一次启动报错: FATAL Error: EACCES: permission denied, stat '.i18nrc.json' 无权限 国际化文件无权限错误 ;第二次启动,就成功了
/home/app/Kibana/kibana-7.6.1-linux-x86_64/bin/kibana # 注意es的ip
# 访问.应先在配置文件配置ip访问
curl http://192.168.191.6:5601/app/sense
# 控制台
http://192.168.191.6:5601/app/kibana #/dev_tools/console # 选择Dev Tools(扳手logo|icon) 进入控制台
- 效果
查询
关联查询
关系型数据库和ES各尽所能,将全量同步到ES,不做冗余存储。
相较mysql,ES 擅长检索,而不是RDB-关系数据库。应结合二者特性,使用 ES 多索引建立相同的别名,针对别名检索到对应 ID 后再回 MySQL 查询,业务层面通过关联 ID join 出需要的数据。
kibana
使用
先扔点数据试试
建索引
PUT api
{
"settings":{
"index":{
"number_of_shards":5,
"number_of_replicas":1
}
}
}
#number_of_shards 分片数量
#number_of_replicas 副本数量
#修改副本数量设置
PUT api/_settings
{
"number_of_replicas":2
}
存数据 PUT需指定id
1)
- 传参数 注意最多只能三级 分别是索引 类型 id
PUT /api/user/123
{
"name": "zhangsan",
"age": 13,
"sex": 1
}
2)
PUT /api/_doc/456
{
"name": "liwu",
"age": 113,
"sex": 0
}
out:
{
"_index" : "api",
"_type" : "_doc",
"_id" : "456",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 0,
"_primary_term" : 1
}
3)post数据
- input:
POST /api/_doc
{
"name": "Liliehuo",
"age": 27,
"sex": 1,
"email": "123789@qq.com"
}
- output:
{
"_index" : "api",
"_type" : "_doc",
"_id" : "X0VmXHYB-rlniG7C132r",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 2,
"_primary_term" : 1
}
4)指定type会导致警告消息出现
5)类型只能使用一个?
保存数据,第二次之后,类型可以使用_doc代替
把刚才新建的数据查出来
- input:
GET /api/_search
{
"query": {
"match_all": {
}
}
}
- output:
{
"took" : 12,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "api",
"_type" : "user",
"_id" : "456",
"_score" : 1.0,
"_source" : {
"name" : "liwu",
"age" : 113,
"sex" : 0
}
},
{
"_index" : "api",
"_type" : "user",
"_id" : "123",
"_score" : 1.0,
"_source" : {
"name" : "zhangsan",
"age" : 13,
"sex" : 1
}
}
]
}
}