这次ES数据库学到很多东西,开始以为很简单,但是当自己开始动手做的时候,发现很多问题,总的来说,是下面几个问题:
1、ES数据库安装
2、ES数据库连接
3、ES数据库迁移
下面根据上面碰到的问题依次记录这次解决的办法,做一次记录,不枉这次这么辛苦的经历:
1、ES数据库安装
安装这个教程安装es数据库即可:https://blog.csdn.net/belonghuang157405/article/details/83301937
其中要注意的是挂载数据库磁盘文件的权限问题:需将data1 data2 data3 开启777权限=> chmod 777 data1 data2 data3
2、ES数据库连接
7.8.1版本,type类型没用了
3、ES数据库迁移
参考:https://cloud.tencent.com/developer/article/1621558
使用elasticdump工具,该工具是基于nodejs开发的,所以需要安装npm来安装elasticdump工具,
npm对应版本下载:https://developer.aliyun.com/mirror/NPM?from=tnpm
这次下载的版本是v12.18.4(注意版本问题,还有位数问题,系统64位,我弄了32位的,搞了好久),
npm安装过程参考:https://my.oschina.net/panquanxing/blog/3058043 (linux下 /bin是全局的一些命令链接、/usr/bin是系统的一些命令链接最好不要动,/usr/local/bin是用户的命令链接,这个修改无所谓,最好软链接到这个目录下)
在安装过程碰到问题参考我另外一篇:https://blog.csdn.net/LFGxiaogang/article/details/108791822
最后就是数据迁移过来,一些索引mapping和setting配置没有迁移过来,手动迁移
参考:https://blog.csdn.net/apple9005/article/details/90415558
创建索引
import sys
from random import random
from elasticsearch import Elasticsearch, helpers
class ESUtility():
def __init__(self, hosts=None, **kwargs):
from elasticsearch import Elasticsearch
self.es = Elasticsearch(hosts=hosts, **kwargs)
def create_index(index_name):
print("begin create index")
setting = {
"settings": {
"number_of_replicas": 0,
"number_of_shards": 5
},
"settings":{
"number_of_shards" : "5",
"blocks" : {
"read_only_allow_delete" : "false"
},
"analysis" : {
"analyzer" : {
"i_analyzer" : {
"filter" : [
"lowercase"
],
"tokenizer" : "i_pinyin"
}
},
"tokenizer" : {
"i_pinyin" : {
"trim_whitespace " : "true",
"lowercase" : "true",
"keep_original" : "true",
"keep_joined_full_pinyin " : "true",
"remove_duplicated_term" : "true",
"keep_first_letter" : "true",
"keep_separate_first_letter" : "false",
"type" : "pinyin",
"limit_first_letter_length" : "16",
"keep_full_pinyin" : "false"
}
}
},
"number_of_replicas" : "1",
},
"mappings":{
"properties":{
"basedataId" : {
"type" : "long"
},
"dicId" : {
"type" : "long"
},
"dicValueId" : {
"type" : "long"
},
"eid" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"number" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"value" : {
"type" : "text",
"fields" : {
"ik" : {
"type" : "text",
"analyzer" : "ik_max_word"
},
"pinyin" : {
"type" : "text",
"analyzer" : "pinyin"
}
}
}
}
}
}
es_client.indices.create(index=index_name, body=setting)
print("end create index")
index_name='test_dic_value'
es_client = Elasticsearch(hosts='172.18.8.118', port=9200)
create_index(index_name)
数据reindex
curl -H "Content-Type:application/json" -X POST http://172.18.8.118:9200/_reindex -d '{"source":{"index":"_test"}, "dest":{"index":"test_dic_value"}}'
# 删除索引
#curl -XDELETE 172.18.8.118:9200/test_dic_value_v1
在刚才参考步骤中,删除原来index时,记得重新创造,然后在使用第5步,