es常见操作

查询检索的时候，不同的语言创建的是不一样的索引的，所以，每一次查询的时候是需要携带查询语言的

#安装pinyin插件
需要单独的编译或者是使用外部提供的编译插件
#安装ik插件
IK中文分词器：https://github.com/medcl/elasticsearch-analysis-ik
拼音分词器：https://github.com/medcl/elasticsearch-analysis-pinyin
具体的操作可以参考文档https://www.cnblogs.com/yijialong/p/9719964.html完成编译替换操作

es常见的命令操作

1.查看es的mapping数据信息
curl -XGET ‘localhost:9200/app_search/_mappings?pretty’
2.删除指定的索引
curl -XDELETE ‘localhost:9200/app_search’
3.查看settings信息
curl -XGET ‘localhost:9200/app_search/_settings?pretty’
4.查看mapping信息
curl -XGET ‘localhost:9200/app_search/_mappings?pretty’
5.查询es中的记录数量
curl -XGET ‘localhost:9200/app_search/_count?pretty’
6.分页查询和测试数据
curl -H “Content-Type:application/json” -XGET localhost:9200/app_search/_search -d ‘{“query” : {“match_all” : {}},“from”: “0”,“size”: “1”}’
7.触发数据重新加载操作
curl -H “Content-Type:application/json” -X POST http://localhost:8080/init/triggerDataLoad
8查询packageName匹配的元素信息，其中com.chaozh.iReaderFree对应的是数值信息
curl -X POST “http://localhost:8080/search/list/ids” -H “accept: /” -H “Content-Type: application/json” -d “{ “ids”: [ “4522” ], “termQueries”: [ { “name”: “package_name”, “value”: “com.chaozh.iReaderFree” } ]}”
curl -H “Content-Type:application/json” -X POST -d ‘[“com.chaozh.iReaderFree”,“com.tencent.tmgp.gzzwyx.bzfgx”]’ http://localhost:8080/appSearch/queryByPackageNames
curl -X POST “http://localhost:8080/search/list/pkg” -H “accept: /” -H “Content-Type: application/json” -d “{ “packageNames”: [ “com.chaozh.iReaderFree” ], “termQueries”: [ { “name”: “download_url”, “value”: “https://newmarket3.tt286.com/app/openzhuoyi/2021/07/19/15/81a795f862a2513c33c110a382ec6df9.apk” } ]}”
9.查询apkId的数据信息
curl -X POST “http://localhost:8080/search/list/ids” -H “accept: /” -H “Content-Type: application/json” -d “{ “ids”: [ “4522” ], “termQueries”: [ { “name”: “package_name”, “value”: “com.chaozh.iReaderFree” } ]}”
curl -H “Content-Type:application/json” -X POST -d ‘[“4522”,“318860”]’ http://localhost:8080/appSearch/queryByApkIds
10.清空所有的数据
curl -H “Content-Type:application/json” -X POST http://localhost:9200/app_search/_delete_by_query -d ‘{“query”: {“match_all”: {}}}’
11.重建索引:重建索引完成之后需要进一步的确认索引是否正确的。
curl -X POST http://localhost:8080/appSearch/createIndex
12.根据关键词检索
curl -X POST “http://localhost:8080/search/list/keyword” -H “accept: /” -H “Content-Type: application/json” -d “{ “from”: 0, “keyword”: “ireader”, “lang”: “string”, “size”: 3, “sortFields”: [ { “ascending”: true, “field”: “apk_id” } ], “termQueries”: [ { “name”: “download_url”, “value”: “https://newmarket3.tt286.com/app/openzhuoyi/2021/07/19/15/81a795f862a2513c33c110a382ec6df9.apk” } ]}”
13.查询当前存在的索引列表
curl -X GET “localhost:9200/_cat/indices?pretty”
#数据库授权操作
create user elasticsearch identified by ‘Hihonor@2021’;
flush privileges;
show grants for elasticsearch;
GRANT USAGE ON . TO ‘elasticsearch’@’%’
GRANT ALL PRIVILEGES ON es.* TO ‘elasticsearch’@’%’

#大量索引数据的清空操作和实现
curl -H “Content-Type:application/json” -X POST http://localhost:9200/app_search_zh_cn/_delete_by_query -d ‘{“query”: {“match_all”: {}}}’
curl -H “Content-Type:application/json” -X POST http://localhost:9200/app_search_uk/_delete_by_query -d ‘{“query”: {“match_all”: {}}}’
curl -H “Content-Type:application/json” -X POST http://localhost:9200/app_search_ka/_delete_by_query -d ‘{“query”: {“match_all”: {}}}’
curl -H “Content-Type:application/json” -X POST http://localhost:9200/app_search_en/_delete_by_query -d ‘{“query”: {“match_all”: {}}}’
curl -H “Content-Type:application/json” -X POST http://localhost:9200/app_search_kk/_delete_by_query -d ‘{“query”: {“match_all”: {}}}’
curl -H “Content-Type:application/json” -X POST http://localhost:9200/app_search_uz/_delete_by_query -d ‘{“query”: {“match_all”: {}}}’
curl -H “Content-Type:application/json” -X POST http://localhost:9200/app_search_be/_delete_by_query -d ‘{“query”: {“match_all”: {}}}’
curl -H “Content-Type:application/json” -X POST http://localhost:9200/app_search_ru/_delete_by_query -d ‘{“query”: {“match_all”: {}}}’
#查询对应索引中当前的记录数据
curl -XGET ‘localhost:9200/app_search_zh_cn/_count?pretty’
curl -XGET ‘localhost:9200/app_search_uk/_count?pretty’
curl -XGET ‘localhost:9200/app_search_ka/_count?pretty’
curl -XGET ‘localhost:9200/app_search_en/_count?pretty’
curl -XGET ‘localhost:9200/app_search_kk/_count?pretty’
curl -XGET ‘localhost:9200/app_search_uz/_count?pretty’
curl -XGET ‘localhost:9200/app_search_be/_count?pretty’
curl -XGET ‘localhost:9200/app_search_ru/_count?pretty’
curl -H “Content-Type:application/json” -XGET localhost:9200/app_search_ru/_search -d ‘{“query” : {“match_all” : {}},“from”: “0”,“size”: “10”}’
curl -H “Content-Type:application/json” -XGET localhost:9200/app_search_zh_cn/_search -d ‘{“query” : {“match_all” : {}},“from”: “0”,“size”: “1”}’
curl -H “Content-Type:application/json” -XGET localhost:9200/app_search_en/_search -d ‘{“query” : {“match_all” : {}},“from”: “0”,“size”: “1”}’

获取es相关的snapshot的操作和配置

curl -X GET “localhost:9200/_snapshot/my_backup?pretty”
#查询存在多少repo记录
curl -X GET “localhost:9200/_snapshot/my_backup/_all?pretty”
curl -XPUT ‘http://localhost:9200/_snapshot/my_backup’ -H ‘Content-Type: application/json’ -d ‘{
“type”: “fs”,
“settings”: {
“location”: “/opt/hihonor/snapshot/my_backup”,
“compress”: true
}
}’
#创建repo操作
curl -X PUT “localhost:9200/_snapshot/my_backup/snapshot_2?wait_for_completion=true&pretty”
#es更新数据执行操作
curl -H “Content-Type:application/json” -XPOST ‘localhost:9200/app_search_ru/_doc/f9ee0578fe1cc94de7482bd41accb329/_update’ -d ‘{
“doc” : {
“apk_name_lang” : null
}
}’

3.查看settings信息
curl -XGET ‘localhost:9200/package_mapping/_settings?pretty’
4.查看mapping信息
curl -XGET ‘localhost:9200/package_mapping/_mappings?pretty’
curl -XGET ‘localhost:9200/package_mapping/_count?pretty’
curl -XDELETE ‘localhost:9200/package_mapping’

搜索数据记录

curl -H “Content-Type:application/json” -XGET localhost:9200/package_mapping/_search -d ‘{“query” : {“match” : {“package_name”:“com.simplywerx.compass3d”}},“from”: “0”,“size”: “1”}’
#执行数据查询操作bdb74094e7a23a92ed9c3aef67a03912
curl -XGET “Content-Type:application/json” ‘http://localhost:9200/package_mapping/_doc/bdb74094e7a23a92ed9c3aef67a03912?pretty’
curl -H “Content-Type:application/json” -XGET localhost:9200/package_mapping/_search -d ‘{“query” : {“match” : {“package_name”:“com.simplywerx.compass3d”}},“from”: “0”,“size”: “1”}’
curl -XGET “Content-Type:application/json” ‘http://localhost:9200/app_search_ru/_doc/640743fb62bf7d552ce7fc1368bf76e5?pretty’
curl -XGET “Content-Type:application/json” ‘http://localhost:9200/app_search_ru/_doc/5de76e9c94f9aeaca1406687250a305f?pretty’
curl -XGET “Content-Type:application/json” ‘http://localhost:9200/app_search_ru/_doc/a694248fe1dcdacfe10681cde8be5b7d?pretty’

需要改写查询操作语句

curl -XPOST localhost:9200/app_search_ru/_search -H “Content-Type: application/json” -d ‘{ “ids”: [ “1318670” ], “lang”: “ru”, “termQueries”: [{ “name”:“download_url”, “value”:“http://18.177.149.123:8001/apk/air.com.hypah.io.slither.apk”},{ “name”:“is_spread”, “value”:“3”} ]}’

几个常见的接口的使用和测试

1.测试根据apkId检索数据记录操作
curl -XPOST localhost:9200/app_search_ru/_search?pretty -H “Content-Type: application/json” -d’{“query”: {“bool”: {“must”: [{ “terms”: { “apk_id”: [“1318670”] }},{ “term”: { “download_url”: “http://18.177.149.123:8001/apk/air.com.hypah.io.slither.apk” }},{ “term”: { “is_spread”: “3” }}]}},“from”:0,“size”:20}’
2.分页查询相关的数据记录
curl -H “Content-Type:application/json” -XGET localhost:9200/app_search_zh_cn/_search?pretty -d ‘{“query” : {“match_all” : {}},“from”: “0”,“size”: “10”}’
3.根据关键字检索es搜索记录
3.1根据关键字搜索
curl -XPOST localhost:9200/app_search_ru/_search?pretty -H “Content-Type: application/json” -d’{“query”: {“bool”: {“should”: [{“match_phrase_prefix”: {“apk_name_lang”:{“query”:".io",“slop”: 1,“boost”: 10}}}, {“match”: {“apk_name_lang”:{ “query”:".io",“boost”:“9”} }}, {“match”: {“seo_key”: {“query”:".io",“boost”:“1”}}}]}},“from”: 0,“size”: 10}’
3.2根据拼音进行搜索
curl -XPOST localhost:9200/app_search_ru/_search?pretty -H “Content-Type: application/json” -d’{“query”: {“bool”: {“must”: [{“multi_match”:{“query”:".io",“fields”:[“apk_name_lang.pinyin”,“new_label_name.pinyin”],“type”:“most_fields”}}]}},“from”: 0,“size”: 10}’
4.根据包名实现数据检索操作
curl -XPOST localhost:9200/app_search_ru/_search?pretty -H “Content-Type: application/json” -d’{“query”: {“bool”: {“must”: [{ “terms”: { “package_name”: [“air.com.hypah.io.slither”] }},{ “term”: { “download_url”: “http://18.177.149.123:8001/apk/air.com.hypah.io.slither.apk” }},{ “term”: { “is_spread”: “3” }}]}},“from”:0,“size”:20}’
#top3的数据测试结果
测试步骤:
1.没有关键字的top3的数据。直接根据keyword在不同语言下面执行查询操作
curl -X POST “http://10.68.161.122:8080/search/list/keyword” -H “accept: /” -H “Content-Type: application/json” -d “{ “from”: 0, “keyword”: “.io”, “lang”: “ru”, “size”: 50, “sortFields”: [ ], “termQueries”: [ ]}”
2.增加了top3的操作之后，对应的查询语句如下的：
curl -X POST “http://10.68.161.122:8082/search/list/keyword?key=.io&lang=ru” -H “accept: /”

#执行操作备份了数据
curl -X PUT “localhost:9200/_snapshot/my_backup/snapshot_2?wait_for_completion=true&pretty”

es的查询更新操作,执行数据更新和数据维护操作

1.检查查询数据操作:查询字段为空的数据
curl -X POST -H “Content-Type: application/json” localhost:9200/app_search_ru/_search?pretty -d ‘{“query”: {“bool”: {“must_not”: [{“exists”: {“field”:“seo_key”}}]}}}’
2.执行数据的更新操作实现,执行数据更新操作实现
curl -X POST -H “Content-Type: application/json” localhost:9200/app_search_ru/_update_by_query?pretty -d ‘{“script”: {“source”: “ctx._source.seo_key=ctx._source.apk_name_lang”},“query”: {“bool”: {“must_not”: [{“exists”: {“field”: “seo_key”}}]}}}’
3.查询处理之后的符合条件的数据的总量
curl -XGET -H “Content-Type: application/json” ‘localhost:9200/app_search_ru/_count?pretty’ -d ‘{“query”: {“bool”: {“must_not”: [{“exists”: {“field”:“seo_key”}}]}}}’
curl -XGET -H “Content-Type: application/json” ‘localhost:9200/app_search_ru/_count?pretty’ -d ‘{“query”: {“bool”: {“must_not”: [{“exists”: {“field”:“apk_name_lang”}}]}}}’
curl -XGET -H “Content-Type: application/json” ‘localhost:9200/app_search_ru/_count?pretty’ -d ‘{“query”: {“bool”: {“must_not”: [{“exists”: {“field”:“apk_id”}}]}}}’
3.1查询apk_name_lang的数据为空的数据的id信息,使用投影方式获取，fields
curl -XGET -H “Content-Type: application/json” ‘localhost:9200/app_search_ru/_search?pretty’ -d ‘{“query”: {“bool”: {“must_not”: [{“exists”: {“field”:“apk_name_lang”}}]}},“from”:0,“size”:100,"_source": [“seo_key”]}’
curl -XGET -H “Content-Type: application/json” ‘localhost:9200/app_search_ru/_search?pretty’ -d ‘{“query”: {“bool”: {“must_not”: [{“exists”: {“field”:“apk_name_lang”}}]}},“from”:0,“size”:200,"_source": [“apk_id”]}’
curl -XGET “Content-Type:application/json” ‘http://localhost:9200/app_search_ru/_doc/39c125b92e07db1b07b0926daa73ed7f?pretty’
3.2删除apk_id的数据为空的数据信息
curl -X POST “http://localhost:9200/app_search_ru/_delete_by_query” -H ‘Content-Type: application/json’ -d’{“query”: {“bool”:{“must_not”: {“exists”: {“field”:“apk_id”}}}}}’
4.不用语言下面的数据操作处理
curl -XGET -H “Content-Type: application/json” ‘localhost:9200/app_search_zh_cn/_count?pretty’ -d ‘{“query”: {“bool”: {“must_not”: [{“exists”: {“field”:“seo_key”}}]}}}’
curl -XGET -H “Content-Type: application/json” ‘localhost:9200/app_search_zh_cn/_count?pretty’ -d ‘{“query”: {“bool”: {“must_not”: [{“exists”: {“field”:“apk_name_lang”}}]}}}’
curl -XGET -H “Content-Type: application/json” ‘localhost:9200/app_search_zh_cn/_count?pretty’ -d ‘{“query”: {“bool”: {“must_not”: [{“exists”: {“field”:“apk_id”}}]}}}’
curl -X POST -H “Content-Type: application/json” localhost:9200/app_search_zh_cn/_update_by_query?pretty -d ‘{“script”: {“source”: “ctx._source.seo_key=ctx._source.apk_name_lang”},“query”: {“bool”: {“must_not”: [{“exists”: {“field”: “seo_key”}}]}}}’
app_search_en
curl -XGET -H “Content-Type: application/json” ‘localhost:9200/app_search_en/_count?pretty’ -d ‘{“query”: {“bool”: {“must_not”: [{“exists”: {“field”:“seo_key”}}]}}}’
curl -XGET -H “Content-Type: application/json” ‘localhost:9200/app_search_en/_count?pretty’ -d ‘{“query”: {“bool”: {“must_not”: [{“exists”: {“field”:“apk_name_lang”}}]}}}’
curl -XGET -H “Content-Type: application/json” ‘localhost:9200/app_search_en/_count?pretty’ -d ‘{“query”: {“bool”: {“must_not”: [{“exists”: {“field”:“apk_id”}}]}}}’
curl -X POST -H “Content-Type: application/json” localhost:9200/app_search_en/_update_by_query?pretty -d ‘{“script”: {“source”: “ctx._source.seo_key=ctx._source.apk_name_lang”},“query”: {“bool”: {“must_not”: [{“exists”: {“field”: “seo_key”}}]}}}’
app_search_uz
curl -X POST -H “Content-Type: application/json” localhost:9200/app_search_uz/_search?pretty -d ‘{“query”: {“bool”: {“must_not”: [{“exists”: {“field”:“seo_key”}}]}}}’
curl -XGET -H “Content-Type: application/json” ‘localhost:9200/app_search_uz/_count?pretty’ -d ‘{“query”: {“bool”: {“must_not”: [{“exists”: {“field”:“seo_key”}}]}}}’
curl -XGET -H “Content-Type: application/json” ‘localhost:9200/app_search_uz/_count?pretty’ -d ‘{“query”: {“bool”: {“must_not”: [{“exists”: {“field”:“apk_name_lang”}}]}}}’
curl -XGET -H “Content-Type: application/json” ‘localhost:9200/app_search_uz/_count?pretty’ -d ‘{“query”: {“bool”: {“must_not”: [{“exists”: {“field”:“apk_id”}}]}}}’
curl -X POST -H “Content-Type: application/json” localhost:9200/app_search_uz/_update_by_query?pretty -d ‘{“script”: {“source”: “ctx._source.seo_key=ctx._source.apk_name_lang”},“query”: {“bool”: {“must_not”: [{“exists”: {“field”: “seo_key”}}]}}}’
#根据id删除数据
curl -XDELETE localhost:9200/app_search_uz/_doc/f9ee0578fe1cc94de7482bd41accb329
#检查其他的数据信息
curl -XGET ‘localhost:9200/app_search_zh_cn/_count?pretty’
curl -XGET ‘localhost:9200/app_search_uk/_count?pretty’
curl -XGET ‘localhost:9200/app_search_ka/_count?pretty’
curl -XGET ‘localhost:9200/app_search_en/_count?pretty’
curl -XGET ‘localhost:9200/app_search_kk/_count?pretty’
curl -XGET ‘localhost:9200/app_search_uz/_count?pretty’
curl -XGET ‘localhost:9200/app_search_be/_count?pretty’
curl -XGET ‘localhost:9200/app_search_ru/_count?pretty’

#排除数据,关键字是puzzles数据没有被召回,查询关键字召回策略
curl -XPOST localhost:9200/app_search_ru/_search?pretty -H “Content-Type: application/json” -d’{“query”: {“bool”: {“must”: [{“multi_match”:{“query”:“puzzles”,“fields”:[“apk_name_lang.pinyin”,“new_label_name.pinyin”],“type”:“most_fields”}}]}},“from”: 0,“size”: 10,"_source": [“seo_key”,“package_name”]}’
curl -XPOST localhost:9200/app_search_ru/_search?pretty -H “Content-Type: application/json” -d’{“query”: {“bool”: {“should”: [{“match_phrase_prefix”: {“apk_name_lang”:{“query”:“puzzles”,“slop”: 1,“boost”: 10}}}, {“match”: {“apk_name_lang”:{ “query”:“puzzles”,“boost”:“9”} }}, {“match”: {“seo_key”: {“query”:“puzzles”,“boost”:“1”}}}]}},“from”: 0,“size”: 10,"_source": [“seo_key”,“package_name”]}’
curl -XPOST localhost:9200/app_search_ru/_search?pretty -H “Content-Type: application/json” -d’{“query”: {“bool”: {“should”: [{“match_phrase_prefix”: {“apk_name_lang”:{“query”:“puzzles”,“slop”: 0,“boost”: 10}}}, {“match”: {“apk_name_lang”:{ “query”:“puzzles”,“boost”:“9”} }}, {“match”: {“seo_key”: {“query”:“puzzles”,“boost”:“1”}}}]}},“from”: 0,“size”: 10,"_source": [“seo_key”,“package_name”]}’
#es查询package_mapping索引的关键字的top3数据
curl -H “Content-Type:application/json” -XGET localhost:9200/package_mapping/_search?pretty -d ‘{“query” : {“match” : {“key”:“puzzles”}},“from”: “0”,“size”: “100”}’
#查询效果
curl -XPOST localhost:9200/app_search_ru/_search?pretty -H “Content-Type: application/json” -d’{“query”: {“bool”: {“must”: [{“multi_match”:{“query”:“puzzles”,“fields”:[“apk_name_lang”,“new_label_name”],“type”:“most_fields”}}]}},“from”: 0,“size”: 10,"_source": [“seo_key”,“package_name”,“apk_name_lang”,“new_label_name”]}’
curl -XPOST localhost:9200/app_search_ru/_search?pretty -H “Content-Type: application/json” -d’{“query”: {“bool”: {“must”: [{“multi_match”:{“query”:“puzzles”,“fields”:[“apk_name_lang.pinyin”,“new_label_name.pinyin”],“type”:“most_fields”}}]}}}’