想使用search_after 这个方法,搜索的数据必须是有唯一的键,用来排序,这里使用_id。
如下每次查询带上sort参数,然后将上次查询的最后一个数据的sort信息,添加到下次查询的search_after参数里
GET books/_search
{
"size": 10,
"query": {
"match_all" : {
}
},
"sort": [
{"_id": "asc"},
{"_score": "desc"},
{"title.keyword":"desc"}
]
}
GET books/_search
{
"size": 10,
"query": {
"match_all" : {
}
},
"search_after": [ "-55upIIB7UkVIfnN6-JF",
1.0,
"Learn Git in a Month of Lunches"],
"sort": [
{"_id": "asc"},
{"_score": "desc"},
{"title.keyword":"desc"}
]
}
elasticsearch_dsl 包
import time
from elasticsearch_dsl import connections
from elasticsearch_dsl import Search,Q
conn = connections.create_connection(hosts=['192.168.214.131'],port=9200, http_auth="elastic:ellischen")
search = Search(using= conn,index='books')
querys={}
querys['must'] = [Q('regexp', **{'logical_address.keyword': {"value": "{}(.)*".format('2')}})]
search.query = Q({"match_all":{}})
res = search.count()
hits_source=[]
start=time.time()
search_data = search.extra(size=1000).sort('-_id').execute()
while len(search_data['hits']['hits']):
hits_source.extend(search_data['hits']['hits'])
search_data = search.extra(size=1000).extra(search_after=search_data['hits']['hits'][-1]['sort']).sort('-_id').execute()
print(time.time()-start)
print(len(hits_source))