一:分行分片前的请求
1.Netty4HttpServerTransport
dispatchRequest:491, Netty4HttpServerTransport (org.elasticsearch.http.netty4)
2.RestController
dispatchRequest:174, RestController (org.elasticsearch.rest)
tryAllHandlers:337, RestController (org.elasticsearch.rest)
dispatchRequest:240, RestController (org.elasticsearch.rest)
3.BaseRestHandler
handleRequest:97, BaseRestHandler (org.elasticsearch.rest)
4.RestSearchAction
accept:-1, 1593864517 (org.elasticsearch.rest.action.search.RestSearchAction$$Lambda
1697
)
l
a
m
b
d
a
1697) lambda
1697)lambdaprepareRequest$2:93, RestSearchAction (org.elasticsearch.rest.action.search)
在RestSearchAction#prepareRequest方法中将请求体解析为SearchRequest 数据结构:
解析请求
public RestChannelConsumer prepareRequest(.. .) {
SearchRequest searchRequest = new SearchRequest();
request.withContentOrSourceParamParserOrNull (parser ->
parseSearchRequest (searchRequest, request, parser, setSize));
}
5.AbstractClient
search:463, AbstractClient (org.elasticsearch.client.support)
execute:336, AbstractClient (org.elasticsearch.client.support)
6.NodeClient
doExecute:72, NodeClient (org.elasticsearch.client.node)
executeLocally:83, NodeClient (org.elasticsearch.client.node)
7.TransportAction
execute:80, TransportAction (org.elasticsearch.action.support)
execute:141, TransportAction (org.elasticsearch.action.support)
proceed:178, TransportAction$RequestFilterChain (org.elasticsearch.action.support)
8.TransportSearchAction
doExecute:62, TransportSearchAction (org.elasticsearch.action.search)
doExecute:207, TransportSearchAction (org.elasticsearch.action.search)
9.Rewriteable
rewriteAndFetch:86, Rewriteable (org.elasticsearch.index.query)
rewriteAndFetch:113, Rewriteable (org.elasticsearch.index.query)
10.ActionListener
onResponse:59, ActionListener$1 (org.elasticsearch.action)
10.返回8.TransportSearchAction
accept:-1, 320835983 (org.elasticsearch.action.search.TransportSearchAction$$Lambda
1699
)
l
a
m
b
d
a
1699) lambda
1699)lambdadoExecute$4:187, TransportSearchAction (org.elasticsearch.action.search)
executeSearch:329, TransportSearchAction (org.elasticsearch.action.search)
构造目的shard列表
将请求涉及的本集群shard列表和远程集群的shard列表(远程集群用于跨集群访问)合并:
private void executeSearch(.. .) {
GroupShardsI terator<shardIterator> localShardsIterator =
clusterService.operationRouting().searchShards (clusterState,
concreteIndices, routingMap, searchRequest.preference (),
searchService.getResponseCollectorService(), nodeSearchCounts);
GroupShardsIterator <SearchShardIterator> shardIterators =
mergeShards Iterators (localShardsIterator, localIndices, remoteShardIterators);
}
二:query阶段:执行分片子查询TransportService#sendRequestInternal
1.AbstractSearchAsyncAction
start:115, AbstractSearchAsyncAction (org.elasticsearch.action.search)
executePhase:145, AbstractSearchAsyncAction (org.elasticsearch.action.search)
2.InitialSearchPhase
run:158, InitialSearchPhase (org.elasticsearch.action.search)
performPhaseOnShard:216, InitialSearchPhase (org.elasticsearch.action.search)
遍历所有shard发送请求
请求是基于shard遍历的,如果列表中有N个shard位于同一个节点,则向其发送N次请求,并不会把请求合并为一个。
public final void run() throws IOException {
if (shardsIts.size() > 0) {
//最大发分片请求数可以通过max_ concurrent_ shard_ requests参数配置(v6.0新增)
int maxConcurrentShardRequests = Math.min(this.maxConcurrentShardRequests, shardsIts.size());
for(int index = 0; index < maxConcurrentShardRequests; index++) {
final SearchShardIterator shardRoutings = shardsIts.get (index);
//执行shard级请求
performPhaseOnShard (index, shardRoutings, shardRoutings.nextOrNull());
}
}
}
3.SearchQueryThenFetchAsyncAction 转发分片
executePhaseOnShard:52, SearchQueryThenFetchAsyncAction (org.elasticsearch.action.search)
shardsIts为本次搜索涉及的所有分片,shardRoutings.nextOrNull()从某 个分片的所有副本中选择一个,例如,从website 中选择主分片。
转发请求同时定义一个Listener,用于处理Response:
private void performPhaseOnShard(. ..) {
executePhaseOnShard(.. .) {
//收到执行成功的回复
public void innerOnResponse (FirstResult result) {
maybeFork (thread, () -> onShardResult (result,shardIt) );
}
//收到执行失败的回复
public void onFailure (Exception t) {
maybeFork(thread, () -> onShardFailure (shardIndex, shard, shard. currentNodeId(),shardIt, t));
}
});
}
收集返回结果
本过程在search线程池中执行:
private void onShardResult (FirstResult result, SearchShardIterator shardIt) {
onShardSuccess(result);
success fulShardExecution(shardIt);
}
onShardSuccess对收集到的结果进行合并。successfulShardExecution方法检查是否所有请求都已收到回复,是否进入
private void successfulShardExecution (SearchShardIterator shardsIt) {
//计数器累加.
final int xTotalOps = totalOps.addAndGet (remainingOpsOnIterator);
//检查是否收到全部回复
if (xTotalOps == expectedTotalOps) {
onPhaseDone ();
} else if (xTota1Ops > expectedTotal0ps) {
throw new AssertionError(. ..);
}
}
4.SearchTransportService
sendExecuteQuery:151, SearchTransportService (org.elasticsearch.action.search)
5.TransportService
sendChildRequest:562, TransportService (org.elasticsearch.transport)
sendChildRequest:571, TransportService (org.elasticsearch.transport)
sendRequest:530, TransportService (org.elasticsearch.transport)
sendRequest:-1, 2065587797 (org.elasticsearch.transport.TransportService$$Lambda$1106)
sendRequestInternal:611, TransportService (org.elasticsearch.transport)
6.发送到本地
sendRequest:138, TransportService$3 (org.elasticsearch.transport)
access$000:68, TransportService (org.elasticsearch.transport)
发到节点上的分片
sendLocalRequest:673, TransportService (org.elasticsearch.transport)
7.RequestHandlerRegistry
processMessageReceived:66, RequestHandlerRegistry (org.elasticsearch.transport)
响应Query请求
以常见的Query请求为例,其action为:indices:data/read/search [phase/query]
主要过程就是执行查询,然后发送Response
transportService.registerRequestHandler (QUERY_ACTION_NAME, ShardSearchTransportRequest::new, ThreadPool.Names.SAME,
new TaskAwareTransportRequestHandler <ShardSearchTransportRequest>() {
//收到Query请求
public void messageReceived(. ..) throws Exception {
//执行查询
searchService.executeQueryPhase (request, (SearchTask) task, new ActionListener<SearchPhaseResult>() {
//处理查询成功的情况
public void onResponse (SearchPhaseResult searchPhaseResult) {
channel . sendResponse (searchPhaseResult) ;
}
//处理查询失败的情况
public void onFailure (Exception е) {
channel.sendResponse(e);
}
});
}
});
8.SearchTransportService
messageReceived:358, SearchTransportService$6 (org.elasticsearch.action.search)
taskManager.#register注册任务
messageReceived:364, SearchTransportService$6 (org.elasticsearch.action.search)
**9.SearchService **
executeQueryPhase:271, SearchService (org.elasticsearch.search)
rewriteShardRequest:990, SearchService (org.elasticsearch.search)
rewriteAndFetch:86, Rewriteable (org.elasticsearch.index.query)
10.Rewriteable
rewriteAndFetch:113, Rewriteable (org.elasticsearch.index.query)
rewriteResponse.onResponse(builder)
11 返回rewriteShardRequest的listener.onResponse
executeQueryPhase(request, task)
public void executeQueryPhase(ShardSearchRequest request, SearchTask task, ActionListener<SearchPhaseResult> listener) {
rewriteShardRequest(request, new ActionListener<ShardSearchRequest>() {
@Override
public void onResponse(ShardSearchRequest request) {
try {
if (!Objects.equals(request.shardId().getIndex().getName(), ".kibana")) {
System.out.println("监听search之后的结果SearchService#SearchService");
}
listener.onResponse(executeQueryPhase(request, task));
} catch (Exception e) {
onFailure(e);
}
}
@Override
public void onFailure(Exception e) {
listener.onFailure(e);
}
});
}
onResponse:271, SearchService$2 (org.elasticsearch.search)
onResponse:278, SearchService$2 (org.elasticsearch.search)
executeQueryPhase:308, SearchService (org.elasticsearch.search)
loadOrExecuteQueryPhase:266, SearchService (org.elasticsearch.search)
execute:101, QueryPhase (org.elasticsearch.search.query)
execute:129, QueryPhase (org.elasticsearch.search.query)
执行真正的lunence查询
Query query = searchContext.query();
static boolean execute(SearchContext searchContext, final IndexSearcher searcher,
Consumer<Runnable> checkCancellationSetter, @Nullable Sort indexSort) throws QueryPhaseExecutionException {
QuerySearchResult queryResult = searchContext.queryResult();
queryResult.searchTimedOut(false);
try {
queryResult.from(searchContext.from());// 设置分页
queryResult.size(searchContext.size());
Query query = searchContext.query();
assert query == searcher.rewrite(query); // already rewritten
}
if (shouldCollect) {
searcher.search(query, queryCollector);
}
}