druid查询源码追踪

待整理和完善
org.apache.druid.cli.Main --> CliBroker getModules()--> BrokerQueryResource.getQueryTargets()
 --> 1.List<LocatedSegmentDescriptor> ServerViewUtil.getTargetLocations(BrokerServerView,datasource,interval,...) 获取segmentLocation,   2.ResourceIOReaderWriter.ok()读取segment

LocatedSegmentDescriptor信息包括：
	segmentMetadata:segment元信息(interval,version,partition,size)
	serverMetadata:实时节点和历史节点地址等。

读取segment
-->Response.ok(newOutputWriter(null, null, false).writeValueAsString(object), contentType) -->


Jetty server（Jetty+Jersey+guice组合）
Broker初始化：
1.CliBroker绑定了 BrokerServerView，CachingClusteredClient，ClientQuerySegmentWalker，TimelineServerView，BrokerQueryResource等
查询入口doPost
QueryResource.doPost()-->
1.根据请求创建对应Query对象，
2.执行查询 final QueryLifecycle.QueryResponse queryResponse = queryLifecycle.execute();

执行查询，返回sequence结果
final Sequence res = QueryPlus.wrap(baseQuery).withIdentity(authenticationResult.getIdentity()).run(texasRanger, responseContext); -->  QueryRunner.run()
创建QueryRunner，执行QueryRunner.run();
说明：
   1.ClientQuerySegmentWalker texasRanger：创建QueryRunner
   2.ResponseContext responseContext：
	   The context for storing and passing data between chains of {@link org.apache.druid.query.QueryRunner}s.
	   The context is also transferred between Druid nodes with all the data it contains.
   3. QueryPlus 包含Query,QueryMetries,Identity  
   4. ClientQuerySegmentWalker创建QueryRunner
    一种根据Query和interval创建对应的QueryRunner
	一种根据SegmentDescriptor创建对应的QueryRunner

ClientQuerySegmentWalker：根据查询中的intervals或segments参数来确定调用getQueryRunnerForIntervals还是getQueryRunnerForSegments
public <T> QueryRunner<T> getQueryRunnerForIntervals(Query<T> query, Iterable<Interval> intervals)
  {
    return makeRunner(query, baseClient.getQueryRunnerForIntervals(query, intervals));
  }
  
makeRunner：返回ResultLevelCachingQueryRunner，ResultLevelCachingQueryRunner装饰FluentQueryRunner，FluentQueryRunner装饰SetAndVerifyContextQueryRunner，SetAndVerifyContextQueryRunner装饰了RetryQueryRunner，RetryQueryRunner装饰baseRunner

getQueryRunnerForIntervals():返回匿名baseRunner,内部run调用 CachingClusteredClient.run --》CachingClusteredClient内部类SpecificQueryRunnable.run

SpecificQueryRunnable.run: 
Sequence<T> run(final UnaryOperator<TimelineLookup<String, ServerSelector>> timelineConverter)
    {
	//根据datasource获取timeline
      @Nullable
      TimelineLookup<String, ServerSelector> timeline = serverView.getTimeline(query.getDataSource());
      if (timeline == null) {
        return Sequences.empty();
      }
      timeline = timelineConverter.apply(timeline);
      if (uncoveredIntervalsLimit > 0) {
        computeUncoveredIntervals(timeline);
      }
//根据timeline获取ServerSelector与对应SegmentDescriptor
      final Set<ServerToSegment> segments = computeSegmentsToQuery(timeline);
	  //根据query获取缓存key
      @Nullable
      final byte[] queryCacheKey = computeQueryCacheKey();
      if (query.getContext().get(QueryResource.HEADER_IF_NONE_MATCH) != null) {
        @Nullable
        final String prevEtag = (String) query.getContext().get(QueryResource.HEADER_IF_NONE_MATCH);
        @Nullable
        final String currentEtag = computeCurrentEtag(segments, queryCacheKey);
        if (currentEtag != null && currentEtag.equals(prevEtag)) {
          return Sequences.empty();
        }
      }
//根据缓存key获取对应的缓存结果
      final List<Pair<Interval, byte[]>> alreadyCachedResults = pruneSegmentsWithCachedResults(queryCacheKey, segments);
	  //获取Server以及所拥有的 SegmentDescriptor集合
      final SortedMap<DruidServer, List<SegmentDescriptor>> segmentsByServer = groupSegmentsByServer(segments);
      return new LazySequence<>(() -> {
        List<Sequence<T>> sequencesByInterval = new ArrayList<>(alreadyCachedResults.size() + segmentsByServer.size());
		//从缓存中拉取segment并反序列化成 sequences
        addSequencesFromCache(sequencesByInterval, alreadyCachedResults);
		// 从Server端拉取segment并反序列化成 sequences
        addSequencesFromServer(sequencesByInterval, segmentsByServer);
		// 合并sequences并返回结果sequence
        return merge(sequencesByInterval);
      });
    }

//broker端异步发起多查询请求，并缓存返回结果
private void addSequencesFromServer(
        final List<Sequence<T>> listOfSequences,
        final SortedMap<DruidServer, List<SegmentDescriptor>> segmentsByServer
    )
    {
	//遍历segmentsByServer，使用NettyHttpClient 发起查询请求，并将每个查询请求返回的sequence添加到集合中
      segmentsByServer.forEach((server, segmentsOfServer) -> {
	     //从 BrokerServerView 获取 QueryRunner
        final QueryRunner serverRunner = serverView.getQueryRunner(server);

        if (serverRunner == null) {
          log.error("Server[%s] doesn't have a query runner", server);
          return;
        }

        final MultipleSpecificSegmentSpec segmentsOfServerSpec = new MultipleSpecificSegmentSpec(segmentsOfServer);

        // Divide user-provided maxQueuedBytes by the number of servers, and limit each server to that much.
        final long maxQueuedBytes = QueryContexts.getMaxQueuedBytes(query, httpClientConfig.getMaxQueuedBytes());
        final long maxQueuedBytesPerServer = maxQueuedBytes / segmentsByServer.size();
        final Sequence<T> serverResults;

        //query中的context属性中获取bySegment，默认false
        if (isBySegment) {
          serverResults = getBySegmentServerResults(serverRunner, segmentsOfServerSpec, maxQueuedBytesPerServer);
        } else if (!server.segmentReplicatable() || !populateCache) {
		//直接查询，不缓存
          serverResults = getSimpleServerResults(serverRunner, segmentsOfServerSpec, maxQueuedBytesPerServer);
        } else {
		//查询并缓存
          serverResults = getAndCacheServerResults(serverRunner, segmentsOfServerSpec, maxQueuedBytesPerServer);
        }
        listOfSequences.add(serverResults);
      });
    }
	
	//直接查询，不缓存，调用从 BrokerServerView 获取的 QueryRunner.run方法
	private Sequence<T> getSimpleServerResults(
        final QueryRunner serverRunner,
        final MultipleSpecificSegmentSpec segmentsOfServerSpec,
        long maxQueuedBytesPerServer
    )
    {
      return serverRunner.run(
	  //统一发起segments查询方式，server端均调用getQueryRunnerForSegments执行查询请求
          queryPlus.withQuerySegmentSpec(segmentsOfServerSpec).withMaxQueuedBytes(maxQueuedBytesPerServer),
          responseContext
      );
    }


BrokerServerView 中的QueryRunner 为子类 DirectDruidClient.run,其中通过 NettyHttpClient.go 异步发送http请求到对应server执行查询请求，query中携带要查询的segments描述列表MultipleSpecificSegmentSpec
发送查询请求，
DirectDruidClient.run:
     ....
	 //httpClient 为NettyHttpClient
      future = httpClient.go(
          new Request(
              HttpMethod.POST,
              new URL(url)
          ).setContent(objectMapper.writeValueAsBytes(QueryContexts.withTimeout(query, timeLeft)))
           .setHeader(
               HttpHeaders.Names.CONTENT_TYPE,
               isSmile ? SmileMediaTypes.APPLICATION_JACKSON_SMILE : MediaType.APPLICATION_JSON
           ),
          responseHandler,
          Duration.millis(timeLeft)
      );

      queryWatcher.registerQuery(query, future);

      openConnections.getAndIncrement();
      Futures.addCallback(
          future,
          new FutureCallback<InputStream>()
          {
            @Override
            public void onSuccess(InputStream result)
            {
              openConnections.getAndDecrement();
            }

            @Override
            public void onFailure(Throwable t)
            {
              openConnections.getAndDecrement();
              if (future.isCancelled()) {
                cancelQuery(query, cancelUrl);
              }
            }
          },
          // The callback is non-blocking and quick, so it's OK to schedule it using directExecutor()
          Execs.directExecutor()
      );
     ....

CliPeon:
初始时绑定关系：QuerySegmentWalker为SingleTaskBackgroundRunner
执行的是 SingleTaskBackgroundRunner.getQueryRunnerForSegments 获取QueryRunner --> RealtimeIndexTask.getQueryRunner-->RealtimePlumber.getQueryRunner
--> SinkQuerySegmentWalker.getQueryRunnerForIntervals
   追加模式任务 SinkQuerySegmentWalker.getQueryRunnerForSegments:
   final Sink theSink = chunk.getObject();
          final SegmentId sinkSegmentId = theSink.getSegment().getId();

          Iterable<QueryRunner<T>> perHydrantRunners = new SinkQueryRunners<>(
              Iterables.transform(
                  theSink,
                  hydrant -> {
                    // Hydrant might swap at any point, but if it's swapped at the start
                    // then we know it's *definitely* swapped.
                    final boolean hydrantDefinitelySwapped = hydrant.hasSwapped();

                    if (skipIncrementalSegment && !hydrantDefinitelySwapped) {
                      return new Pair<>(Intervals.ETERNITY, new NoopQueryRunner<>());
                    }

                    // Prevent the underlying segment from swapping when its being iterated
					//其中segment为 IncrementalIndexSegment，包含 IncrementalIndex 对象，该对象为拥有实时摄取的数据
                    final Pair<Segment, Closeable> segmentAndCloseable = hydrant.getAndIncrementSegment();
                    try {
					//根据segment获取QueryRunner对象，factory:TopNQueryRunnerFactory，返回匿名QueryRunner，内部run调用 TopNQueryEngine.run
                      QueryRunner<T> runner = factory.createRunner(segmentAndCloseable.lhs);

                      // 1) Only use caching if data is immutable
                      // 2) Hydrants are not the same between replicas, make sure cache is local
                      if (hydrantDefinitelySwapped && cache.isLocal()) {
                        runner = new CachingQueryRunner<>(
                            makeHydrantCacheIdentifier(hydrant),
                            descriptor,
                            objectMapper,
                            cache,
                            toolChest,
                            runner,
                            // Always populate in foreground regardless of config
                            new ForegroundCachePopulator(
                                objectMapper,
                                cachePopulatorStats,
                                cacheConfig.getMaxEntrySize()
                            ),
                            cacheConfig
                        );
                      }
                      // Make it always use Closeable to decrement()
                      runner = QueryRunnerHelper.makeClosingQueryRunner(
                          runner,
                          segmentAndCloseable.rhs
                      );
                      return new Pair<>(segmentAndCloseable.lhs.getDataInterval(), runner);
                    }
                    catch (RuntimeException e) {
                      CloseQuietly.close(segmentAndCloseable.rhs);
                      throw e;
                    }
                  }
              )
          );
    
 实时Peon执行查询：	
	QueryRunner.run-->TopNQueryEngine.query:
	public Sequence<Result<TopNResultValue>> query(
      final TopNQuery query,
      final StorageAdapter adapter,
      final @Nullable TopNQueryMetrics queryMetrics
  )
  {
    if (adapter == null) {
      throw new SegmentMissingException(
          "Null storage adapter found. Probably trying to issue a query against a segment being memory unmapped."
      );
    }

    final List<Interval> queryIntervals = query.getQuerySegmentSpec().getIntervals();
    final Filter filter = Filters.convertToCNFFromQueryContext(query, Filters.toFilter(query.getDimensionsFilter()));
    final Granularity granularity = query.getGranularity();
	//核心topN查询算法，adapter为 IncrementalIndexStorageAdapter
    final TopNMapFn mapFn = getMapFn(query, adapter, queryMetrics);

    Preconditions.checkArgument(
        queryIntervals.size() == 1, "Can only handle a single interval, got[%s]", queryIntervals
    );

    return Sequences.filter(
        Sequences.map(
            adapter.makeCursors(
                filter,
                queryIntervals.get(0),
                query.getVirtualColumns(),
                granularity,
                query.isDescending(),
                queryMetrics
            ),
            input -> {
              if (queryMetrics != null) {
                queryMetrics.cursor(input);
              }
              return mapFn.apply(input, queryMetrics);
            }
        ),
        Predicates.notNull()
    );
  }
  
  TopNMapFn.apply:
  public Result<TopNResultValue> apply(final Cursor cursor, final @Nullable TopNQueryMetrics queryMetrics)
  {
    final ColumnSelectorPlus<TopNColumnAggregatesProcessor<?>> selectorPlus =
        DimensionHandlerUtils.createColumnSelectorPlus(
            new TopNColumnAggregatesProcessorFactory(query.getDimensionSpec().getOutputType()),
            query.getDimensionSpec(),
            cursor.getColumnSelectorFactory()
        );

    if (selectorPlus.getSelector() == null) {
      return null;
    }

    TopNParams params = null;
    try {
      params = topNAlgorithm.makeInitParams(selectorPlus, cursor);
      if (queryMetrics != null) {
        queryMetrics.columnValueSelector(selectorPlus.getSelector());
        queryMetrics.numValuesPerPass(params);
      }


      TopNResultBuilder resultBuilder = BaseTopNAlgorithm.makeResultBuilder(params, query);

      topNAlgorithm.run(params, resultBuilder, null, queryMetrics);

      return resultBuilder.build();
    }
    finally {
      topNAlgorithm.cleanup(params);
    }
  }
  
  
  BaseTopNAlgorithm.run--》runWithCardinalityUnknown()-->HeapBasedTopNAlgorithm.scanAndAggregate(),updateResults()
  
  NullableNumericTopNColumnAggregatesProcessor.scanAndAggregate() updateResults()或
  StringTopNColumnAggregatesProcessor.scanAndAggregateWithCardinalityUnknown():
  private long scanAndAggregateWithCardinalityUnknown(
      TopNQuery query,
      Cursor cursor,
      DimensionSelector selector
  )
  {
    long processedRows = 0;
	//cursor游标IncrementalIndexStorageAdapter.IncrementalIndexCursor，控制扫描分组条数，实际调用RollupFactsHolder.timeRangeIterable(startTime,endTime)返回的迭代器
    while (!cursor.isDone()) {
	//IndexerDimensionSelector.getRow，维度值选择器返回维度值下标
      final IndexedInts dimValues = selector.getRow();
      for (int i = 0, size = dimValues.size(); i < size; ++i) {//可能是多值维度
        final int dimIndex = dimValues.get(i);
		//根据维度值下标查找到原维度值(Object类型)，调用apply转成对应类型String或Long...
        final Comparable<?> key = dimensionValueConverter.apply(selector.lookupName(dimIndex));
        Aggregator[] aggs = aggregatesStore.computeIfAbsent(
            key,
            k -> BaseTopNAlgorithm.makeAggregators(cursor, query.getAggregatorSpecs())
        );
        for (Aggregator aggregator : aggs) {
          aggregator.aggregate();
        }
      }
      cursor.advance();
      processedRows++;
    }
    return processedRows;
  }
  
  
  updateResults：
  //TopNNumericResultBuilder
  public void updateResults(TopNResultBuilder resultBuilder)
  {
    for (Map.Entry<?, Aggregator[]> entry : aggregatesStore.entrySet()) {
      Aggregator[] aggs = entry.getValue();
      if (aggs != null) {
        Object[] vals = new Object[aggs.length];
        for (int i = 0; i < aggs.length; i++) {
          vals[i] = aggs[i].get();
        }

        final Comparable<?> key = dimensionValueConverter.apply(entry.getKey());
		//DimValHolder添加到 PriorityQueue 按排序TopN指标大小排序优先级队列 threshold 限制队列长度，添加至队列时完成了对指标的排序
        resultBuilder.addEntry(key, key, vals);
      }
    }
  }
  

CliHistorical:
绑定关系：QuerySegmentWalker为ServerManager
--> buildAndDecorateQueryRunner:
private <T> QueryRunner<T> buildAndDecorateQueryRunner(
      final QueryRunnerFactory<T, Query<T>> factory,
      final QueryToolChest<T, Query<T>> toolChest,
      final ReferenceCountingSegment adapter,
      final SegmentDescriptor segmentDescriptor,
      final AtomicLong cpuTimeAccumulator
  )
  {
    SpecificSegmentSpec segmentSpec = new SpecificSegmentSpec(segmentDescriptor);
    SegmentId segmentId = adapter.getId();
    String segmentIdString = segmentId.toString();

    MetricsEmittingQueryRunner<T> metricsEmittingQueryRunnerInner = new MetricsEmittingQueryRunner<>(
        emitter,
        toolChest,
        new ReferenceCountingSegmentQueryRunner<>(factory, adapter, segmentDescriptor),
        QueryMetrics::reportSegmentTime,
        queryMetrics -> queryMetrics.segment(segmentIdString)
    );

    CachingQueryRunner<T> cachingQueryRunner = new CachingQueryRunner<>(
        segmentIdString,
        segmentDescriptor,
        objectMapper,
        cache,
        toolChest,
        metricsEmittingQueryRunnerInner,
        cachePopulator,
        cacheConfig
    );

    BySegmentQueryRunner<T> bySegmentQueryRunner = new BySegmentQueryRunner<>(
        segmentId,
        adapter.getDataInterval().getStart(),
        cachingQueryRunner
    );

    MetricsEmittingQueryRunner<T> metricsEmittingQueryRunnerOuter = new MetricsEmittingQueryRunner<>(
        emitter,
        toolChest,
        bySegmentQueryRunner,
        QueryMetrics::reportSegmentAndCacheTime,
        queryMetrics -> queryMetrics.segment(segmentIdString)
    ).withWaitMeasuredFromNow();

    SpecificSegmentQueryRunner<T> specificSegmentQueryRunner = new SpecificSegmentQueryRunner<>(
        metricsEmittingQueryRunnerOuter,
        segmentSpec
    );

    PerSegmentOptimizingQueryRunner<T> perSegmentOptimizingQueryRunner = new PerSegmentOptimizingQueryRunner<>(
        specificSegmentQueryRunner,
        new PerSegmentQueryOptimizationContext(segmentDescriptor)
    );

    return new SetAndVerifyContextQueryRunner<>(
        serverConfig,
        CPUTimeMetricQueryRunner.safeBuild(
            perSegmentOptimizingQueryRunner,
            toolChest,
            emitter,
            cpuTimeAccumulator,
            false
        )
    );
  }

-->TopNQueryRunnerFactory.createRunner:


-->创建列值选择器：QueryableIndexColumnSelectorFactory
 public ColumnValueSelector<?> makeColumnValueSelector(String columnName)
  {
    Function<String, ColumnValueSelector<?>> mappingFunction = name -> {
      if (virtualColumns.exists(columnName)) {
        ColumnValueSelector<?> selector = virtualColumns.makeColumnValueSelector(columnName, index, offset);
        if (selector == null) {
          return virtualColumns.makeColumnValueSelector(columnName, this);
        } else {
          return selector;
        }
      }

      BaseColumn column = getCachedColumn(columnName, BaseColumn.class);

      if (column != null) {
        return column.makeColumnValueSelector(offset);
      } else {
        return NilColumnValueSelector.instance();
      }
    };

参考资料：
https://www.jianshu.com/p/f8de4531c30c
https://blog.csdn.net/weixin_38736107/article/details/98482951