(四)PrestoDB源码解析(二)
执行计划模块
概要
- PrestoDB对传入的SQL语句进行解析后将生成执行计划,本模块将解析PrestoDB如何生成执行计划。
- (a). 用户输入SQL语句,通过JDBC或者presto-cli客户端将SQL通过HTTP的形式传入Coordinator中的StatementResource接口的createQuery方法中,进入执行计划生成模块;
(b).通过SQLQueryManager类的createQuery方法来实现完整的词法语法分析,通过sqlParser.createStatement方法来实现分析的,词法分析主要的作用是进行一些sql语句大小写,处理函数等词法解析,语法分析主要是形成一个该语句的所有方法的包装对象;
( c).通过Statement类型生成对应的工厂,在工厂中通过queryExecutionFactory.createQueryExecution方法中SqlQueryExecution类下的analyzer.analyze进行语义分析,语义分析的作用是得到该sql语句的一系列字段信息;
(d).在createQueryExecution方法中SqlQueryExecution类下的logicalPlanner.plan(analysis)根据不同的类型进行执行计划的生成,并对执行计划进行优化和分段
- 整体流程
源码
- 进入语法分析主入口:SqlQueryManager类(语法分析在此类中进行)
package com.facebook.presto.execution;
import com.facebook.presto.ExceededCpuLimitException;
import com.facebook.presto.Session;
.............
//语法分析入口
@ThreadSafe
public class SqlQueryManager
implements QueryManager
{
private static final Logger log = Logger.get(SqlQueryManager.class);
private final QueryPreparer queryPreparer;
private final EmbedVersion embedVersion;
private final ExecutorService queryExecutor;
private final ThreadPoolExecutorMBean queryExecutorMBean;
private final ResourceGroupManager<?> resourceGroupManager;
private final ClusterMemoryManager memoryManager;
...............
- 进入语法分析的主要方法:SqlQueryManager类的createQuery方法
//分析词法和语法的方法
@Override
public ListenableFuture<?> createQuery(QueryId queryId, SessionContext sessionContext, String query)
{
QueryCreationFuture queryCreationFuture = new QueryCreationFuture();
queryExecutor.submit(embedVersion.embedVersion(() -> {
try {
//进行词法和语法分析
createQueryInternal(queryId, sessionContext, query, resourceGroupManager);
queryCreationFuture.set(null);
}
catch (Throwable e) {
queryCreationFuture.setException(e);
}
}));
return queryCreationFuture;
}
- 进入语法词法分析具体方法:createQueryInternal
private <C> void createQueryInternal(QueryId queryId, SessionContext sessionContext, String query, ResourceGroupManager<C> resourceGroupManager)
{
requireNonNull(queryId, "queryId is null");
requireNonNull(sessionContext, "sessionFactory is null");
requireNonNull(query, "query is null");
checkArgument(!query.isEmpty(), "query must not be empty string");
checkArgument(!queryTracker.tryGetQuery(queryId).isPresent(), "query %s already exists", queryId);
Session session = null;
SelectionContext<C> selectionContext = null;
QueryExecution queryExecution;
PreparedQuery preparedQuery;
Optional<QueryType> queryType = Optional.empty();
try {
clusterSizeMonitor.verifyInitialMinimumWorkersRequirement();
if (query.length() > maxQueryLength) {
int queryLength = query.length();
query = query.substring(0, maxQueryLength);
throw new PrestoException(QUERY_TEXT_TOO_LARGE, format("Query text length (%s) exceeds the maximum length (%s)", queryLength, maxQueryLength));
}
// decode session
session = sessionSupplier.createSession(queryId, sessionContext);
WarningCollector warningCollector = warningCollectorFactory.create();
// prepare query
//分析词法和语法的方法
preparedQuery = queryPreparer.prepareQuery(session, query, warningCollector);
// select resource group