solr源码分析—创建core
假设客户端发送如下请求,
http://127.0.0.1/solr-6.1.0/admin/cores?_=1475575790875&action=CREATE&config=solrconfig.xml&dataDir=data&instanceDir=test&name=test&schema=schema.xml&wt=json
根据上一章的分析,该请求到达客户端后,会根据路径admin调用handleAdminRequest函数处理,进而调用CoreAdminOperation的call函数,代码如下,
SolrDispatchFilter::doFilter->HttpSolrCall::call->handleAdminRequest->CoreAdminHandler::handleRequest->handleRequestBody->CallInfo::call->CoreAdminOperation::call
public void call(CallInfo callInfo) {
SolrParams params = callInfo.req.getParams();
String coreName = params.required().get(CoreAdminParams.NAME);
Map<String, String> coreParams = buildCoreParams(params);
CoreContainer coreContainer = callInfo.handler.coreContainer;
Path instancePath = coreContainer.getCoreRootDirectory().resolve(coreName);
coreContainer.create(coreName, instancePath, coreParams);
callInfo.rsp.add("core", coreName);
}
CoreAdminOperation的call函数主要调用了CoreContainer的create函数创建对应的core,coreName为core的名称,instancePath为core要安装的目录,coreParams为经过处理的请求参数。
SolrDispatchFilter::doFilter->HttpSolrCall::call->handleAdminRequest->CoreAdminHandler::handleRequest->handleRequestBody->CallInfo::call->CoreAdminOperation::call->CoreContainer::create
public SolrCore create(String coreName, Path instancePath, Map<String, String> parameters) {
CoreDescriptor cd = new CoreDescriptor(this, coreName, instancePath, parameters);
SolrCore core = create(cd, true);
coresLocator.create(this, cd);
return core;
}
首先创建CoreDescriptor,其构造函数会读取solr的配置信息,然后调用create函数创建对应的SolrCore,最后通过CorePropertiesLocator的create函数向core.properties文件写入配置信息。
SolrDispatchFilter::doFilter->HttpSolrCall::call->handleAdminRequest->CoreAdminHandler::handleRequest->handleRequestBody->CallInfo::call->CoreAdminOperation::call->CoreContainer::create->create
private SolrCore create(CoreDescriptor dcore, boolean publishState) {
SolrCore core = null;
ConfigSet coreConfig = coreConfigService.getConfig(dcore);
core = new SolrCore(dcore, coreConfig);
...
return core;
}
getConfig读取并解析solrconfig.xml配置文件,然后创建SolrCore读取其余配置文件。
SolrDispatchFilter::doFilter->HttpSolrCall::call->handleAdminRequest->CoreAdminHandler::handleRequest->handleRequestBody->CallInfo::call->CoreAdminOperation::call->CoreContainer::create->create->ConfigSetService::getConfig
public final ConfigSet getConfig(CoreDescriptor dcore) {
SolrResourceLoader coreLoader = createCoreResourceLoader(dcore);
SolrConfig solrConfig = createSolrConfig(dcore, coreLoader);
IndexSchema schema = createIndexSchema(dcore, solrConfig);
NamedList properties = createConfigSetProperties(dcore, coreLoader);
return new ConfigSet(configName(dcore), solrConfig, schema, properties);
}
createCoreResourceLoader创建并返回一个SolrResourceLoader,createSolrConfig函数创建SolrConfig用于保存solrconfig.xml配置信息,createIndexSchema函数创建索引相关的ManagedIndexSchemaFactory,并加载managed-schema配置文件,createConfigSetProperties函数用来加载ConfigSet属性,加载configsetprops.json文件,最后创建ConfigSet封装前面的所有配置信息。
SolrDispatchFilter::doFilter->HttpSolrCall::call->handleAdminRequest->CoreAdminHandler::handleRequest->handleRequestBody->CallInfo::call->CoreAdminOperation::call->CoreContainer::create->create->ConfigSetService::getConfig->createSolrConfig
protected SolrConfig createSolrConfig(CoreDescriptor cd, SolrResourceLoader loader) {
return SolrConfig.readFromResourceLoader(loader, cd.getConfigName());
}
public static SolrConfig readFromResourceLoader(SolrResourceLoader loader, String name) {
return new SolrConfig(loader, name, null);
}
public SolrConfig(SolrResourceLoader loader, String name, InputSource is)
throws ParserConfigurationException, IOException, SAXException {
super(loader, name, is, "/config/");
initLibs();
luceneMatchVersion = getLuceneVersion("luceneMatchVersion");
String indexConfigPrefix = "indexConfig";
indexConfig = new SolrIndexConfig(this, "indexConfig", null);
...
solrRequestParsers = new SolrRequestParsers(this);
}
createSolrConfig函数首先通过其父类Config的构造函数解析solrhome下conf目录中的solrconfig.xml文件,initLibs函数用来加载solr项目下contrib和dist目录中的jar文件,getLuceneVersion函数获得lucene的版本,然后创建SolrIndexConfig封装索引方面的配置,最后SolrRequestParsers的构造函数创建各个解析器。
SolrDispatchFilter::doFilter->HttpSolrCall::call->handleAdminRequest->CoreAdminHandler::handleRequest->handleRequestBody->CallInfo::call->CoreAdminOperation::call->CoreContainer::create->create->ConfigSetService::getConfig->createSolrConfig->SolrConfig::readFromResourceLoader->Config::Config
public Config(SolrResourceLoader loader, String name, InputSource is, String prefix, boolean substituteProps) throws ParserConfigurationException, IOException, SAXException
{
this.loader = loader;
this.name = name;
this.prefix = (prefix != null && !prefix.endsWith("/"))? prefix + '/' : prefix;
javax.xml.parsers.DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
InputStream in = loader.openConfig(name);
is = new InputSource(in);
is.setSystemId(SystemIdResolver.createSystemIdFromResourceName(name));
final DocumentBuilder db = dbf.newDocumentBuilder();
db.setEntityResolver(new SystemIdResolver(loader));
db.setErrorHandler(xmllog);
doc = db.parse(is);
origDoc = copyDoc(doc);
DOMUtil.substituteProperties(doc, getSubstituteProperties());
}
openConfig函数获得solrconfig.xml的输入流,然后解析solrconfig.xml文件,并将配置的键值对设置进coreProperties中。
SolrDispatchFilter::doFilter->HttpSolrCall::call->handleAdminRequest->CoreAdminHandler::handleRequest->handleRequestBody->CallInfo::call->CoreAdminOperation::call->CoreContainer::create->create->SolrCore::SolrCore
public SolrCore(CoreDescriptor cd, ConfigSet coreConfig) {
this(cd.getName(), null, coreConfig.getSolrConfig(), coreConfig.getIndexSchema(), coreConfig.getProperties(),
cd, null, null, null);
}
public SolrCore(String name, String dataDir, SolrConfig config,
IndexSchema schema, NamedList configSetProperties,
CoreDescriptor coreDescriptor, UpdateHandler updateHandler,
IndexDeletionPolicyWrapper delPolicy, SolrCore prev) {
this.coreDescriptor = coreDescriptor;
setName(name);
resourceLoader = config.getResourceLoader();
this.solrConfig = config;
this.configSetProperties = configSetProperties;
directoryFactory = initDirectoryFactory();
solrCoreState = new DefaultSolrCoreState(directoryFactory);
this.dataDir = initDataDir(dataDir, config, coreDescriptor);
this.ulogDir = initUpdateLogDir(coreDescriptor);
...
initIndex(prev != null);
reqHandlers = new RequestHandlers(this);
reqHandlers.initHandlersFromConfig(solrConfig);
initSearcher(prev);
...
}
initDirectoryFactory创建并返回NRTCachingDirectoryFactory,initDataDir和initUpdateLogDir函数获得data目录的绝对路径,initIndex函数创建data目录,并初始化。initWriters初始化创建RequestHandlers,其initHandlersFromConfig函数会绑定各个请求路径到具体的Handler中。initSearcher内部会创建lucene的IndexSearcher。
SolrDispatchFilter::doFilter->HttpSolrCall::call->handleAdminRequest->CoreAdminHandler::handleRequest->handleRequestBody->CallInfo::call->CoreAdminOperation::call->CoreContainer::create->create->SolrCore::SolrCore->initIndex
void initIndex(boolean reload) throws IOException {
String indexDir = getNewIndexDir();
boolean indexExists = getDirectoryFactory().exists(indexDir);
...
SolrIndexWriter writer = SolrIndexWriter.create(this, "SolrCore.initIndex", indexDir, getDirectoryFactory(), true, getLatestSchema(), solrConfig.indexConfig, solrDelPolicy, codec);
writer.close();
cleanupOldIndexDirectories();
}
getNewIndexDir函数中会创建data目录。SolrIndexWriter的create函数在该目录下创建index目录并初始化lucene。
SolrDispatchFilter::doFilter->HttpSolrCall::call->handleAdminRequest->CoreAdminHandler::handleRequest->handleRequestBody->CallInfo::call->CoreAdminOperation::call->CoreContainer::create->create->SolrCore::SolrCore->initIndex->SolrIndexWriter::create
public static SolrIndexWriter create(SolrCore core, String name, String path, DirectoryFactory directoryFactory, boolean create, IndexSchema schema, SolrIndexConfig config, IndexDeletionPolicy delPolicy, Codec codec) throws IOException {
final Directory d = directoryFactory.get(path, DirContext.DEFAULT, config.lockType);
SolrIndexWriter w = new SolrIndexWriter(core, name, path, d, create, schema,
config, delPolicy, codec);
w.setDirectoryFactory(directoryFactory);
return w;
}
directoryFactory的get函数在data目录下创建index目录。SolrIndexWriter的构造函数建立索引,SolrIndexWriter继承自lucene的IndexWriter,因此会调用IndexWriter的构造函数初始化lucene索引。
SolrDispatchFilter::doFilter->HttpSolrCall::call->handleAdminRequest->CoreAdminHandler::handleRequest->handleRequestBody->CallInfo::call->CoreAdminOperation::call->CoreContainer::create->CorePropertiesLocator::create
public void create(CoreContainer cc, CoreDescriptor... coreDescriptors) {
for (CoreDescriptor cd : coreDescriptors) {
Path propertiesFile = this.rootDirectory.resolve(cd.getInstanceDir()).resolve(PROPERTIES_FILENAME);
writePropertiesFile(cd, propertiesFile);
}
}
propertiesFile为core下的core.properties文件,该函数向该文件写入配置信息。
SolrDispatchFilter::doFilter->HttpSolrCall::call->handleAdminRequest->CoreAdminHandler::handleRequest->handleRequestBody->CallInfo::call->CoreAdminOperation::call->CoreContainer::create->CorePropertiesLocator::create->writePropertiesFile
private void writePropertiesFile(CoreDescriptor cd, Path propfile) {
Properties p = buildCoreProperties(cd);
Files.createDirectories(propfile.getParent());
try (Writer os = new OutputStreamWriter(Files.newOutputStream(propfile), StandardCharsets.UTF_8)) {
p.store(os, "Written by CorePropertiesLocator");
}
}
buildCoreProperties函数将CoreDescriptor中的属性添加到Properties中,该函数创建core.properties文件的输出流,并将属性写入该文件中。