前面的两篇介绍了安装katta及ZooKeeper,后边来介绍katta的Node.
我们回到step1 后边提到的solr-katta-plugin项目,源码导入后会出现很多的错误,在项目中继承了solr-core,和solrj中的类尝试着把访问修饰private改为protected.
如:solr-core org.apache.solr.handler.component.SearchHandler类中的shardHandlerFactory成员变量
protected ShardHandlerFactory shardHandlerFactory = new HttpShardHandlerFactory();
同时借鉴
https://issues.apache.org/jira/browse/SOLR-1395 Tomliu的做法,把
the bugs is :
1. solr's ShardDoc.java, ShardFieldSortedHitQueue line 210 :
final float f1 = e1.score == null ? 0.00f : e1.score;
final float f2 = e2.score == null ? 0.00f : e2.score;
等等.直到项目基本错误解决.
下载solr并且copy其中的apache-solr-3.6.1/example/solr到//data下.
目录结构如:
solr
--bin/
--conf/
--schama.xml
--solrconfig.xml
--...
--data/
--index/
--spellchecker/
--solr.xml
--README.txt
我们视/data/solr 为solr.home,改变${solr.home}/solr.xml为:
<solr persistent="false">
<!--
adminPath: RequestHandler path to manage cores.
If 'null' (or absent), cores will not be manageable via request handler
-->
<cores adminPath="/admin/cores" defaultCoreName="proxy">
<core name="proxy" instanceDir="proxy" />
</cores>
</solr>
改变后的目录结构为[
在solr.home
中新建proxy文件夹并把conf/ data/放到proxy文件夹下面],如:
solr/
--proxy/
--bin/
--conf/
--schama.xml
--solrconfig.xml
--....
--data/
--index/
--spellchecker
--solr.xml
上面的步骤在我尝试的时候费神了很久,为什么是这样的?我在代码发现的:org.apache.solr.katta.DeployableSolrKattaServer
final public static String serverNameDefault = "proxy";
final public static String serverNameProperty = "solr.server.name";
final public static String solrHomeDefault = "solrHome";
final public static String solrHomeProperty = "solr.home";
final public static String solrConfigFileDefault = "solr.xml";
final public static String solrConfigFileProperty = "solr.config.name";
public static String getServerName() {
return System.getProperty(serverNameProperty, serverNameDefault);
}
所以solr.home/solr.xml中配置的默认的solr coreName必须为proxy,否则启动Node的时候需要加上-Dsolr.server.name=youname,这里我不更改了使用它的配置
更改solrconfig.xml中增加[可加在<requestHandler name="search" class="solr.SearchHandler">之前,并去掉search的default=true]:
<requestHandler name="standard" class="solr.MultiEmbeddedSearchHandler" default="true">
<!-- default values for query parameters -->
<lst name="defaults">
<str name="echoParams">explicit</str>
<int name="rows">10</int>
</lst>
</requestHandler>
ok,我们在把配置好了的solr复制一份为solrhome1.这时我们有了/data/solrhome1
现在copy ${katta_install}/conf/ katta.node.properties, katta.zk.properties, log4j.properties到solr-katta-plugin项目的test/下[test为测试源码目录],更改katta.node.properties为
#node.server.class=net.sf.katta.lib.lucene.LuceneServer
node.server.class=org.apache.solr.katta.DeployableSolrKattaServer
创建一个启动类:
public class Launcher {
public final static DateFormat DATE_FORMAT = new SimpleDateFormat("yy_MM_dd_HH_mm_ss");
static {
System.setProperty("solr.home", "/data/solr");
System.setProperty("solr.server.name", "proxy");
System.setProperty("solr.directoryFactory", "solr.MMapDirectoryFactory");
System.setProperty("katta.log.dir", "/data/logs");
System.setProperty("katta.log.file", "katta_" + DATE_FORMAT.format(new Date()) + ".log");
}
public Launcher() {
}
public static void main(String[] args) throws Exception {
Katta.main(new String[]{"startNode", "-c", "org.apache.solr.katta.DeployableSolrKattaServer"});
}
}
运行该类,我们可以看到它加载了solr
10-18 14:22:02 [INFO 级别][ core.SolrCore ]类 (83 行) QuerySenderListener done.
10-18 14:22:02 [INFO 级别][ component.SpellCheckComponent ]类 (673 行) Loading spell index for spellchecker: default
10-18 14:22:02 [INFO 级别][ core.SolrCore ]类 (1325 行) [] Registered new searcher Searcher@70f5d656 main
10-18 14:22:02 [INFO 级别][ ipc.Server ]类 (328 行) Starting SocketReader
10-18 14:22:02 [INFO 级别][ node.Node ]类 (226 行) DeployableSolrKattaServer server started on : zhenqin-K45VM:20000
10-18 14:22:02 [INFO 级别][ ipc.Server ]类 (598 行) IPC Server Responder: starting
10-18 14:22:02 [INFO 级别][ ipc.Server ]类 (434 行) IPC Server listener on 20000: starting
10-18 14:22:02 [INFO 级别][ ipc.Server ]类 (1358 行) IPC Server handler 0 on 20000: starting
然后,回到终端执行:$bin/katta addIndex solrhome1 file:///data/solrhome1
当然,我们也可以把该solrhome1打包放入HDFS,$bin/katta addIndex solrhome1 hdfs://localhost:9000/solr/solrhome1.zip
$ bin/katta addIndex solrhome1 file:///data/solrhome1
.....
deployed index 'solrhome1' in 4840 ms
这是可以看见Eclipse控制台输出了:
10-18 14:24:02 [INFO 级别][ node.Node ]类 (279 行) executing ShardDeployOperation:3cca2475:[solrhome1#proxy]
10-18 14:24:02 [INFO 级别][ node.AbstractShardOperation ]类 (55 行) deploy shard 'solrhome1#proxy'
10-18 14:24:02 [INFO 级别][ node.ShardManager ]类 (104 行) install shard 'solrhome1#proxy' from file:/media/Study/data/katta/testIndexA/proxy
10-18 14:24:03 [WARN 级别][ util.NativeCodeLoader ]类 (52 行) Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
zhenqin-K45VM:20000 addShard solrhome1#proxy solrhome:/media/Study/data/katta/katta-shards/zhenqin-K45VM_20000/solrhome1#proxy
10-18 14:24:04 [INFO 级别][ core.SolrResourceLoader ]类 (103 行) new SolrResourceLoader for directory: '/media/Study/data/katta/katta-shards/zhenqin-K45VM_20000/solrhome1#proxy/'
10-18 14:24:04 [INFO 级别][ core.CoreContainer ]类 (448 行) Creating SolrCore 'solrhome1#proxy' using instanceDir: /media/Study/data/katta/katta-shards/zhenqin-K45VM_20000/solrhome1#proxy
10-18 14:24:04 [INFO 级别][ core.SolrResourceLoader ]类 (103 行) new SolrResourceLoader for directory: '/media/Study/data/katta/katta-shards/zhenqin-K45VM_20000/solrhome1#proxy/'
10-18 14:24:05 [WARN 级别][ core.SolrConfig ]类 (148 行) <indexDefaults> and <mainIndex> configuration sections are deprecated (but still work). Please use <indexConfig> instead.
10-18 14:24:05 [INFO 级别][ core.SolrConfig ]类 (162 行) Using Lucene MatchVersion: LUCENE_36
10-18 14:24:05 [INFO 级别][ core.Config ]类 (248 行) Loaded SolrConfig: solrconfig.xml
10-18 14:24:05 [INFO 级别][ schema.IndexSchema ]类 (413 行) Reading Solr Schema
10-18 14:24:05 [INFO 级别][ schema.IndexSchema ]类 (427 行) Schema name=collection1
10-18 14:24:05 [INFO 级别][ node.AbstractShardOperation ]类 (75 行) publish shard 'solrhome1#proxy'
10-18 14:24:05 [INFO 级别][ core.SolrCore ]类 (1325 行) [solrhome1#proxy] Registered new searcher Searcher@61db327f main
10-18 14:24:05 [INFO 级别][ core.SolrCore ]类 (43 行) QuerySenderListener sending requests to Searcher@48b4e8e2 main
10-18 14:24:05 [INFO 级别][ core.SolrCore ]类 (1386 行) [solrhome1#proxy] webapp=null path=null params={event=firstSearcher&q=solr+first+searching..} hits=0 status=0 QTime=6
10-18 14:24:05 [INFO 级别][ core.SolrCore ]类 (83 行) QuerySenderListener done.
10-18 14:24:05 [INFO 级别][ component.SpellCheckComponent ]类 (673 行) Loading spell index for spellchecker: default
10-18 14:24:05 [INFO 级别][ search.SolrIndexSearcher ]类 (256 行) Closing Searcher@61db327f main
fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=1,evictions=0,size=1,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
10-18 14:24:05 [INFO 级别][ core.SolrCore ]类 (1325 行) [solrhome1#proxy] Registered new searcher Searcher@48b4e8e2 main
至此,一个Node加载一个Shard完成.
step3 is over!