https://blog.csdn.net/lexang1/article/details/52496114
https://www.cnblogs.com/fengjian2016/p/5858320.html
SolrCloud是一个分布式解决方案,安装之前需要我们评估自己应用的数据规模,规划SolrCloud集群如划分多少个Shard,每个Shard需要几分冗余即Replication等;通过下面的例子详细讲解下安装SolrCloud的过程;
0,准备条件:
Apache Solr-4.2.0,Apache Tomcat 7.0.x,Apache ZooKeeper 3.4.5;
1,规划SolrCloud
单一SolrCloud数据集合(Collection):mycollection
ZooKeeper集群(ensemble):5台ZooKeeper服务器
3个SolrCloud实例节点,复制因子为3(即索引冗余数)
3个索引分片(Shard)分布在3个SolrCloud节点(node)上
手动将3个索引分片(Shard)的复本(Replica)分布在3个SolrCloud节点上
2,Solr安装
分别在192.168.1.1-3上搭建3个Solr服务器节点。
3,ZooKeeper集群(ensemble)安装
3.1,下载Apache ZooKeeper 3.4.5;
3.2,解压zookeeper-3.4.5.tar.gz, tar -xvf zookeeper-3.4.5.tar.gz
3.3,创建ZooKeeper配置文件:将zookeeper_home/conf/zoo_sample.cfg重命名为zoo.cfg
3.4,修改ZooKeeper配置文件zoo.cfg如下:
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial synchronization phase can take
initLimit=10
# The number of ticks that can pass between sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored. Choose appropriately for your environment
dataDir=/home/myuser/zookeeper/data/
# the port at which the clients will connect
clientPort=2181
# the directory where transaction log is stored. this parameter provides dedicated log device for ZooKeeper dataLogDir=/home/myuser/zookeeper/log/
# ZooKeeper server and its port no. ZooKeeper ensemble should know about every other machine in the ensemble specify server id by creating 'myid' file in the dataDir
server.1=192.168.1.1:2888:3888
server.2=192.168.1.2:2888:3888
server.3=192.168.1.3:2888:3888
server.4=192.168.1.4:2888:3888
server.5=192.168.1.5:2888:3888
为每个ZooKeeper服务器分配合适的id(用于标识ZooKeeper服务器)写入myid文件并将其放入dataDir文件夹中
3.5,启动ZooKeeper服务器
# cd /zookeeper_home/bin/
# ./zkServer.sh start
在192.168.1.2-5机器上重复以上步骤;完成ZooKeeper集群(ensemble)的搭建;
4,SolrCloud搭建
4.1,在192.168.1.1(可任选一台机器)上创建SolrCloud配置文件目录:/usr/local/solrcloud/config-files
4.2,在192.168.1.1(可任选一台机器)上创建Solr Zk CLI库文件目录:/usr/local/solrcloud/solr-lib
4.3,将solr example中的配置文件Copy到SolrCloud配置文件目录中:cp solr-4.2.0/example/solr/collection1/conf/* /usr/local/solrcloud/config-files
4.4,将solr.war WEB-INF/lib中的jar文件Copy到Solr Zk CLI库文件目录中:cp solr_home/WEB/lib/* /usr/local/solrcloud/solr-cli-lib
4.5,之所以能分布式是因为引入ZooKeeper来统一保存配置文件,故而需要将SolrCloud的配置文件上传到ZooKeeper中:
将配置文件上传到ZooKeeper中
java -classpath .:/usr/local/solrcloud/solr-lib/* org.apache.solr.cloud.ZkCLI -cmd upconfig -zkhost 192.168.1.1:2181,192.168.1.2:2181,192.168.1.3:2181,192.168.1.4:2181,192.168.1.5:2181 -confdir /usr/local/solrcloud/config-files/ -confname myconf
将上传到ZooKeeper中配置文件与Collection相关联
java -classpath .:/usr/local/solrcloud/solr-lib/* org.apache.solr.cloud.ZkCLI -cmd linkconfig -collection mycollection -confname myconf -zkhost 192.168.1.1:2181,192.168.1.2:2181,192.168.1.3:2181,192.168.1.4:2181,192.168.1.5:2181
可连上ZooKeeper查看上传的配置信息,5台ZooKeeper服务器可任选一台
cd zookeeper_home/bin
./zkCli.sh -server localhost:2181
执行查询命令: ls /configs/myconf,即可看到上传到ZooKeeper的所有配置信息
4.6,在3个Solr服务器节点(端口都为8080)上进行如下操作:
4.6.1,在solr.home中创建solr.xml文件内容如下:
4.6.2,在Tomcat启动时加上如下JVM参数:
"-Dsolr.solr.home=/usr/local/solr -DzkHost=192.168.1.1:2181,192.168.1.2:2181,192.168.1.3:2181,192.168.1.4:2181,192.168.1.5:2181"
4.6.3,启动Tomcat: /usr/local/tomcat/bin/startup.sh
4.6.4,访问solr管理页: http://192.168.1.1:8080/solr,即可见提示没有core的界面(因为当前还未创建Collection及Core)
到此SolrCloud已经搭建好啦,不过里面啥也没有,需要配置好才能用;
4.7,在SolrCloud中创建数据集(Collection),分片(Shard(s)),复本(Replicas(s))
4.7.1,创建数据集(Collection)
curl 'http://192.168.1.1:8080/solr/admin/collections?action=CREATE&name=mycollection&numShards=3&replicationFactor=3&maxShardsPerNode=3'
name:数据集名称
numShards:分片数
replicationFactor:每个分片的复本数
maxShardsPerNode:每个Solr服务器节点上最大分片数(Solr 4.2新增)
4.7.2,将分片(Shards)复本(Replicas)分布在3个Solr服务器节点上
分片(Shard) 1 Replica's(Shard 1 Replica_1 at 192.168.1.2)
curl 'http://192.168.1.1:8080/solr/admin/cores?action=CREATE&name=mycollection_shard1_replica_2&collection=mycollection&shard=shard1'
curl 'http://192.168.1.3:8080/solr/admin/cores?action=CREATE&name=mycollection_shard1_replica_3&collection=mycollection&shard=shard1'
分片(Shard) 2 Replica's(Shard 2 Repkica_1 at 192.168.1.3)
curl 'http://192.168.1.1:8080/solr/admin/cores?action=CREATE&name=mycollection_shard2_replica_2&collection=mycollection&shard=shard2'
curl 'http://192.168.1.2:8080/solr/admin/cores?action=CREATE&name=mycollection_shard2_replica_3&collection=mycollection&shard=shard2'
分片(Shard) 3 Replica's(Shard 3 Repkica_1 at 192.168.1.1)
curl 'http://192.168.1.2:8080/solr/admin/cores?action=CREATE&name=mycollection_shard3_replica_2&collection=mycollection&shard=shard3'
curl 'http://192.168.1.3:8080/solr/admin/cores?action=CREATE&name=mycollection_shard3_replica_3&collection=mycollection&shard=shard3'
5,为SolrCloud创建索引
在SolrCloud上创建索引跟在单个Solr上创建索引方法一样,只不过SolrCloud会自动进行索引分片(Shard)和复制(Replica);
6,在SolrCloud上进行查询
http://192.168.1.1:8080/solr/mycollection/select?q=*:*
q为查询关键字,修改q参数即可对SolrCloud中的所有索引数据进行查询(可通过监控每台Solr服务器节点的日子看到查询会在各个节点上进行,由此可知SolrCloud会将查询分配到各个节点上进行后将各个节点的查询结果汇总);
7.笔者安装过程出现的问题:
严重: null:java.lang.NoClassDefFoundError: org/apache/solr/analysis/BaseTokenizerFactory……
这是个生坑,笔者原来一直以为是lib路径问题,各种改配置文件,后来发现真正原因是:mmseg4j这个jar包,我用的还是在solr 3.5下的mmseg4j-all-1.8.5.jar文件,而现在在4.2.1上肯定版本是变化,问题就出现这里!我到maven官网下了mmseg4j-analysis-1.9.0.jar、mmseg4j-core-1.9.0.jar、mmseg4j-solr-1.9.0.jar三个jar文件,放到我的solr发布的lib目录下,solr cloud ok!
在查询IKAnalyzer2012FF_u1中文分词文档的时候,发现原来在Solr4.0发布以后,官方取消了BaseTokenizerFactory接口,而直接使用Lucene Analyzer标准接口。因此IK分词器2012 FF版本也取消了org.wltea.analyzer.solr.IKTokenizerFactory类。这也是solr 3.x跟 solr 4.x 有很大变化的一个地方!