CentOS7 Solr6.6 集群部署

Zookeeper配置

Solr

添加zk地址

vi bin/solr.in.sh

# Set the ZooKeeper connection string if using an external ZooKeeper ensemble
# e.g. host1:2181,host2:2181/chroot
# Leave empty if not using SolrCloud
ZK_HOST="hbase201:2181,hbase202:2181,hbase203:2181/solr"

# Set the ZooKeeper client timeout (for SolrCloud mode)
ZK_CLIENT_TIMEOUT="15000"

上传配置文件

./zkcli.sh -zkhost hbase201,hbase202,hbase203:2181/solr -cmd upconfig -confdir /usr/solr/solr-6.6.0/server/solr/configsets/data_driven_schema_configs/conf -confname solr

同步到其它服务器

scp -r /usr/solr hbase201:/usr/
scp -r /usr/solr hbase202:/usr/

分别启动solr服务

bin/solr start -force

集群操作

查看状态

http://hbase203:8983/solr/#/~cloud

创建集群

http://hbase203:8983/solr/admin/collections?action=CREATE&name=test&numShards=3&replicationFactor=3&maxShardsPerNode=3&collection.configName=solr

删除集群

http://hbase203:8983/solr/admin/collections?action=DELETE&name=test

重新加载

http://hbase203:8983/solr/admin/collections?action=RELOAD&name=test

更新配置文件，添加DIH

修改solrconfig.xml

vi solrconfig.xml

<lib dir="${solr.install.dir:../../../..}/dist/" regex="solr-dataimporthandler-.*\.jar" />  

<requestHandler name="/dataimport" class="solr.DataImportHandler">  
  <lst name="defaults">  
    <str name="config">data-config.xml</str>  
  </lst>  
</requestHandler> 

创建data-config.xml

vi data-config.xml

<dataConfig>  
  <dataSource type="JdbcDataSource"  
              driver="com.mysql.jdbc.Driver"  
              convertType="true"  
              url="jdbc:mysql://127.0.0.1:3306/db_name"  
              user="sa"  
              password="123456"/>  
  <document>  
    <entity name="query_news" query="SELECT id, title, content, tags FROM news" >  
    </entity>  
  </document>  
</dataConfig> 

创建schema.xml

vi schema.xml

<schema name="query_news">
  <uniqueKey>id</uniqueKey>
  <field name="id" type="int" indexed="true" stored="true" required="true" multiValued="false" />
  <field name="title" type="string" indexed="true" stored="true" required="true" multiValued="false" />
  <field name="content" type="string" indexed="true" stored="true" required="false" multiValued="false" />
  <field name="tags" type="string" indexed="true" stored="false" required="false" multiValued="false" />
  <field name="_version_" type="long" indexed="true" stored="true"/>

  <field name="info_text" type="text_general" indexed="true" stored="false" multiValued="true" />

  <copyField source="title" dest="info_text" />
  <copyField source="content" dest="info_text" />
  <copyField source="tags" dest="info_text" />

  <fieldType name="string" class="solr.StrField" sortMissingLast="true" />
  <fieldType name="int" class="solr.TrieIntField" precisionStep="0" positionIncrementGap="0"/>
  <fieldType name="text_general" class="solr.TextField" positionIncrementGap="100">
    <analyzer type="index">
      <tokenizer class="solr.StandardTokenizerFactory"/>
      <filter class="solr.LowerCaseFilterFactory"/>
    </analyzer>
    <analyzer type="query">
      <tokenizer class="solr.StandardTokenizerFactory"/>
      <filter class="solr.LowerCaseFilterFactory"/>
    </analyzer>
  </fieldType>
</schema>

上传配置文件

./zkcli.sh -zkhost hbase201,hbase202,hbase203:2181/solr -cmd putfile /configs/solr/solrconfig.xml /usr/solr/solr-6.6.0/server/solr/configsets/data_driven_schema_configs/conf/solrconfig.xml
./zkcli.sh -zkhost hbase201,hbase202,hbase203:2181/solr -cmd putfile /configs/solr/data-config.xml /usr/solr/solr-6.6.0/server/solr/configsets/data_driven_schema_configs/conf/data-config.xml
./zkcli.sh -zkhost hbase201,hbase202,hbase203:2181/solr -cmd putfile /configs/solr/schema.xml /usr/solr/solr-6.6.0/server/solr/configsets/data_driven_schema_configs/conf/schema.xml

更新配置文件

http://hbase201:8983/solr/admin/collections?action=RELOAD&name=test

备份和恢复

备份

http://hbase201:8983/solr/test/replication?command=backup&location=/home/test_bak

恢复

1. 停止Solr服务
2. Copy备份文件到CoreIndex目录
3. 启动Solr服务

Index存储为HDFS

修改solrconfig.xml

vi solrconfig.xml

<directoryFactory name="DirectoryFactory" class="solr.HdfsDirectoryFactory">
  <str name="solr.hdfs.home">hdfs://hbase201:9000/solr</str>
  <bool name="solr.hdfs.blockcache.enabled">true</bool>
  <int name="solr.hdfs.blockcache.slab.count">1</int>
  <bool name="solr.hdfs.blockcache.direct.memory.allocation">true</bool>
  <int name="solr.hdfs.blockcache.blocksperbank">16384</int>
  <bool name="solr.hdfs.blockcache.read.enabled">true</bool>
  <bool name="solr.hdfs.nrtcachingdirectory.enable">true</bool>
  <int name="solr.hdfs.nrtcachingdirectory.maxmergesizemb">16</int>
  <int name="solr.hdfs.nrtcachingdirectory.maxcachedmb">192</int>
</directoryFactory>

<lockType>${solr.lock.type:hdfs}</lockType>

上传配置文件到zookeeper

./zkcli.sh -zkhost hbase201,hbase202,hbase203:2181/solr -cmd putfile /configs/solr/data-config.xml /usr/solr/solr-6.6.0/server/solr/configsets/data_driven_schema_configs/conf/data-config.xml

更新hadoop处理jar包（solr自带版本不一致）

rm -f server/solr-webapp/webapp/WEB-INF/lib/hadoop-*
rm -f server/solr-webapp/webapp/WEB-INF/lib/protobuf-java-*

cd /usr/hadoop/hadoop-2.7.4/share

cp hadoop/common/hadoop-common-2.7.4.jar /usr/solr/solr-6.6.0/server/solr-webapp/webapp/WEB-INF/lib/
cp hadoop/common/lib/hadoop-* /usr/solr/solr-6.6.0/server/solr-webapp/webapp/WEB-INF/lib/
cp hadoop/common/lib/protobuf-java-2.5.0.jar /usr/solr/solr-6.6.0/server/solr-webapp/webapp/WEB-INF/lib/
cp hadoop/hdfs/hadoop-hdfs-2.7.4.jar /usr/solr/solr-6.6.0/server/solr-webapp/webapp/WEB-INF/lib/