注:全量索引和增量索引data-config.xml和delta-data-config.xml配置文件默认放在和solrconfig.xml同级目录
solrconfig.xml配置如下:
<requestHandler name="/dataimport"
class="org.apache.solr.handler.dataimport.DataImportHandler">
<lst name="defaults">
<str name="config">delta-data-config.xml</str>
</lst>
</requestHandler>
<dataConfig>
<dataSource name="jdbc" driver="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost:3306/test?zeroDateTimeBehavior=convertToNull"
user="root" password="shyh"/>
<document name="st_data">
<entity name="solrtext" pk="id"
query="select * from solrtext"
deltaImportQuery="select * from solrtext where id='${dih.delta.id}'"
deltaQuery="select id from solrtext where addon > '${dih.last_index_time}'"
transformer="RegexTransformer">
<field column="id" name="id" />
<field column="url" name="url" />
<field column="title" name="title" />
<field column="author" name="author" />
<field column="addon" name="addon"/>
<field column="path" name="path"/>
</entity>
</document>
</dataConfig>
schemal.xml
<field name="id" type="string" indexed="true" stored="true" required="true" multiValued="false" />
<field name="url" type="text_general" indexed="true" stored="true" />
<field name="title" type="text_general" indexed="true" stored="true"/>
<field name="author" type="text_general" indexed="true" stored="true"/>
<field name="addon" type="string" indexed="true" stored="true"/>
<field name="path" type="string" indexed="false" stored="true"/>
上面主要是通过内置变量“${dih.delta.id}”和 “${dih.last_index_time}”来记录本次索引的id和最后索引时间。这里,会保存在deltaimport.properties文件中,示例如下:
#Mon Jan 26 11:13:07 CST 2015
solrtext.last_index_time=2015-01-26 11\:12\:35
last_index_time=2015-01-26 11\:12\:35
配置定时任务:
- 将 apache-solr-dataimportscheduler-1.0.jar 和solr自带的 apache-solr-dataimporthandler-.jar, apache-solr-dataimporthandler-extras-.jar 放到tomcat/webapps/solr/WEB-INF的lib目录下面
- 修改solr中WEB-INF/web.xml
<listener> <listener-class> org.apache.solr.handler.dataimport.scheduler.ApplicationListener </listener-class> </listener>
将apache-solr-dataimportscheduler-.jar 中 dataimport.properties 取出并根据实际情况修改,然后放到 solr.home/conf (不是solr.home/core/conf) 目录下面 ,我的位置为:F:\solr\solrhome\conf(如不存在conf可手动新建,dataimport.properties存放在tomcat的solr.xml里配置的solr/home路径的conf文件夹下)
- solr.xml配置如下
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <Context docBase="F:\solr\apache-tomcat-7.0.53\webapps\solr.war" debug="0" crossContext="true" > <Environment name="solr/home" type="java.lang.String" value="F:\solr\solrhome" override="true" /> </Context>
- dataimport.properties配置说明
################################################# # # # dataimport scheduler properties # # # ################################################# # to sync or not to sync # 1 - active; anything else - inactive syncEnabled=1 # which cores to schedule # in a multi-core environment you can decide which cores you want syncronized # leave empty or comment it out if using single-core deployment syncCores=collection1 # solr server name or IP address # [defaults to localhost if empty] server=localhost # solr server port # [defaults to 80 if empty] port=8080 # application name/context # [defaults to current ServletContextListener's context (app) name] webapp=solr # URL params [mandatory] # remainder of URL params=/dataimport?command=delta-import&clean=false&commit=true # schedule interval # number of minutes between two runs # [defaults to 30 if empty] interval=1 # 重做索引的时间间隔,单位分钟,默认7200,即5天; # 为空,为0,或者注释掉:表示永不重做索引 reBuildIndexInterval=7200 # 重做索引的参数 reBuildIndexParams=/dataimport?command=full-import&clean=true&commit=true # 重做索引时间间隔的计时开始时间,第一次真正执行的时间=reBuildIndexBeginTime+reBuildIndexInterval*60*1000; # 两种格式:2012-04-11 03:10:00 或者 03:10:00,后一种会自动补全日期部分为服务启动时的日期 reBuildIndexBeginTime=03:10:00