打上SOLR-236_collapsing.patch补丁,实现 solr 搜索结果折叠、除去重复的搜索结果,可以实现类似google搜索结果的“站内的其它相关信息 ”。solr collapsing patch 是用 hash 某个字段来实现折叠重复结果的。下面我演示下应用这个补丁并搜索试用下。
其实 solr 上已经有了这功能的实现:solr 1.3 collapse patch, 请看:https://issues.apache.org/jira/browse/SOLR-236 ,我这里下载是了新的:https://issues.apache.org/jira/secure/attachment/12403590/SOLR-236_collapsing.patch 。
下载 好后就需要打上补丁了,先准备一份源码在D:/apache-solr-1.3.0目录下。没有可以去下载:http: //archive.apache.org/dist/lucene/solr/1.3.0/apache-solr-1.3.0.zip。把SOLR- 236_collapsing.patch文件放在D:/apache-solr-1.3.0目录下, 打补丁有我知道的有两种:用linux工具 patch(windows 下有 cygwin);用 ant 的 patch。
windows cygwin 的 patch:
D:\apache-solr-1.3.0>patch -p0 < SOLR-236_collapsing.patch
patching file src/test/org/apache/solr/search/TestDocSet.java
patching file src/java/org/apache/solr/search/CollapseFilter.java
patching file src/java/org/apache/solr/search/DocSet.java
patching file src/java/org/apache/solr/search/NegatedDocSet.java
patching file src/java/org/apache/solr/search/SolrIndexSearcher.java
patching file src/java/org/apache/solr/common/params/CollapseParams.java
patching file src/java/org/apache/solr/handler/component/CollapseComponent.java
ant patch,把下面的内容保存为 patch-build.xml 放到 D:\apache-solr-1.3.0 目录下:
- <? xml version = “1.0″ encoding = “UTF-8″ ?>
- < project name = “solr-patch” default = “apply-patch” basedir = “.” >
- < target name = “apply-patch” description = “Apply a patch file. Set -Dpatch.file” >
- < patch patchfile = “${patch.file}” strip = “0″ />
- </ target >
- </ project >
ant 打补丁:
D:\apache-solr-1.3.0>ant -Dpatch.file=SOLR-236_collapsing.patch -f patch-build.xml
Buildfile: patch-build.xmlapply-patch:
[patch] patching file src/test/org/apache/solr/search/TestDocSet.java
[patch] patching file src/java/org/apache/solr/search/CollapseFilter.java
[patch] patching file src/java/org/apache/solr/search/DocSet.java
[patch] patching file src/java/org/apache/solr/search/NegatedDocSet.java
[patch] patching file src/java/org/apache/solr/search/SolrIndexSearcher.java
[patch] patching file src/java/org/apache/solr/common/params/CollapseParams.java
[patch] patching file src/java/org/apache/solr/handler/component/CollapseComponent.javaBUILD SUCCESSFUL
Total time: 0 seconds
源码打上了补丁,然后用 ant 构建源码:
D:\apache-solr-1.3.0>ant dist
在 D:/apache-solr-1.3.0/dist 目录下可以找到编译好的 solr 了。然后把 solr 放到 tomcat 中去运行它,把下面的内容保存在 TOMCAT_HOME/conf/Catalina/localhost/solr.xml 文件中:
- < Context docBase = “D:\apache-solr-1.3.0\dist\apache-solr-1.3.0.war” reloadable = “true” >
- < Environment name = “solr/home” type = “java.lang.String” value = “D:\apache-solr-1.3.0\example\solr” override = “true” />
- </ Context >
修改 D:\apache-solr-1.3.0\example\solr\conf\solrconfig.xml 使 solr 可以支持 collapse。
定义搜索组件,在 QueryComponent 附近:
- < searchComponent name = “collapse” class = “org.apache.solr.handler.component.CollapseComponent” />
定义一个 handler 使用上面的搜索组件:
- < requestHandler name = “collapse” class = “solr.SearchHandler” >
- <!– default values for query parameters –>
- < lst name = “defaults” >
- < str name = “echoParams” > explicit </ str >
- </ lst >
- < arr name = “components” >
- < str > collapse </ str >
- < str > debug </ str >
- </ arr >
- </ requestHandler >
安装启动 tomcat,现在提交一些数据给它,用官方的示例数据就可以了。运行:
D:\apache-solr-1.3.0\example\exampledocs>java -Durl=http://localhost:8080/solr/update -Dcommit=yes -jar post.jar *.xml
SimplePostTool: version 1.2
SimplePostTool: WARNING: Make sure your XML documents are encoded in UTF-8, other encodings are not currently supported
SimplePostTool: POSTing files to http://localhost:8080/solr/update..
SimplePostTool: POSTing file hd.xml
SimplePostTool: POSTing file ipod_other.xml
SimplePostTool: POSTing file ipod_video.xml
SimplePostTool: POSTing file mem.xml
SimplePostTool: POSTing file monitor.xml
SimplePostTool: POSTing file monitor2.xml
SimplePostTool: POSTing file mp500.xml
SimplePostTool: POSTing file sd500.xml
SimplePostTool: POSTing file solr.xml
SimplePostTool: POSTing file spellchecker.xml
SimplePostTool: POSTing file utf8-example.xml
SimplePostTool: POSTing file vidcard.xml
SimplePostTool: COMMITting Solr index changes..
http://localhost:8080/solr/admin/stats.jsp 有结果了? 有了。然后开始查询试试看。
结果:
- <? xml version = “1.0″ encoding = “UTF-8″ ?>
- < response >
- < lst name = “responseHeader” >
- < int name = “status” > 0 </ int >
- < int name = “QTime” > 0 </ int >
- < lst name = “params” >
- < str name = “collapse.field” > popularity </ str >
- < str name = “fl” > id </ str >
- < str name = “collapse.threshold” > 1 </ str >
- < str name = “indent” > on </ str >
- < str name = “q” > *:* </ str >
- < str name = “qt” > collapse </ str >
- < str name = “collapse” > true </ str >
- </ lst >
- </ lst >
- < lst name = “collapse_counts” >
- < str name = “field” > popularity </ str >
- < lst name = “doc” >
- < int name = “SP2514N” > 4 </ int >
- < int name = “F8V7067-APL-KIT” > 1 </ int >
- < int name = “MA147LL/A” > 1 </ int >
- < int name = “TWINX2048-3200PRO” > 1 </ int >
- < int name = “VS1GB400C3″ > 3 </ int >
- < int name = “1″ > 10 </ int >
- </ lst >
- < lst name = “count” >
- < int name = “6″ > 4 </ int >
- < int name = “1″ > 1 </ int >
- < int name = “10″ > 1 </ int >
- < int name = “5″ > 1 </ int >
- < int name = “7″ > 3 </ int >
- < int name = “0″ > 10 </ int >
- </ lst >
- < str name = “debug” > HashDocSet(6) Time(ms): 0/0/0/0 </ str >
- </ lst >
- < result name = “response” numFound = “6″ start = “0″ >
- < doc >
- < str name = “id” > SP2514N </ str >
- </ doc >
- < doc >
- < str name = “id” > F8V7067-APL-KIT </ str >
- </ doc >
- < doc >
- < str name = “id” > MA147LL/A </ str >
- </ doc >
- < doc >
- < str name = “id” > TWINX2048-3200PRO </ str >
- </ doc >
- < doc >
- < str name = “id” > VS1GB400C3 </ str >
- </ doc >
- < doc >
- < str name = “id” > 1 </ str >
- </ doc >
- </ result >
- </ response >
可以看到 collapse_counts 相关的输出:
- < lst name = “collapse_counts” >
- < str name = “field” > popularity </ str >
- < lst name = “doc” >
- < int name = “SP2514N” > 4 </ int >
- …
- </ lst >
- < lst name = “count” >
- < int name = “6″ > 4 </ int >
- < int name = “1″ > 1 </ int >
- < int name = “10″ > 1 </ int >
- < int name = “5″ > 1 </ int >
- < int name = “7″ > 3 </ int >
- < int name = “0″ > 10 </ int >
- </ lst >
- < str name = “debug” > HashDocSet(6) Time(ms): 0/0/0/0 </ str >
- </ lst >
上面的 count 下的内容(它的顺序是result/doc的顺序),表示 popularity=6 相同的结果还有 4 个,与 popularity=1 相同的结果还有 1 个,依此类推。这样就可以显示给用户的界面里提示“相同的其它内容不有N个”。
使用的参数有:
- #启用 collapse 组件
- collapse=true
- #用那个字段来 hash 去除重量内容
- collapse.field=popularity
- #要结果中可以最多出现几个相同的文档
- collapse.threshold=1
当然还有其它参数,请看:org.apache.solr.common.params.CollapseParams 类。
原文出处:http://blog.chenlb.com/2009/04/apply-solr-collapsing-patch-remove-duplicate-result.html