问题
1.操作
2.报错异常信息:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException:ERROR:
[doc=7] multiple values encountered for non multiValued field col1:[test, test]
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:519)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:207)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:202)
at org.apache.solr.client.solrj.impl.LBHttpSolrServer.doRequest(LBHttpSolrServer.java:312)
at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:273)
at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:310)
at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
at com.ngdata.hbaseindexer.indexer.DirectSolrInputDocumentWriter.retryAddsIndividually(DirectSolrInputDocumentWriter.java:123)
at com.ngdata.hbaseindexer.indexer.DirectSolrInputDocumentWriter.add(DirectSolrInputDocumentWriter.java:108)
at com.ngdata.hbaseindexer.indexer.Indexer.indexRowData(Indexer.java:140)
at com.ngdata.hbaseindexer.indexer.IndexingEventListener.processEvents(IndexingEventListener.java:84)
at com.ngdata.sep.impl.SepEventExecutor$1.run(SepEventExecutor.java:97)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
3.solr对应Collection中的schema.xml定义:
<dynamicField name="*_i" type="int" indexed="true"
stored="true" multiValued="false" />
<dynamicField name="*_l" type="long" indexed="true"
stored="true" multiValued="false" />
<dynamicField name="*_f" type="double" indexed="true"
stored="true" multiValued="false" />
<dynamicField name="*_d" type="string" indexed="true"
stored="true" multiValued="false" />
<dynamicField name="*" type="string" indexed="true"
stored="true" multiValued="false" />
<field name="*_busi_" type="string" indexed="true" stored="true" />
分析&处理
ks_indexer使用zk记录通过不hbase日志同步点,把RegionServer的HBase日志,以增量方式,同步到Solr中,同步可按频率或者数据量触发。HBase中,同一row-key对应的记录,在统一同步点时,根据row-key做合并处理,同一个非id字段,会出现多值情况,schema.xml没设置multiValued="true",同步线程代码会抛上面错误信息。
处理方式有两种:
1.如果字段没排序需求,设置 multiValued="true",即可解决field多值问题
2.如果字段有排序需求,hbase数据多版本没业务需求,即可通过修改SolrInputField,保证同一个doc中的field只有单值。