Lucene问题之：Too many documents

最新推荐文章于 2021-07-27 14:48:30 发布

空杯Begin

最新推荐文章于 2021-07-27 14:48:30 发布

阅读量555

点赞数

分类专栏： Lucene系列文章标签： lucene too many documents lucene太多文档

本文链接：https://blog.csdn.net/qq_14838603/article/details/85717213

版权

Lucene系列专栏收录该内容

6 篇文章 0 订阅

订阅专栏

Too many documents

写好的Lucene程序在运行期间抛出了如下的异常，详细的异常日志如下：

There was an unexpected error (type=Internal Server Error, status=500).
Too many documents: composite IndexReaders cannot exceed 2147483519 but readers have total maxDoc=2302747920
java.lang.IllegalArgumentException: Too many documents: composite IndexReaders cannot exceed 2147483519 but readers have total maxDoc=2302747920
	at org.apache.lucene.index.BaseCompositeReader.<init>(BaseCompositeReader.java:86)
	at org.apache.lucene.index.MultiReader.<init>(MultiReader.java:59)
	at com.lhczf.lucenedb.util.LuceneUtil.makeCommSearcher(LuceneUtil.java:111)
	at com.lhczf.lucenedb.example.RestFulManager.facetByFieldName(RestFulManager.java:154)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

    if (maxDoc > IndexWriter.getActualMaxDocs()) {
      if (this instanceof DirectoryReader) {
        // A single index has too many documents and it is corrupt (IndexWriter prevents this as of LUCENE-6299)
        throw new CorruptIndexException("Too many documents: an index cannot exceed " + IndexWriter.getActualMaxDocs() + " but readers have total maxDoc=" + maxDoc, Arrays.toString(subReaders));
      } else {
        // Caller is building a MultiReader and it has too many documents; this case is just illegal arguments:
        throw new IllegalArgumentException("Too many documents: composite IndexReaders cannot exceed " + IndexWriter.getActualMaxDocs() + " but readers have total maxDoc=" + maxDoc);
      }
    }

查看对应的源码，给出的解释信息如下：

public class IndexWriter implements Closeable, TwoPhaseCommit, Accountable,
    MergePolicy.MergeContext {

  /** Hard limit on maximum number of documents that may be added to the
   *  index.  If you try to add more than this you'll hit {@code IllegalArgumentException}. */
  // We defensively subtract 128 to be well below the lowest
  // ArrayUtil.MAX_ARRAY_LENGTH on "typical" JVMs.  We don't just use
  // ArrayUtil.MAX_ARRAY_LENGTH here because this can vary across JVMs:
  public static final int MAX_DOCS = Integer.MAX_VALUE - 128;

  /** Maximum value of the token position in an indexed field. */
  public static final int MAX_POSITION = Integer.MAX_VALUE - 128;

  // Use package-private instance var to enforce the limit so testing
  // can use less electricity:
  private static int actualMaxDocs = MAX_DOCS;