边学边记（八） lucene索引结构详解五（_N.tis,_N.tii）

最新推荐文章于 2019-10-22 14:37:55 发布
最新推荐文章于 2019-10-22 14:37:55 发布
阅读量2.3k 收藏
点赞数
分类专栏： Lucene 文章标签： tis tii文件详解 lucene索引结构详解 lucene索引结构分析 lucene索引结构教程
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/a276202460/article/details/5651471
版权

_N.tis保存了此段内容中的项（term）信息，因为lucene是倒排的索引格式 所以分词出来的term保存在tis文件里 每个term的信息包含了出现此term的doc的频率(多少个doc存在)等信息，每个term的具体信息中包含了出现此term的域编号（fieldnum）等信息

tis的文件结构：

TermInfoFile (.tis)--> TIVersion, TermCount, IndexInterval, SkipInterval, MaxSkipLevels, TermInfos

TIVersion --> UInt32

TermCount --> UInt64

IndexInterval --> UInt32

SkipInterval --> UInt32

MaxSkipLevels --> UInt32

TermInfos --> <TermInfo> TermCount

TermInfo --> <Term, DocFreq, FreqDelta, ProxDelta, SkipDelta>

Term --> <PrefixLength, Suffix, FieldNum>

Suffix --> String

PrefixLength, DocFreq, FreqDelta, ProxDelta, SkipDelta
--> VInt

读取tis文件内容（文档中没有详细说明文件保存的细节可参看org.apache.lucene.index.TermInfosWriter 类）

 

/****************
 *
 *Create Class:ReadTermIndex.java
 *Author:a276202460
 *Create at:2010-6-7
 */
package com.rich.lucene.io;
public class ReadTerminfo {
	/**
	 * @param args
	 * @throws Exception 
	 */
	public static void main(String[] args) throws Exception {
		 String indexfile = "D:/lucenetest/indexs/txtindex/index4/_0.tis";
		 IndexFileInput input = null;
		 try{
			 input = new IndexFileInput(indexfile);
			 System.out.println("term index version:"+input.readInt());
			 long termcount = input.readLong();
			 System.out.println("term count:"+termcount);
			 System.out.println("term IndexInterval:"+input.readInt());
			 System.out.println("term SkipInterval:"+input.readInt());
			 System.out.println("term MaxSkipLevels:"+input.readInt());
			 for(long i = 0 ;i < termcount;i++){
				 System.out.println("*****read term info["+i+"]******");
				 System.out.println("the term share prefixlength is :"+input.readVInt());
				 System.out.println("term's own stuffix is:"+input.readString());
				 System.out.println("exists this term's field number is:"+input.readVInt());
				 int doccount = input.readVInt();
				 System.out.println("the doc count contain this term is:"+doccount);
				 System.out.println("the position of this term's TermFreqs within the .frq file is:"+input.readVLong());
				 System.out.println("the position of this term's TermPositions within the .prx file is:"+input.readVLong());
				 if(doccount >= 16)
				 System.out.println("the position of this term's SkipData within the .frq file is:"+input.readVInt());
			 }
			 
		 }finally{
			 input.close();
		 }
   
	}
}

运行结果：

 

term index version:-4
term count:22
term IndexInterval:128
term SkipInterval:16
term MaxSkipLevels:10
*****read term info[0]******
the term share prefixlength is :0
term's own stuffix is:做
exists this term's field number is:2
the doc count contain this term is:1
the position of this term's TermFreqs within the .frq file is:0
the position of this term's TermPositions within the .prx file is:0
*****read term info[1]******
the term share prefixlength is :1
term's own stuffix is:??
exists this term's field number is:2
the doc count contain this term is:1
the position of this term's TermFreqs within the .frq file is:1
the position of this term's TermPositions within the .prx file is:1
*****read term info[2]******
the term share prefixlength is :1
term's own stuffix is:??
exists this term's field number is:2
the doc count contain this term is:1
the position of this term's TermFreqs within the .frq file is:1
the position of this term's TermPositions within the .prx file is:1
*****read term info[3]******
the term share prefixlength is :1
term's own stuffix is:??
exists this term's field number is:2
the doc count contain this term is:1
the position of this term's TermFreqs within the .frq file is:1
the position of this term's TermPositions within the .prx file is:1
*****read term info[4]******
the term share prefixlength is :1
term's own stuffix is:??
exists this term's field number is:2
the doc count contain this term is:2
the position of this term's TermFreqs within the .frq file is:1
the position of this term's TermPositions within the .prx file is:1
*****read term info[5]******
the term share prefixlength is :1
term's own stuffix is:??
exists this term's field number is:2
the doc count contain this term is:1
the position of this term's TermFreqs within the .frq file is:2
the position of this term's TermPositions within the .prx file is:2
*****read term info[6]******
the term share prefixlength is :1
term's own stuffix is:??
exists this term's field number is:2
the doc count contain this term is:2
the position of this term's TermFreqs within the .frq file is:1
the position of this term's TermPositions within the .prx file is:1
*****read term info[7]******
the term share prefixlength is :0
term's own stuffix is:搜
exists this term's field number is:2
the doc count contain this term is:2
the position of this term's TermFreqs within the .frq file is:3
the position of this term's TermPositions within the .prx file is:3
*****read term info[8]******
the term share prefixlength is :1
term's own stuffix is:??
exists this term's field number is:2
the doc count contain this term is:2
the position of this term's TermFreqs within the .frq file is:3
the position of this term's TermPositions within the .prx file is:3
*****read term info[9]******
the term share prefixlength is :1
term's own stuffix is:??
exists this term's field number is:2
the doc count contain this term is:1
the position of this term's TermFreqs within the .frq file is:3
the position of this term's TermPositions within the .prx file is:3
*****read term info[10]******
the term share prefixlength is :0
term's own stuffix is:球
exists this term's field number is:2
the doc count contain this term is:1
the position of this term's TermFreqs within the .frq file is:1
the position of this term's TermPositions within the .prx file is:1
*****read term info[11]******
the term share prefixlength is :1
term's own stuffix is:??
exists this term's field number is:2
the doc count contain this term is:1
the position of this term's TermFreqs within the .frq file is:1
the position of this term's TermPositions within the .prx file is:1
*****read term info[12]******
the term share prefixlength is :1
term's own stuffix is:??
exists this term's field number is:2
the doc count contain this term is:2
the position of this term's TermFreqs within the .frq file is:1
the position of this term's TermPositions within the .prx file is:1
*****read term info[13]******
the term share prefixlength is :1
term's own stuffix is:??
exists this term's field number is:2
the doc count contain this term is:2
the position of this term's TermFreqs within the .frq file is:2
the position of this term's TermPositions within the .prx file is:2
*****read term info[14]******
the term share prefixlength is :0
term's own stuffix is:度
exists this term's field number is:0
the doc count contain this term is:1
the position of this term's TermFreqs within the .frq file is:3
the position of this term's TermPositions within the .prx file is:3
*****read term info[15]******
the term share prefixlength is :0
term's own stuffix is:搜
exists this term's field number is:0
the doc count contain this term is:2
the position of this term's TermFreqs within the .frq file is:1
the position of this term's TermPositions within the .prx file is:1
*****read term info[16]******
the term share prefixlength is :1
term's own stuffix is:??
exists this term's field number is:0
the doc count contain this term is:1
the position of this term's TermFreqs within the .frq file is:2
the position of this term's TermPositions within the .prx file is:2
*****read term info[17]******
the term share prefixlength is :0
term's own stuffix is:百
exists this term's field number is:0
the doc count contain this term is:1
the position of this term's TermFreqs within the .frq file is:1
the position of this term's TermPositions within the .prx file is:1
*****read term info[18]******
the term share prefixlength is :1
term's own stuffix is:??
exists this term's field number is:0
the doc count contain this term is:2
the position of this term's TermFreqs within the .frq file is:1
the position of this term's TermPositions within the .prx file is:1
*****read term info[19]******
the term share prefixlength is :0
term's own stuffix is:谷
exists this term's field number is:0
the doc count contain this term is:1
the position of this term's TermFreqs within the .frq file is:2
the position of this term's TermPositions within the .prx file is:2
*****read term info[20]******
the term share prefixlength is :0
term's own stuffix is:http://www.baidu.com
exists this term's field number is:1
the doc count contain this term is:1
the position of this term's TermFreqs within the .frq file is:1
the position of this term's TermPositions within the .prx file is:1
*****read term info[21]******
the term share prefixlength is :11
term's own stuffix is:g.cn
exists this term's field number is:1
the doc count contain this term is:1
the position of this term's TermFreqs within the .frq file is:1
the position of this term's TermPositions within the .prx file is:1

 

由运行结果看貌似对又貌似错为什么有些乱码呢 ，如果按照java的string的charat然后做equal操作的话两个汉字是不可能有相同的前缀的

看了下文档有看了下源码 发现lucene比较前后两个term的公共前缀使用的是UTF8的字节码

比如说"做" 转换为utf8的字节数组是

-27
-127
-102

"全"转换为utf8的字节数组时：

-27
-123
-88

term ‘做’ 和term ‘全’ 比较的话byte[0]的值是相同的。由于term‘做’作为第一个term保存 所以保存term ‘做’ 的value信息就是

【3】【-27】【-127】【-102】  就是一个String类型的保存格式 作为相邻的term ‘全’ 共享了byte【0】

那么此时term ‘全’  的stuff 字符串的就是【2】【-123】【-88】 虽然也是string格式的存储但是作为UTF8编码格式 两位的byte是不能保存汉字的 如果是纯英文的话就不会出现乱码问题 。

 

修改代码如下：

 

/****************
 *
 *Create Class:ReadTermIndex.java
 *Author:a276202460
 *Create at:2010-6-7
 */
package com.rich.lucene.io;
import org.apache.lucene.util.UnicodeUtil;
public class ReadTerminfo {
	/**
	 * @param args
	 * @throws Exception 
	 */
	public static void main(String[] args) throws Exception {
		 String indexfile = "D:/lucenetest/indexs/txtindex/index4/_0.tis";
		 IndexFileInput input = null;
		 try{
			 input = new IndexFileInput(indexfile);
			 System.out.println("term index version:"+input.readInt());
			 long termcount = input.readLong();
			 System.out.println("term count:"+termcount);
			 System.out.println("term IndexInterval:"+input.readInt());
			 System.out.println("term SkipInterval:"+input.readInt());
			 System.out.println("term MaxSkipLevels:"+input.readInt());
			 int doccount = 0;
			 int prefixlength = 0;
			  
			 
			 String termvalue = null;
			 byte[] lasttermbyte = null;
			 int stufflenth;
			 for(long i = 0 ;i < termcount;i++){
				 System.out.println("*****read term info["+i+"]******");
				 prefixlength = input.readVInt();
				 System.out.println("the term share prefixlength is :"+prefixlength);
				 stufflenth = input.readVInt();
				 byte[] stuffbyte = new byte[stufflenth];
				 input.readBytes(stuffbyte, 0, stufflenth);
				 
				 if(prefixlength == 0){
					 termvalue = new String(stuffbyte,"UTF-8");
					 lasttermbyte = stuffbyte;
				 }else{
					 byte[] termbyte = new byte[prefixlength+stufflenth];
					 System.arraycopy(lasttermbyte, 0, termbyte, 0, prefixlength);
					 System.arraycopy(stuffbyte, 0, termbyte, prefixlength, stufflenth);
					 termvalue = new String(termbyte,"UTF-8");
					 lasttermbyte = termbyte;
				 }
				 System.out.println("term's value is:"+termvalue);
				 System.out.println("exists this term's field number is:"+input.readVInt());
				 doccount = input.readVInt();
				 System.out.println("the doc count contain this term is:"+doccount);
				 System.out.println("the position of this term's TermFreqs within the .frq file is:"+input.readVLong());
				 System.out.println("the position of this term's TermPositions within the .prx file is:"+input.readVLong());
				 if(doccount >= 16)
				 System.out.println("the position of this term's SkipData within the .frq file is:"+input.readVInt());
			 }
			 
		 }finally{
			 input.close();
		 }
   
	}
} 

 

运行结果：

 

term index version:-4
term count:22
term IndexInterval:128
term SkipInterval:16
term MaxSkipLevels:10
*****read term info[0]******
the term share prefixlength is :0
term's value is:做
exists this term's field number is:2
the doc count contain this term is:1
the position of this term's TermFreqs within the .frq file is:0
the position of this term's TermPositions within the .prx file is:0
*****read term info[1]******
the term share prefixlength is :1
term's value is:全
exists this term's field number is:2
the doc count contain this term is:1
the position of this term's TermFreqs within the .frq file is:1
the position of this term's TermPositions within the .prx file is:1
*****read term info[2]******
the term share prefixlength is :1
term's value is:内
exists this term's field number is:2
the doc count contain this term is:1
the position of this term's TermFreqs within the .frq file is:1
the position of this term's TermPositions within the .prx file is:1
*****read term info[3]******
the term share prefixlength is :1
term's value is:国
exists this term's field number is:2
the doc count contain this term is:1
the position of this term's TermFreqs within the .frq file is:1
the position of this term's TermPositions within the .prx file is:1
*****read term info[4]******
the term share prefixlength is :1
term's value is:大
exists this term's field number is:2
the doc count contain this term is:2
the position of this term's TermFreqs within the .frq file is:1
the position of this term's TermPositions within the .prx file is:1
*****read term info[5]******
the term share prefixlength is :1
term's value is:度
exists this term's field number is:2
the doc count contain this term is:1
the position of this term's TermFreqs within the .frq file is:2
the position of this term's TermPositions within the .prx file is:2
*****read term info[6]******
the term share prefixlength is :1
term's value is:引
exists this term's field number is:2
the doc count contain this term is:2
the position of this term's TermFreqs within the .frq file is:1
the position of this term's TermPositions within the .prx file is:1
*****read term info[7]******
the term share prefixlength is :0
term's value is:搜
exists this term's field number is:2
the doc count contain this term is:2
the position of this term's TermFreqs within the .frq file is:3
the position of this term's TermPositions within the .prx file is:3
*****read term info[8]******
the term share prefixlength is :1
term's value is:擎
exists this term's field number is:2
the doc count contain this term is:2
the position of this term's TermFreqs within the .frq file is:3
the position of this term's TermPositions within the .prx file is:3
*****read term info[9]******
the term share prefixlength is :1
term's value is:最
exists this term's field number is:2
the doc count contain this term is:1
the position of this term's TermFreqs within the .frq file is:3
the position of this term's TermPositions within the .prx file is:3
*****read term info[10]******
the term share prefixlength is :0
term's value is:球
exists this term's field number is:2
the doc count contain this term is:1
the position of this term's TermFreqs within the .frq file is:1
the position of this term's TermPositions within the .prx file is:1
*****read term info[11]******
the term share prefixlength is :1
term's value is:百
exists this term's field number is:2
the doc count contain this term is:1
the position of this term's TermFreqs within the .frq file is:1
the position of this term's TermPositions within the .prx file is:1
*****read term info[12]******
the term share prefixlength is :1
term's value is:的
exists this term's field number is:2
the doc count contain this term is:2
the position of this term's TermFreqs within the .frq file is:1
the position of this term's TermPositions within the .prx file is:1
*****read term info[13]******
the term share prefixlength is :1
term's value is:索
exists this term's field number is:2
the doc count contain this term is:2
the position of this term's TermFreqs within the .frq file is:2
the position of this term's TermPositions within the .prx file is:2
*****read term info[14]******
the term share prefixlength is :0
term's value is:度
exists this term's field number is:0
the doc count contain this term is:1
the position of this term's TermFreqs within the .frq file is:3
the position of this term's TermPositions within the .prx file is:3
*****read term info[15]******
the term share prefixlength is :0
term's value is:搜
exists this term's field number is:0
the doc count contain this term is:2
the position of this term's TermFreqs within the .frq file is:1
the position of this term's TermPositions within the .prx file is:1
*****read term info[16]******
the term share prefixlength is :1
term's value is:歌
exists this term's field number is:0
the doc count contain this term is:1
the position of this term's TermFreqs within the .frq file is:2
the position of this term's TermPositions within the .prx file is:2
*****read term info[17]******
the term share prefixlength is :0
term's value is:百
exists this term's field number is:0
the doc count contain this term is:1
the position of this term's TermFreqs within the .frq file is:1
the position of this term's TermPositions within the .prx file is:1
*****read term info[18]******
the term share prefixlength is :1
term's value is:索
exists this term's field number is:0
the doc count contain this term is:2
the position of this term's TermFreqs within the .frq file is:1
the position of this term's TermPositions within the .prx file is:1
*****read term info[19]******
the term share prefixlength is :0
term's value is:谷
exists this term's field number is:0
the doc count contain this term is:1
the position of this term's TermFreqs within the .frq file is:2
the position of this term's TermPositions within the .prx file is:2
*****read term info[20]******
the term share prefixlength is :0
term's value is:http://www.baidu.com
exists this term's field number is:1
the doc count contain this term is:1
the position of this term's TermFreqs within the .frq file is:1
the position of this term's TermPositions within the .prx file is:1
*****read term info[21]******
the term share prefixlength is :11
term's value is:http://www.g.cn
exists this term's field number is:1
the doc count contain this term is:1
the position of this term's TermFreqs within the .frq file is:1
the position of this term's TermPositions within the .prx file is:1

 

Tis - term 的详细信息存储

Tii – term 详细信息的索引文件（标识详细信息的页索引 每 128 个 term 在 tii 文件中建立一个索引项）

两个文件的头信息都是一样的

TIVersion --> UInt32   文件的格式版本号

TermCount --> UInt64   文件中保存的term 的数量 （tis 中就是此段索引中的所有分隔的term （项）的数量，不论源来自哪个field,tii 文件中记录的也是此文件中term 的数量但是不是全部，是每页的最后一项（第一页为空最后一页没有记录，128 （IndexInterval ）个term 为一页）

IndexInterval --> UInt32 （每页存储的term 数量 ）

SkipInterval --> UInt32

MaxSkipLevels --> UInt32

SkipInterval 和 MaxSkipLevels 的意义和其他的文件存储有关系，现在还不知道具体的含义，但是和查看TIS,TII 文件的结构没有关系，以后学习frq ，prx 文件的结构的时候在检验这个标识的意思

头信息完后就是每个term 的具体信息了

TermInfo --> <Term, DocFreq, FreqDelta, ProxDelta, SkipDelta>

Term --> <PrefixLength, Suffix, FieldNum>

PrefixLength 表示后面一个 term 共享的前面 term 的 byte 长度（ utf8 ）

Suffix 表示自己独有的后缀信息 文档中说是字符串，对英文来说没有异议，对中文的话就可能是一个不完整的字符，是长度和后缀 utf8 字节

FieldNum term 来源的 field number

DocFreq 出现此 term 的 document 的数量

FreqDelta frq 文件中词 term 的位置（具体此位置的信息还得接下来看 frq 文件）

ProxDelta, SkipDelta 和 FreqDelta 意思差不多也是位置信息，指定了位置也就对此位置的信息建立了指针也是一个索引

 

两个文件的内容格式：

 

 

 

 

图中在tis保存第一个term的时候tii保存了一个空的term信息进去

如果tis刚好存了128*n个数据的话 那么最后一页的末项term是不会被记录到tii文件中的。接下来将frq，prx，nrm的信息读取完以后 了解lucene整个查询检索的过程和索引创建的结构就很清楚了。

内容都是边学边写到博客的，欢迎拍砖指正。

 

一洽客服系统
关注
  • 0
    点赞
  • 踩
  • 0
    收藏
    觉得还不错? 一键收藏
  • 2
    评论
专栏目录
lucene索引结构(四)-词典(Term Dictionary)索引文件结构分析
wangzhengnb的专栏
07-26 7314
搜索引擎检索系统索引的核心是什么？     显然是反向索引。     好了，现在我们开始介绍反向索引部分。 0. 一些说明     反向索引分成两部分，如下图(图来自《信息检索导论》)：          左面是词项词典(Term Dictionary)，右边是倒排记录表(Posting)。          在Lucene中，词典和倒排是分开存储的，词典存储在
lucene tis和 tii 文件
benbendy1984的专栏
06-26 3926
        在lucene 中使用 tis 保存了所有term的信息，为了加速检索，还保存了tii文件，他是tis文件的索引，下面图简单的表示了他们之间的关系 上图的左边表示tis中保存的所有的term ,右边tii是保存需要索引的term以及在tis中的位置，这样通过检索tii文件，定位到我们需要查找的term的大概位置，再通关过遍历(或者二分)找到term。 这样做好像比直接对
lucene结构详解之五词典(tis)及词典索引(tii)信息
july_2的专栏
09-19 1161
反向信息是索引文件的核心，也即反向索引。  反向索引包括两部分，左面是词典(Term Dictionary)，右面是倒排表(Posting List)。  在 Lucene 中，这两部分是分文件存储的，词典是存储在 tii，tis中的，倒排表又包括两部分， 一部分是文档号及词频，保存在 frq中，一部分是词的位置信息，保存在 prx中。    Term Dictionary (tii,
CLucene中文件.tii文件和.tis文件格式
wenxinfly的专栏
10-22 570
词典(tis)及词典索引(tii)信息 在词典中，所有的词是按照字典顺序排序的。 词典文件(tis) TermCount：词典中包含的总的词数 IndexInterval：为了加快对词的查找速度，也应用类似跳跃表的结构，假设IndexInterval为4，则在词典索引(tii)文件中保存第4个，第8个，第12个词，这样可以加快在词典文件中查找词的速度。 SkipInterval...
Lucene系列-index弹性扩展
机器不学习
11-19 804
Lucene系列-index扩展Shard与Replica Shard：index数据分片.当索引数据到达百亿级别的,单份数据索引读取和搜索都是非常耗时间的,即使在用earlytermination的情况下,latency也在100ms左右. 数据分片成shard,可以快速的提高搜索效率. Replica：shard的备份，在搜索QPS较高的时候,Replica可以提供搜索的吞吐量. 一般包含一个p
Lucene学习（二）：Lucene的总体架构
小江专栏
12-09 652
转载：http://blog.csdn.net/lengyuhong/article/details/6218614 Lucene总的来说是： 一个高效的，可扩展的，全文检索库。全部用Java实现，无须配置。仅支持纯文本文件的索引(Indexing)和搜索(Search)。不负责由其他格式的文件抽取纯文本文件，或从网络中抓取文件的过程。 在Lucene in action
Lucene学习总结之三：Lucene的索引文件格式(3)
forfuture1978的专栏
12-10 5512
 四、具体格式4.2. 反向信息反向信息是索引文件的核心，也即反向索引。反向索引包括两部分，左面是词典(Term Dictionary)，右面是倒排表(Posting List)。在Lucene中，这两部分是分文件存储的，词典是存储在tii，tis中的，倒排表又包括两部分，一部分是文档号及词频，保存在frq中，一部分是词的位置信息，保存在prx中。Ter
vb.net2.0_Lucene_test.rar_lucene_lucene vb.n_lucene.net vb
09-24
标题中的“vb.net2.0_Lucene_test.rar_lucene_lucene vb.n_lucene.net vb”表明这是一个关于使用VB.NET 2.0版本实现Lucene搜索引擎的测试项目。Lucene是一个高性能、全文本搜索库，广泛应用于Java开发，而这里则是将...
java_lucene_bus.rar_Java bus_javabus.com
09-20
标题中的“java_lucene_bus.rar”表明这是一个使用Java编程语言，并结合了Lucene库来实现的公交搜索系统。Lucene是Apache软件基金会的开源全文搜索引擎库，它提供了文本分析、索引创建、搜索功能，是Java开发人员...
chinese_lucene.rar_Lucene C#_Lucene for .net_PanGu4Lucene_lucene
最新发布
09-24
本文将探讨“chinese_lucene.rar”这个压缩包中的内容，特别是针对C#环境的中文分词和Lucene的集成应用。 首先，我们关注的是“c#中文分词类库”。在处理中文文本时，分词是至关重要的一步，因为它决定了信息检索的...
paoding_analysis.rar_PaodingAnalysis_lucene paoding_paodi
09-14
《 PaodingAnalysis与Lucene中文索引技术解析》 在当今大数据时代，高效的信息检索成为企业和个人处理海量信息的关键。Lucene，作为一个强大的全文搜索引擎库，被广泛应用在各种信息检索系统中。而针对中文处理，...
java+lucene)1236.rar_Lucene 搜索_Luncene_lucene_lucene web
09-23
【标题】"java+lucene)1236.rar_Lucene 搜索_Luncene_lucene_lucene web" 提供的信息表明这是一个使用Java和Lucene框架实现的搜索系统，特别是针对公交数据的搜索。Lucene是Apache软件基金会的一个开源全文检索库，...
Lucene 7.5.0 索引文件之cfs&&cfe
q364367207的专栏
07-10 2139
索引文件.cfs、.cfe被称为复合（compound）索引文件，在IndexWriterConfig可以配置是否生成复合索引文件，默认开启。   在前面的文章中，我们介绍了其他的索引文件，而复合索引文件则是将这些索引文件的数据组合到一个文件中，这种设计的目的是为了减少文件描述符的使用。   正如上文中描述的那样，复合文件实质是索引文件的组合，意思是无论是否设置了使用复合文件，总是先生成非复合...
Solr详解
crazzy_lp的博客
11-20 446
1.  全文索引 什么叫做全文索引呢？这要从我们生活中的数据说起。 我们生活中的数据总体分为两种：结构化数据和非结构化数据：结构化数据—指具有固定格式或有限长度的数据，如数据库，元数据等；非结构化数据—指不固定长或无固定格式的数据，如邮箱，Word文档等；半结构化数据—如XML、HTML等，当根据需要可按照结构化数据来处理，也可抽取出纯文本按非机构化数据来处理；非结构化数据有一种叫法叫...
搜索系统18：lucene索引文件结构
北漂程序员-中中
11-16 1329
1.多文件索引与复合索引的区别 2.segments(段)文件 3.段元数据 .si 保存了索引段的元数据信息 4.fnm(域名)文件 相当于solr里的schemal.xml文件 5.项词典(.TIS、TII）文件 lucene老版本的项词典是tis文件，新版本是tim文件，如图: 我没在这个文件里找到字段名，可能已经转成了数值。 《lu
不选择使用Lucene的6大原因
热门推荐
我要去桂林-田春峰
03-22 1万+
不选择使用Lucene的6大原因     Lucene是开放源代码的全文搜索引擎工具包，凭借着其强劲的搜索功能和简单易用的实现，在国内已经很普及，甚至一度出现了言搜索必称Lucene的盛景。上个月Lucene的开发团队发布了 Java Lucene 2.3.1 ，相信很多朋友们都用上了。在国内对Lucene的介绍可以分为3块儿： 第一类是：以车东 的Lucene：基于Java的全文检索引擎
Lucene4.10文件格式说明
jianjun200607的专栏
09-14 1335
Lucene索引的最终结果就是生成一堆的文件，要想了解Lucene的原理首先得弄明白Lucene的文件含义，下面解释下Lucene的主要文件含义： .fdt文件：存储文档的域数据（fdt只存放那些需要被存储的域数据），文件结构包括：fieldCount(当前文档包含域的个数)，fieldNum(当前域号)，bits（标志位），value(当前域的数据值) .fdx文件：域数据索引文件，包含了对
《开发自己的搜索引擎》读书笔记——索引的建立
仰起脸笑得像满月
04-12 3211
Lucene的Document。 Document的意义为文档，在Lucene中，它代表一种逻辑文件。Lucene本身无法对物理文件建立索引，而只能识别并处理Document类型的文件。在某些时候可以将一个Document与一个物理文件进行对应，用一个Document来代替一个物理文件，然而更多的时候，Document和物理的文件没有关系，它作为一种数据源的集合，向Lucene提供原始的要索引的
边学边记（五） lucene索引存储结构详解二（segments_N）
一洽秦江利的专栏
06-02 5320
lucene索引结构，segment_n 文件结构，lucene字符串存储
评论 2
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值