lucene在执行索引和检索时能同时打开多个文件,打开多个与共同使用一个有多大的区别?
在索引时能否对同一文件进行并发打开?
在检索时并发打开一个文件与公用一个打开句柄,哪个性能更好?
导致文件打开过多的可能用法:
Why am I getting an IOException that says "Too many open files"?
The number of files that can be opened simultaneously is a system-wide limitation of your operating system. Lucene might cause this problem as it can open quite some files depending on how you use it, but the problem might also be somewhere else.
-
Always make sure that you explicitly close all file handles you open, especially in case of errors. Use a try/catch/finally block to open the files, i.e. open them in the try block, close them in the finally block. Remember that Java doesn't have destructors, so don't close file handles in a finalize method -- this method is not guaranteed to be executed.
-
Use the compound file format (it's activated by default starting with Lucene 1.4) by calling IndexWriter's setUseCompoundFile(true)
-
Don't set IndexWriter's mergeFactor to large values. Large values speed up indexing but increase the number of files that need to be opened simultaneously.
-
If the exception occurs during searching, optimize your index calling IndexWriter's optimize() method after indexing is finished.
-
Make sure you only open one IndexSearcher, and share it among all of the threads that are doing searches -- this is safe, and it will minimize the number of files that are open concurently.
-
Try to increase the number of files that can be opened simultaneously. On Linux using bash this can be done by calling ulimit -n <number> .