java的文件流,Java中的文件流

I'm currently developing 3D graphics application using JOGL (Java OpenGL binding). In brief, I have a huge landscape binary file. Due to its size, I have to stream terrain chunks in the run-time. Therefore, we explicitly see the random access concern. I have already finished the first (and dirty :)) implementation (perhaps it is multi-threaded), where I'm using a foolish approach... Here is the initialization of it:

dataInputStream = new DataInputStream(new BufferedInputStream(fileInputStream,4 * 1024);

dataInputStream.mark(dataInputStream.available());

And when I need to read (stream) special chunk (I already know its "offset" in the file) I'm performing the following (shame on me :)):

dataInputStream.reset();

dataInputStream.skipBytes(offset);

dataInputStream.read(whatever I need...);

Since I had little experience that was the first thing I could think about :)

So, until now I have read 3 useful and quite interesting articles (I'm suggesting you to read them, perhaps if you are interested in this topic)

Java tip: How to read files quickly [http://nadeausoftware.com/articles/2008/02/java_tip_how_read_files_quickly] - That's an interesting benchmark.

Articles: Tuning Java I/O Performance [http://java.sun.com/developer/technicalArticles/Programming/PerfTuning/] - Simple Sun recommendations, but please scroll down and have a look at "Random Access" section there; they show a simple implementation of RandomAccessFile (RAF) with self-buffering improvement.

Mr. Gregory provides several *.java files in the end of his article. One of them is a benchmarking between FileChannel + ByteBuffer + Mapping (FBM) and RAF. He says that he noticed 4x speedup when using FBM compared to RAF. I have ran this benchmark in the following conditions:

The offset (e. g. place of access) is generated randomly (in the file scope, e. g. 0 - file.length());

File size is 220MB;

1 000 000 accesses (75% reads and 25% writes)

The results were stunning:

~ 28 sec for RAF!

~ 0.2 sec for FBM!

However, his implementation of RAF in this benchmark doesn't have self-buffering (the 3rd article tells about one), so I guess it is the "RandomAccessFile.seek" method calling, who drops performance so hard.

Ok, now after all those things I've learnt there is 1 question and 1 dilemma :)

Question: When we are mapping a file using "FileChannel.map" does Java copy the whole file contents into the MappedByteBuffer? Or does it just emulate it? If it copies, then using FBM approach is not suitable for my situation, is it?

Dilemma: Depends on your answers on the question...

If mapping copies a file, then it seems like I have only 2 possible solutions to go: RAF + self-buffering (the one from the 3rd article) or make use of position in FileChannel (not with mapping)... Which one would be better?

If mapping doesn't copy a file, then I have 3 options: two previous ones and FBM itself.

Edit: Here is one more question. Some of you here say that mapping doesn't copy file into MappedByteBuffer. Ok then, why can't I map 1GB file then, I'm getting "failed to map" message...

P. S. I would like to receive a fulfilled answer with advices, since I'm not able to find the consistent information over this topic in the internet.

Thanks :)

解决方案

No, the data is not buffered. A MappedByteBuffer references the data using a pointer. In other words, the data is not copied, it is simply mapped into physical memory. See the API docs if you haven't already.

A memory-mapped file is a segment of

virtual memory which has been assigned

a direct byte-for-byte correlation

with some portion of a file or

file-like resource. This resource is

typically a file that is physically

present on-disk, but can also be a

device, shared memory object, or other

resource that the operating system can

reference through a file descriptor.

Once present, this correlation between

the file and the memory space permits

applications to treat the mapped

portion as if it were primary memory.

If you are going to be reading data quite frequently, it is a good idea to at least cache some of it.