File Streaming in Java

Question

I\'m currently developing 3D graphics application using JOGL (Java OpenGL binding). In brief, I have a huge landscape binary file. Due to its size, I have t

Answer 1

For a 220 MB file I would memory map the whole thing into virtual memory. The reason FBM is so fast is it doesn't actually read the data into memory, it just makes it available.

Note: when you run the test you need to compare like for like i.e. when the file is in the OS cache it will be much faster no matter how you do it. You need to repeat the test multiple times to get a reproduce able result.

Answer 2

No, the data is not buffered. A MappedByteBuffer references the data using a pointer. In other words, the data is not copied, it is simply mapped into physical memory. See the API docs if you haven't already.

A memory-mapped file is a segment of virtual memory which has been assigned a direct byte-for-byte correlation with some portion of a file or file-like resource. This resource is typically a file that is physically present on-disk, but can also be a device, shared memory object, or other resource that the operating system can reference through a file descriptor. Once present, this correlation between the file and the memory space permits applications to treat the mapped portion as if it were primary memory.

Source: Wikipedia

If you are going to be reading data quite frequently, it is a good idea to at least cache some of it.

Answer 3

Have you noticed that if you run a program, then close it, then run it again it starts up much faster the second time? This happens because the OS has cached the parts of the files that were accessed in the first run, and doesn't need to access the disk for them. Memory mapping a file essentially allows a program access to these buffers, thus minimizing copies made when reading it. Note that memory mapping a file does not cause it to be read whole into memory; the bits and pieces that you read are read from disk on-demand. If the OS determines that there is low memory, it may decide to free up some parts of the mapped file from memory, and leave them on disk.

Edit: What you want is FileInputStream.getChannel().map(), then adapt that to an InputStream, then connect that to the DataInputStream.