Alex Rivera | Logout

File Streaming in Java

Asked 2011-01-18T20:11:03.810
9

I'm currently developing 3D graphics application using JOGL (Java OpenGL binding). In brief, I have a huge landscape binary file. Due to its size, I have to stream terrain chunks in the run-time. Therefore, we explicitly see the random access concern. I have already finished the first (and dirty :)) implementation (perhaps it is multi-threaded), where I'm using a foolish approach... Here is the initialization of it:

dataInputStream = new DataInputStream(new BufferedInputStream(fileInputStream,4 * 1024);
dataInputStream.mark(dataInputStream.available());

And when I need to read (stream) special chunk (I already know its "offset" in the file) I'm performing the following (shame on me :)):

dataInputStream.reset();
dataInputStream.skipBytes(offset);
dataInputStream.read(whatever I need...);

Since I had little experience that was the first thing I could think about :) So, until now I have read 3 useful and quite interesting articles (I'm suggesting you to read them, perhaps if you are interested in this topic)

  1. Byte Buffers and Non-Heap Memory - Mr. Gregory seems to be literate in Java NIO.

  2. Java tip: How to read files quickly [http://nadeausoftware.com/articles/2008/02/java_tip_how_read_files_quickly] - That's an interesting benchmark.

  3. Articles: Tuning Java I/O Performance [http://java.sun.com/developer/technicalArticles/Programming/PerfTuning/] - Simple Sun recommendations, but please scroll down and have a look at "Random Access" section there; they show a simple implementation of RandomAccessFile (RAF) with self-buffering improvement.

Mr. Gregory provides several *.java files in the end of his article. One of them is a benchmarking between FileChannel + ByteBuffer + Mapping (FBM) a

java streaming random-access
Edit
Report

1 Answer

1

Have you noticed that if you run a program, then close it, then run it again it starts up much faster than the second time? This happens because the OS has cached the parts of the files that were accessed in the first run, and doesn't need to access the disk for them. Memory mapping a file essentially allows a program access to these buffers, thus minimizing copies made when reading it. Note that memory mapping a file does not cause it to be read whole into memory; the bits and pieces that you read are read from disk on-demand. If the OS determines that there is low memory, it may decide to free up some parts of the mapped file from memory, and leave them on disk.

Edit: What you want is FileInputStream.getChannel().map(), then adapt that to an InputStream, then connect that to the DataInputStream.

answered 2011-01-19T22:02:43.527

Your Answer