How to memory map (mmap) a linux block device (e.g. /dev/sdb) in Java?

Question

I can read/write a linux block device with Java using java.nio. The following code works:

Path fp = FileSystems.getDefault().getPath("/dev", "sdb");
FileChannel fc = null;
try {
  fc = FileChannel.open(fp, EnumSet.of(StandardOpenOption.READ, StandardOpenOption.WRITE));
} catch (Exception e) {
  System.out.println("Error opening file: " + e.getMessage());
}
ByteBuffer buf = ByteBuffer.allocate(50);
try {
  if(fc != null)
    fc.write(buf);
} catch (Exception e) {
  System.out.println("Error writing to file: " + e.getMessage());
}

However, memory mapping does not work. The following code fails:

MappedByteBuffer mbb = null;
try {
  mbb = fc.map(FileChannel.MapMode.READ_WRITE, 0, 100);
} catch (IOException e) {
  System.out.println("Error mapping file: " + e.getMessage());
}

This fails with Error:

java.io.IOException: Invalid argument
    at sun.nio.ch.FileDispatcherImpl.truncate0(Native Method)
    at sun.nio.ch.FileDispatcherImpl.truncate(FileDispatcherImpl.java:79)
    at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:817)

Is there a work around to this? Perhaps by using a different library? I read somewhere that maybe by using JNI I could do this, but I couldn't find any sources.

Answer 1

According to the documentation the mechanics of actually mapping a file are left to the implementation. It appears that the implementation is attempting to truncate the file (maybe because the block device size is not the same as the size you specify?).

I am curious why you are reading from the block device directly (unless you are trying to write some sort of filesystem utility or something that needs to do raw I/O). If you need to read from the block device as a memory mapped file directly, you may need to write some C/C++ code to map the file and handle reading/writing to it and use a Java/JNI bridge class to bridge calls to your C/C++ code. That way you handle calling mmap() yourself and can specify any options you need. Looking at the mmap() documentation you may not be able to specify block devices on your platform (I'm guessing Linux but I could be wrong).

If you absolutely need to do this within Java you may need to do read() calls and write() calls of the appropriate length and offset.

Answer 2

I found that the easiest approach is to use JNA and a bit of sun.*/com.sun.* magic. First of all you need to wrap libc like this:

import com.sun.jna.LastErrorException;
import com.sun.jna.Library;
import com.sun.jna.Native;
import com.sun.jna.Pointer;

public interface LibC extends Library {
    LibC instance = (LibC) Native.loadLibrary("c", LibC.class);

    Pointer mmap(Pointer address, long length, 
                 int protect, int flags, int fd, 
                 long offset) throws LastErrorException;
    // implement more methods if you like
}

And then you are almost done! All you need is to obtain file descriptor. It may be a bit tricky:

RandomAccessFile randomAccessFile = new RandomAccessFile(file, "rw");
int fd = sun.misc.SharedSecrets.getJavaIOFileDescriptorAccess()
                               .get(randomAccessFile.getFD());

That's it. Now you can call libc from java:

Pointer result = LibC.instance.mmap(
    /*pass your desired parameters along with obtained fd*/
);

Answer 3

FileChannel.map attempts to truncate the file to the specified size: See the implementation

You will need to get the size of the block device and pass that exact size to the map call.

Don't worry if the size of the actual device is larger than your available memory. The OS will handle swapping pages in and out of memory as needed.