Force JVM to do all IO without page cache (e.g. O_DIRECT)

Question

I\'m doing some benchmarks of an application written in Java. It is very important for the experiments that the results are not influenced by the page cache (I\'m using linu

Answer 1

You can use O_DIRECT under Java, leveraging the Java Native Access (JNA). Implementations for InputStream and OutputStream, with O_DIRECT enabled, are provided here.

Answer 2

  "The thing that has always disturbed me about O_DIRECT is that the
   whole interface is just stupid, and was probably designed by a deranged
   monkey on some serious mind-controlling substances  [*]."

[*] In other words, it's an Oracleism.

-- Linus Torvalds from Transmeta, 11 May 2002

Check NOTES section of the man 2 open:

O_DIRECT

The O_DIRECT flag may impose alignment restrictions on the length and address of userspace buffers and the file offset of I/Os. In Linux alignment restrictions vary by file system and kernel version ....

Under Linux 2.4, transfer sizes, and the alignment of the user buffer and the file offset must all be multiples of the logical block size of the file system. Under Linux 2.6, alignment to 512-byte boundaries suffices. ....

In summary, O_DIRECT is a potentially powerful tool that should be used with caution. It is recommended that applications treat use of O_DIRECT as a performance option which is disabled by default.

I think, there are some usages of FileInputStream in the JRE (classloader) which has reads with offsets or sizes not aligned to 512 bytes. (For Advanced Format the minimal alignment may be bigger, even 4096 bytes, or one 4K page.)

The behaviour of kernel for unaligned offsets is the grey zone, some info is here: RFC: Clarifying Direct I/O Semantics, Theodore Ts'o, tytso@mit, LWN, 2009

Other interesting discussion is here: Linux: Accessing Files With O_DIRECT (kerneltrap, 2007)

Hmm, seems like there should be fallback to buffered I/O when something with DIRECT fails. All IO operations with DIRECT are synchronous. May be some DMA effects? Or combination of O_DIRECT and mmap?

UPDATE:

Thanks for strace output. Here is the error (grep O_DIRECT, then check file descriptor operations):

28290 open("...pact/perf/TestDirectIO.class", O_RDONLY|O_DIRECT) = 11
28290 fstat(11, {st_mode=S_IFREG|0644, st_size=2340, ...}) = 0
28290 fcntl(11, F_GETFD)                = 0
28290 fcntl(11, F_SETFD, FD_CLOEXEC)    = 0
...skip
28290 stat("...pact/perf/TestDirectIO.class", {st_mode=S_IFREG|0644, st_size=2340, ...}) = 0
...skip
28290 read(11, "\312\376\272\276\0\0\0003\0\215\n\0-\0D\t\0E\0F\7\0G\n\0\3\0D\10\0H\n"..., 1024) = 1024
28290 read(11, 0x7f1d76d23a00, 1316)    = -1 EINVAL (Invalid argument)

Unaligned read size results in EINVAL error. Your classfile is 2340 bytes long, it is 1024+1316 bytes, which is not aligned.

Answer 3

Old post, but I recently wrote a small library called Jaydio hoping to solve this exact problem. Maybe you will find it useful.