How to stream data to database BLOB using Hibernate (no in-memory storing in byte[])

Question

I\'m looking for a way to stream binary data to/from database. If possible, i\'d like it to be done with Hibernate (in database agnostic way). All solutions I\'ve found invo

Answer 1

For those looking for same thing.

My bad, the code works as supposed to (streams without trying to copy to memory) for PostgreSQL (and probably lots of others). The inner work of Hibernate depends on selected dialect. The one I used in the first place overrides direct use of streams in favor of BinaryStream backed by byte[].

Also there are no problems with performance, since it loads only OID (number) in case of PostgreSQL, and probably lazy loads data in case of other dialects (including byte[] implementation). Just ran some dirty tests, no visible difference in 10000 loads of entity with and without binary data field.

Storing data in database seems to be slower than just saving it on disk as external file though. But it saves you a lot of headache when backing up, or dealing with limitations of particular file system, or concurrent updates, etc. But it's an off-topic.

Answer 2

You are storing the Blob in your POJO SimpleBean. This means if the blob is larger than your heap space, anytime you work with this object or access the data field, you're going to get the OutOfMemoryError because the entire thing is loaded into memory.

I don't think there's a way to set or get a database field using a Stream in hibernate, and HQL inserts only into SELECT statements.

What you may have to do is remove the data field from the SimpleBean object so that it's not stored in memory when you load or save. But when you need to save a blob, you can use hibernate's save() to create the row, then use a jdbc PreparedStatement and the setBinaryStream() method. When you need to access the stream, you can use hibernate's load() method to get a SimpleBean object and do a jdbc select to get a ResultSet then use the getBinaryStream() method to read the blob. The docs for setBinaryStream() say:

The data will be read from the stream as needed until end-of-file is reached.

So the data won't be stored entirely in memory.

Answer 3

Your solution using Hibernate's lobHelper should work, but you may need to make sure that the use of streams is enforced. Set property hibernate.jdbc.use_streams_for_binary = true This is a system-level property, so it has to be set at startup (I defined it on the command line during testing:

java -Dhibernate.jdbc.use_streams_for_binary=true blobTest

You can prove it's changed in your code:

Object prop = props.get("hibernate.jdbc.use_streams_for_binary");
System.out.println("hibernate.jdbc.use_streams_for_binary" + "/" + prop);