What is the use of Hibernate batch processing

Question

I am new to hibernate i have doubt in hibernate batch processing, i read some tutorial for hibernate batch processing they said

Session session = SessionFactory.openSession();
Transaction tx = session.beginTransaction();
for ( int i=0; i<100000; i++ ) 
{
  Employee employee = new Employee(.....);
  session.save(employee);

}
tx.commit();
session.close();

Hibernate will cache all the persisted objects in the session-level cache and ultimately your application would fall over with an OutOfMemoryException somewhere around the 50,000th row. You can resolve this problem if you are using batch processing with Hibernate like,

Session session = SessionFactory.openSession();
Transaction tx = session.beginTransaction();
for ( int i=0; i<100000; i++ ) 
{
 Employee employee = new Employee(.....);
 session.save(employee);
 if( i % 50 == 0 ) 
 { // Same as the JDBC batch size
   //flush a batch of inserts and release memory:
   session.flush();
   session.clear();
 }
}
tx.commit();
session.close();

My doubt is instead of initializing the session outside, why can't we initialize it in to the for loop like,

Session session = null;
Transaction tx = session.beginTransaction();
for ( int i=0; i<100000; i++ ) 
{
 session =SessionFactory.openSession()
 Employee employee = new Employee(.....);
 session.save(employee);
}
tx.commit();
session.close();

Is it correct way or not any one suggest me the correct way?

Answer 1

No. Don't initialize the session in the for loop; every time you start a new session you're starting a new batch (so you have a batch size of one your way, that is it is non-batching). Also, it would be much slower your way. That is why the first example has

if( i % 50 == 0 ) { 
  //flush a batch of inserts and release memory:
  session.flush();
  session.clear();
}

that is what "flush a batch of inserts and release memory" was for.

Answer 2

Batch Processing in Hibernate means to divide a task of huge numbers to some smaller tasks.

When you fire session.save(obj), hibernate will actually cache that object into its memory (still the object is not written into database), and would save it to database when you commit your transaction i.e when you call transactrion.commit().

Lets say you have millions of records to insert, so firing session.save(obj) would consume a lot of memory and eventually would result into OutOfMemoryException.

Solution : Creating a simple batch of smaller size and saving them to database.

if( i % 50 == 0 ) { 
  //flush a batch of inserts and release memory:
  session.flush();
  session.clear();
}

Note : In code above session.flush() would flush i.e actually save the objects into database and session.clear() would clear any memory occupied by those objects for a batch of size 50.

Answer 3

As I explained in this article, batch processing allows you to optimize writing data.

However, the usual advice of flushing and clearing the Hibernate Session is incomplete.

You need to commit the transaction at the end of the batch to avoid long-running transactions which can hurt performance and, if the last item fails, undoing all changes is going to put a lot of pressure on the DB.

Therefore, this is how you should do batch processing:

int entityCount = 50;
int batchSize = 25;

EntityManager entityManager = entityManagerFactory().createEntityManager();

EntityTransaction entityTransaction = entityManager.getTransaction();

try {
    entityTransaction.begin();

    for (int i = 0; i < entityCount; i++) {
        if (i > 0 && i % batchSize == 0) {
            entityTransaction.commit();
            entityTransaction.begin();

            entityManager.clear();
        }

        Post post = new Post(
            String.format("Post %d", i + 1)
        );

        entityManager.persist(post);
    }

    entityTransaction.commit();
} catch (RuntimeException e) {
    if (entityTransaction.isActive()) {
        entityTransaction.rollback();
    }
    throw e;
} finally {
    entityManager.close();
}