neo4j handaling a long transation on one iterator

Question

I would like to go over all my "user" nodes in the graph and do some action on them.

The problem is that I have too many "user" nodes, and I can't hold all of them on the heap.

so, doing something like:

try (Transaction tx = graphDb.beginTx())
{
   Iterator<Node> n_column = autoNodeIndex.get( "type", "user" );   


   while (n_column.hasNext()) {
     //create relationship for node...
   }
   tx.success();
}

will cause GC overhead exception.

How can I split it to few transactions , but on the same iterator?

I thought about nested transaction, but the manual says differently. Nested transaction will roll the local memory to the top transactions.

Answer 1

Just add a counter and, commit and start a new transaction every 50.000 nodes.

   Transaction tx = graphDb.beginTx();
   Iterator<Node> n_column = autoNodeIndex.get( "type", "user" );   

   int counter=0;
   while (n_column.hasNext()) {
     //create relationship for node...
     if (++counter % 50000 == 0) {
        tx.success(); tx.close();
        tx = graphDb.beginTx();
     }
   }
   tx.success(); 
   tx.close();