Parsing large XML documents in JAVA

Question

I have the following problem:

I\'ve got an XML file (approx 1GB), and have to iterate up and down (i.e. not sequential; one after the other) in order to get the req

Answer 1

If you don't want to be bound by the memory limits, I certainly recommend you to use your current approach, and store everything in database.

The parsing of the XML file should be done by a SAX parser, as everybody has recommended (including me). This way you can create one object at a time, and you can immediately persist it into the database.

For the post-processing (resolving cross-references), you can use SELECTs from the database, make primary keys, indexes, etc. You can use ORM (Eclipselink, Hibernate) as well if you feel comfortable with that.

Actually I don't really recommend SQLite, it's easier to set up a MySQL server, and store the data there. Later you can even reuse the XML data (if you don't delete).