See Saxon support for streaming mode. http://www.saxonica.com/html/documentation/sourcedocs/streaming/
If this streaming mode isn't for you, you can try to use tiny tree mode of Saxon, which is optimized for smaller memory usage. (It is default anyway)
