As one poster above suggested, use profiler to measure the memory (and/or cpu) usage of certain parts of your program rather than trying to guess it. You may be surprised at what you find!
There is an added benefit to that as well. You'll understand about your programming language and your application more.
I use VisualVM for profiling and recommend it greatly. It comes with jdk/jre distribution.
