Google App Engine DB Query Memory Usage

Question

When I run a query on a large set of small objects (15k objects with only a few short string and boolean properties), without doing anything with these objects, I see my ins

Answer 1

I have reported this to the app engine team, and they seem to confirm this is actually a problem (suspected to be with the handling of cursors).

Answer 2

Each model entity has some over head.

You query returns objects as Protobufs for starters.

So you will a series of batched protobufs for the result set.

Then it is decoded. Each decoded entity includes the property names as well as the data for each entity. You have 15K entities. How big are your property names for instance.

So you have at least two copies of the result set in memory in various forms (possibly more), not including anything else you do with instances of the model class.

You code/loop has no opportunity for garbage collections, and that can/will happen later.

Have a look at tools like apptrace to help memory profiling.