google-cloud-datastore

I have 400,000 entities of a certain type, and I'd like to perform a simple operation on each of them (adding a property). I can't process them serially because it would take forever. I don't want to use the MapReduce library because it is complicated and overwhelming. Basically I'd like to create 100 tasks on the taskqueue, each task taking a segment of ~4,000 entities and performing this operation on each one. Hopefully this wouldn't take more than a few minutes to process all 400k entities, when all tasks are executing in parallel. However, I'm not sure how to use GAE queries to do this. My

What should I know about FetchOptions withLimit, prefetchSize and chunkSize? The docs say the following: prefetchSize is the number of results retrieved on the first call to the datastore. chunkSize determines the internal chunking strategy of the Iterator returned by PreparedQuery.asIterator(FetchOptions) ... prefetchSize and chunkSize have no impact on the result of the PreparedQuery, but rather only the performance of the PreparedQuery. I'm not too sure how to use that in anger. What are the performance implications of different options? Any examples of how changes you've made have improved

This is related to a previous question of mine , but with new information. I'm trying to configure the bulkdownloader to download data from my Java appengine app in such a way that a list of owned objects is transformed into a nested XML path inside each parent object. I have been using the export transform transform.child_node_from_list . However, the list property is never actually being downloaded and passed to that transform! This seems similar to the effect you get when looking at an entity in the appengine datastore viewer - the parent entity just doesn't show the list property at all.