Google Datastore combine (union) multiple sets of entity results to achieve OR condition

Question

I am working with NodeJS on Google App Engine with the Datastore database.

Due to the fact that Datastore does not have support the OR operator, I need to run multip

Answer 1

IMHO the most efficient way would be to use Keys-only queries in the 1st stage, then perform the combination of the keys obtained into a single list (including deduplication), followed by obtaining the entities simply by key lookup. From Projection queries:

Keys-only queries

A keys-only query (which is a type of projection query) returns just the keys of the result entities instead of the entities themselves, at lower latency and cost than retrieving entire entities.

It is often more economical to do a keys-only query first, and then fetch a subset of entities from the results, rather than executing a general query which may fetch more entities than you actually need.

Here's how to create a keys-only query:

const query = datastore.createQuery()
  .select('__key__')
  .limit(1);

This method addresses several problems you may encounter when trying to directly combine lists of entities obtained through regular, non-keys-only queries:

• you can't de-duplicate properly because you can't tell the difference between different entities with identical values and the same entity appearing in multiply query results
• comparing entities by property values can be tricky and is definitely slower/more computing expensive than comparing just entity keys
• if you can't process all the results in a single request you're incurring unnecessary datastore costs for reading them without actually using them
• it is much simpler to split processing of entities in multiple requests (via task queues, for example) when handling just entity keys

There are some disadvantages as well:

• it may be a bit slower because you're going to the datastore twice: once for the keys and once to get the actual entities
• you can't take advantage of getting just the properties you need via non-keys-only projection queries