Tag huge list of elements with lat/long with large list of geolocation data
I have a huge list of geolocation events: Event (1 billion) ------ id datetime lat long And a list of point of interest loaded from open street map: POI (1 million) ------ id tag (shop, restaurant, etc.) lat long I would like to assign to each to each event the tag of the point of interest. What is the best architecture to achieve this problem? We tried using Google BigQuery but we have to do a cross join and it does not work. We are open to use any other big data system. Jeremy Lewi Using Dataflow you can do a cross join pretty easily using CoGroupByKey . Using this approach only the Event