How to do Async Http Call with Apache Beam (Java)?

Question

Input PCollection is http requests, which is a bounded dataset. I want to make async http call (Java) in a ParDo , parse response and put results into output PCollection. My

Answer 1

The issue that your hitting is that your outputting outside the context of a processElement or finishBundle call.

You'll want to gather all your outputs in memory and output them eagerly during future processElement calls and at the end within finishBundle by blocking till all your calls finish.