Dataflow pipeline and pubsub emulator

问题

I'm trying to setup my development environment. Instead of using google cloud pubsub in production, I've been using the pubsub emulator for development and testing. To achieve this I set the following environment variable:

export PUBSUB_EMULATOR_HOST=localhost:8586

This worked for the python google pubsub library but when I switched to using java apache beam for google dataflow, the pipeline still points to production google pubsub. Is there a setting, environment variable or method on the pipeline that I need to set so that the pipeline reads for the local pubsub emulator?

回答1:

I found the solution in the PubsubOptions interface, and extending it for my own PipelineOptions implementation. Then using setPubsubRootUrl() to set it to localhost:port of the emulator.

来源：https://stackoverflow.com/questions/43354599/dataflow-pipeline-and-pubsub-emulator