Access HTTP service running in GKE from Google Dataflow

Question

I have an HTTP service running on a Google Container Engine cluster (behind a kubernetes service).

My goal is to access that service from a Dataflow job running on t

Answer 1

The Dataflow job running on GCP will not be part of the Google Container Engine cluster, so it will not have access to the internal cluster DNS by default.

Try setting up a load balancer for the service that you want to expose which knows how to route the "external" traffic to it. This will allow you to connect to the IP address directly from a Dataflow job executing on GCP.