Apache Beam Counter/Metrics not available in Flink WebUI

问题

I'm using Flink 1.4.1 and Beam 2.3.0, and would like to know is it possible to have metrics available in Flink WebUI (or anywhere at all), as in Dataflow WebUI ?

I've used counter like:

import org.apache.beam.sdk.metrics.Counter;
import org.apache.beam.sdk.metrics.Metrics;
...
Counter elementsRead = Metrics.counter(getClass(), "elements_read");
...
elementsRead.inc();

but I can't find "elements_read" counts available anywhere (Task Metrics or Accumulators) in Flink WebUI. I thought this will be straightforward after BEAM-773.

回答1:

Once you have selected a job in your dashboard, you will see the DAG for that job and below the DAG there are a list of tabs.

Click on "Task Metrics" Tab
Click on a box of your DAG
Click on the Add Metric Button, to show that operator metric

回答2:

If your pipeline is running in detached mode, metrics are not supported to be queried. Refer this.

public class FlinkDetachedRunnerResult implements PipelineResult {

  FlinkDetachedRunnerResult() {}

  @Override
  public State getState() {
    return State.UNKNOWN;
  }

  @Override
  public MetricResults metrics() {
    throw new UnsupportedOperationException("The FlinkRunner does not currently support metrics.");
  }

  @Override
  public State cancel() throws IOException {
    throw new UnsupportedOperationException("Cancelling is not yet supported.");
  }

  @Override
  public State waitUntilFinish() {
    return State.UNKNOWN;
  }

  @Override
  public State waitUntilFinish(Duration duration) {
    return State.UNKNOWN;
  }

  @Override
  public String toString() {
    return "FlinkDetachedRunnerResult{}";
  }
}

However, I was able to view the metrics using slf4j reporter

来源：https://stackoverflow.com/questions/49013889/apache-beam-counter-metrics-not-available-in-flink-webui

标签

java

apache-flink

metrics

apache-beam