How to specify insertId when spreaming insert to BigQuery using Apache Beam

后端 未结 2 1460
长发绾君心
长发绾君心 2021-01-14 03:00

BigQuery supports de-duplication for streaming insert. How can I use this feature using Apache Beam?

https://cloud.google.com/bigquery/streaming-data-into-bigquery#d

2条回答
  •  既然无缘
    2021-01-14 03:25

    As Felipe mentioned in the comment, it seems that Dataflow is already using insertId for itself to implement "exactly once". so we can not manually specify insertId.

提交回复
热议问题