Writing to Google Cloud Storage from PubSub using Cloud Dataflow using DoFn

后端 未结 2 1355
温柔的废话
温柔的废话 2020-12-13 15:30

I am trying write Google PubSub messages to Google Cloud Storage using Google Cloud Dataflow. I know that TextIO/AvroIO do not support streaming pipelines. However, I read i

2条回答
  •  遥遥无期
    2020-12-13 16:26

    As of Beam 2.0, TextIO/AvroIO do support writing unbounded collections - see documentation, in particular, you have to specify withWindowedWrites().

提交回复
热议问题