How can I inspect the internal flink timestamp for an item in a stream using the Processing Time Model?

杀马特。学长 韩版系。学妹 提交于 2019-12-12 04:41:14

问题


I am looking to tag the data in my stream with the time it arrived in flink so that I can perform some calculations. I recognize when using the Event Time Model I would have direct control over that, but I was hoping there was some easy way to discover the Timestamp flink was using when making Window decisions on a stream.


回答1:


Flink supports three modes to work with time:

  • Processing Time: Events are processed with respect to the current time of each operator
  • Event Time: Events are processes with respect to a timestamp which was manually assigned.
  • Ingestion Time: Events are processed with respect to a timestamp that is automatically assigned when the event is ingested by Flink.

From your description, it seems that you are looking for Ingestion Time. Internally, ingestion time is handed just like event time, but the difference is that the timestamps are automatically assigned (and watermarks are automatically generated).

The time mode is set via the StreamExecutionEnvironment.setStreamTimeCharacteristic() method.

Event time and ingestion time timestamps are not exposed to any function except for the ProcessFunction. A ProcessFunction can read the timestamp of an event in ProcessFunction.processElement() via the Context.timestamp() method.



来源:https://stackoverflow.com/questions/46498772/how-can-i-inspect-the-internal-flink-timestamp-for-an-item-in-a-stream-using-the

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!