Close window after based on element value

不羁岁月 提交于 2019-12-06 19:10:43

I don't think that this is available. There is only one Data Driven Trigger right now, elementCountAtLeast.

https://cloud.google.com/dataflow/model/triggers#data-driven-triggers

A work around for this would be to copy the sessions window function code and write a custom window function.

https://github.com/apache/beam/blob/890bc1a23f493b042f8c2de5c042970ce5ddca96/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/windowing/Sessions.java

In short, you keep assigning elements into the same window, until you see your terminating element. Then start creating a new window.

https://github.com/apache/beam/blob/890bc1a23f493b042f8c2de5c042970ce5ddca96/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/windowing/Sessions.java#L60

At present, there is no way to trigger off the content of an element, unfortunately. From the Apache Beam Docs:

Beam provides one data-driven trigger, AfterPane.elementCountAtLeast(). This trigger works on an element count; it fires after the current pane has collected at least N elements.

There is currently an open ticket for more robust data-driver triggers. However (again, at present), it appears that the Beam team is filling out use-cases for data-driven triggers one at a time (i.e. element count or timestamp), as opposed to adding broad-based support for triggering off arbitrary values within an element.

An ExecutableTrigger wraps a Trigger object for execution. See ExecutableTrigger docs.

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!