Kafka Streams (Suppress): Closing a TimeWindow by timeout

≡放荡痞女 提交于 2020-01-06 04:40:07

问题


I have the following piece of code to aggregate data hourly based on event time

KStream<Windowed<String>, SomeUserDefinedClass> windowedResults = inputStream
.groupByKey(Grouped.with(Serdes.String(), new SomeUserDefinedSerde<>()))
.windowedBy(TimeWindows.of(Duration.ofMinutes(60)).grace(Duration.ofMinutes(15)))
.aggregate
(
    // do some aggregation
)
.suppress(Suppressed.untilTimeLimit(Duration.ofMinutes(75), Suppressed.BufferConfig.unbounded()))
.toStream();

The issue is that I am unable to close the time window and emit the results if I don't receive data with the same key and a timestamp later than the time limit + grace period.

I would like to know what are the alternatives I can use to ensure the window is closed and data is emitted once a given time has passed (without waiting for any new data for the same key).

Is there an option/feature to make the untilTimeLimit parameter based on real time, and not the event time?

Note: This question is not about why a TimeWindow is not closed, but how to close it in the absence of new data

来源:https://stackoverflow.com/questions/54890239/kafka-streams-suppress-closing-a-timewindow-by-timeout

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!