Kafka Stream Suppress session-windowed-aggregation

隐身守侯 提交于 2019-11-26 21:53:07

问题


I have written this code in a Kafka stream application:

KGroupedStream<String, foo> groupedStream = stream.groupByKey();
groupedStream.windowedBy(
SessionWindows.with(Duration.ofSeconds(3)).grace(Duration.ofSeconds(3)))
    .aggregate(() -> {...})
    .suppress(Suppressed.untilWindowCloses(unbounded()))
    .toStream()...

which should (if i understood it correctly) emit records per Key after the window is closed. Somehow the behavior is the following:

The stream doesn't emit the first record and only forward it after the second record even with a different Key and then the second record is emitted only after the 3rd and so forth..

I have tried multiple StreamConfigs with "exactly_once" and with or without Caching also, this behavior persists.

Thanks in advance for your help !


回答1:


That is expected behavior. Note, that suppress() is based on event-time. Thus, as long as no new data arrives, time cannot advance and thus evicting the record earlier would be wrong, because there is no guarantee, that the next record might belong to the current window.



来源:https://stackoverflow.com/questions/54222594/kafka-stream-suppress-session-windowed-aggregation

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!