Kafka KStream - using AbstractProcessor with a Window

Question

Im hoping to group together windowed batches of output from a KStream and write them to a secondary store.

I was expecting to see .punctuate() get call

Answer 1

Ok - I think this is a bug in Kafka.

Here's why:

In my original testing I was using a single machine to run both the Producer and Consumer. I would run the Producer for a few minutes to generate some test data and then run my tests. This would give the strange output I posted originally.

Then I decided to push the Producer to the background and leave it running. Now I see 100% perfect 30 second intervals between calls to .punctuate(). No more problems with this.

In other words - if the kafka server isn't processing any inbound data then it doesn't seem to be consistent with running the KStreams processes.