How to read data using Kafka Consumer API from beginning?

Question

Please can anyone tell me how to read messages using the Kafka Consumer API from the beginning every time when I run the consumer.

Answer 1

So for me what worked was a combination of what has been suggested above. The key change was to include

props.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");

and have a randomly generated GROUP ID each time. But this alone didn't work for me. For some reason the first time I polled the consumer it never got any records. I had to hack it to get it to work -

consumer.poll(0); // without this the below statement never got any records
final ConsumerRecords consumerRecords = consumer.poll(Duration.ofMillis(100));

I'm new to KAFKA and have no idea why this is happening, but for anyone else still trying to get this to work, hope this helps.