Kafka consumer offsets out of range with no configured reset policy for partitions

Question

I\'m receiving exception when start Kafka consumer.

org.apache.kafka.clients.consumer.OffsetOutOfRangeException: Offsets out of range with no confi

Answer 1

I hit this SO question when running a Kafka Streams state store with a specific changelog topic config:

• cleanup.policy=compact,delete
• retention of 4 days

If Kafka Streams still has a snapshot file pointing to an offset that doesn't exist anymore, the restore consumer is configured to fail. It doesn't fall back to the earliest offset. This scenario can happen when very few data comes in or when the application is down. In both cases, when there's no commit within the changelog retention period, the snapshot file won't be updated. (This is on partition basis)

Easiest way to resolve this issue is to stop your kafka streams application, remove its local state directory and restart your application.