How to change start offset for topic?

Question

It is possible to change the start offset for a new topic? I would like to create a new topic and start reading from the offset 10000. How?

Answer 1

You can do this with the help of the zookeeper shell. Kafka uses zookeeper to track the consumer offsets.

Go to kafka bin directory and invoke zookeeper shell.(my kafka version is 0.8.0)

./zookeeper-shell.sh localhost:2181

Now use the zookeeper get command

get /consumers/consumer_group_id/offsets/topic/0

it shows something like

2043
cZxid = 0x4d
ctime = Wed Mar 18 03:56:32 EDT 2015
...

Here 2043 is the maximum offset consumed. Set it to desired value by using zookeeper set command

set /consumers/consumer_group_id/offsets/topic/0 10000

The path is framed like this /consumers/[consumer_group_id]/offsets/[topic]/[partition_id].
You will have to substitute with appropriate consumer group, topic and partition id.

*Also since you mentioned it's a new instance of kafka, I am not sure whether consumers would have connected and consumer groups created.