Effective strategy to avoid duplicate messages in apache kafka consumer

Question

I have been studying apache kafka for a month now. I am however, stuck at a point now. My use case is, I have two or more consumer processes running on different machines. I

Answer 1

Whatever done on producer side, still the best way we believe to deliver exactly once from kafka is to handle it on consumer side:

1. Produce msg with a uuid as the Kafka message Key into topic T1
2. consumer side read the msg from T1, write it on hbase with uuid as rowkey
3. read back from hbase with the same rowkey and write to another topic T2
4. have your end consumers actually consume from topic T2