Getting the last message sent to a kafka topic

Question

I\'m new to Kafka and working on a prototype to connect a proprietary streaming service into Kafka.

I\'m looking to get the key of the last message sent on a topic a

Answer 1

The problem is on line final long offset = consumer.committed(partition).offset(), as link api refers committed method is to get the last committed offset for the given partition, i.e: the last offset your consumer tell kafka server that it had already read. So, definitely you will got messages replayed, because you always read from specific offset. As I think I only have to remove the first for block.

Answer 2

Check the record count and get the last message:

    // Poll so we know we're connected
    consumer.poll(100);
    // Get the assigned partitions
    Set<TopicPartition> assignedPartitions = consumer.assignment();
    // Seek to the end of those partitions
    consumer.seekToEnd(assignedPartitions);

    for (TopicPartition partition : assignedPartitions) {
        final long offset = consumer.committed(partition).offset();
        // Seek to the previous message
        consumer.seek(partition, offset - 1);
    }

    // Now get the last message
    ConsumerRecords<String, String> records = consumer.poll(100);
    int size = records.count();
    int index = 0;
    for (ConsumerRecord<String, String> record : records) {
        index = index + 1;
        if (index == size) {
            String value = record.value();
            System.out.println("Last Message = " + value);
        }
    }
    consumer.close();