How does kafka consumer auto commit work?

前端 未结 2 1701
再見小時候
再見小時候 2020-12-10 11:18

I am reading this one:

Automatic Commit The easiest way to commit offsets is to allow the consumer to do it for you. If you configure enable.auto.co

相关标签:
2条回答
  • 2020-12-10 11:59

    The auto-commit check is called in every poll and it checks that the time elapsed is greater than the configured time. If so, the offset is committed.

    In case the commit interval is 5 seconds and poll is happening in 7 seconds, the commit will happen after 7 seconds only.

    0 讨论(0)
  • 2020-12-10 12:19

    It would try to autocommit ASAP after poll completes. You can have a look on the source code of consumer coordinator, which has set of local fields defined on class level to understand whether autocommit is enabled, what is the interval, and what is the next deadline to perform autocommit.

    https://github.com/apache/kafka/blob/10cd98cc894b88c5d1e24fc54c66361ad9914df2/clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java#L625

    And one of the places within poll that perform a call to do storage https://github.com/apache/kafka/blob/10cd98cc894b88c5d1e24fc54c66361ad9914df2/clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java#L279

    That being said for instance poll executed every 7 seconds, and autocommit set to 5:

    0 - poll, + set deadline to 5th second

    7 - poll + commit due to deadline, update deadline to 7+5=12

    14 - poll + commit due to deadline, update deadline to 12+5=17

    However if polling set to every 3 seconds, and autocommit is set to 5:

    0 - poll, + set deadline to 5th second

    3 - poll, no commit

    6 - poll + commit due to deadline, update deadline to 6+5=11

    0 讨论(0)
提交回复
热议问题