Apache Kafka order of messages with multiple partitions

后端 未结 6 704
你的背包
你的背包 2020-12-07 17:31

As per Apache Kafka documentation, the order of the messages can be achieved within the partition or one partition in a topic. In this case, what is the parallelism benefit

6条回答
  •  温柔的废话
    2020-12-07 18:15

    In kafka Messages with the same key, from the same Producer, are delivered to the Consumer in order

    another thing on top of that is, Data within a Partition will be stored in the order in which it is written therefore, data read from a Partition will be read in order for that partition

    So if you want to get your messages in order across multi partitions, then you really need to group your messages with a key, so that messages with same key goes to same partition and with in that partition the messages are ordered.

    In a nutshell, you will need to design a two level solution like above logically to get the messages ordered across multi partition.

提交回复
热议问题