Kafka topic per producer

前端 未结 2 1723
盖世英雄少女心
盖世英雄少女心 2021-01-01 05:40

Lets say I have multiple devices. Each device has different type of sensors. Now I want to send the data from each device for each sensor to kafka. But I am confused about t

2条回答
  •  一生所求
    2021-01-01 06:01

    I would create topics based on sensors and partitions based on devices:

    A sensor on Device 1 -> topic A, partition 1.
    A sensor on Device 2 -> topic A, partition 2.
    B sensor on Device 2 -> topic B, partition 2.
    

    and so on.

    I don't know what kind of sensors you have, but they seems to belong semantically to the same set of data. With the help of partitions you can have parallel processing.

    But it depends on how you want to process you data: is it more important to process sensors together or devices?

提交回复
热议问题