Amazon Kinesis and guaranteed ordering

前端 未结 3 1083
抹茶落季
抹茶落季 2020-12-28 17:51

Amazon claims their Kinesis streaming product guarantees record ordering.

It provides ordering of records, as well as the ability to read and/or repla

3条回答
  •  挽巷
    挽巷 (楼主)
    2020-12-28 18:43

    If you need guaranteed order of all data in the stream you can only have one shard. That, of course, doesn't scale very well. What you need to determine is whether you really need that level of ordered data. Is all the data in the stream related to all the other data? The key is to put data in shards when the data is related. Use multiple shards to allow your data to be processed in parallel. If all related data is together in one shard you can take advantage of the guaranteed ordering. If you really need all the data to be ordered you're just going to have to deal with the limited scaling that necessarily comes with that.

提交回复
热议问题