multiple consumers per kinesis shard

后端 未结 2 1887
遇见更好的自我
遇见更好的自我 2020-12-28 17:28

I read you can have multiple consumer apps per kinesis stream.

http://docs.aws.amazon.com/kinesis/latest/dev/developing-consumers-with-kcl.html

however, I h

2条回答
  •  星月不相逢
    2020-12-28 18:07

    Late to the party, but the answer is that you can have multiple consumers per kinesis shard. A KCL instance will only start one process per shard, but you can have another KCL instance consuming the same stream (and shard), assuming the second one has permission.

    There are limits, though, as laid out in the docs, including:

    Each shard can support up to 5 transactions per second for reads, up to a maximum total data read rate of 2 MB per second.

    If you want a stream with multiple consumers where each message will be processed once, you're probably better off with something like Amazon Simple Queue Service.

提交回复
热议问题