Can we write an AWS Lambda function to query Kinesis Streams

你说的曾经没有我的故事 提交于 2019-12-25 17:16:37

问题


I have Amazon Kinesis Streams, which has all the click stream data and we want to write an API to query Kinesis Streams.

My plan is to create a API gateway which calls an AWS Lambda function that would query Kinesis Streams and return it back.

Is it possible to use Lambda to query Kinesis Streams or should we use Kinesis Analytics and Lambda?


回答1:


You can use AWS Kinesis with Lambda via the Stream-based model. AWS Lambda will itself poll the kinesis stream and fetch the number of records(configured by you) from the Kinesis.

Here's more information: http://docs.aws.amazon.com/lambda/latest/dg/with-kinesis.html




回答2:


Your approach is not the typical way to use Amazon Kinesis Streams.

The normal situation is that data is sent to a Kinesis stream, which holds the data for 24 hours (but can be configured for up to 7 days). Your app can then extract the data from the stream and processes it -- either updating something in real-time or storing information in a database. In fact, Amazon Kinesis Firehose can automatically store incoming data into Amazon S3, Amazon Redshift or Amazon Elasticsearch Service.

Thus, streams are processed rather than queried. Just think of them as replayable queues (rather than databases).

Alternatively, you can have Amazon Kinesis Streams trigger an AWS Lambda function whenever a new message arrives in the stream. This allows near-realtime processing of incoming information.




回答3:


Use something like Firehose and then send it to Elasticsearch which you can then use to query the data. A Kinesis stream is like a temporary buffer to retain data in until you can do something else with it such as load it into a data store.

See my comment below you can use Kinesis Analytics for what you would like to do.



来源:https://stackoverflow.com/questions/43403211/can-we-write-an-aws-lambda-function-to-query-kinesis-streams

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!