amazon-sqs

AWS sqs limit to ec2 with iam role

北慕城南 提交于 2019-12-12 04:08:42
问题 I have an SQS to which i want to limit the access of services allowed to send/receive. After reading and trying out I have found that this should be doable by using an Access Policy on the SQS. The policy i wrote: "Version": "2012-10-17", "Id": "arn:aws:sqs:eu-west-1:123456789:HACKsqs03/SQSDefaultPolicy", "Statement": [ { "Sid": "Sid456789", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::123456789:role/HACKiam01" }, "Action": "SQS:ReceiveMessage", "Resource": "arn:aws:sqs:eu-west-1

Celery import and SQS connection issue

淺唱寂寞╮ 提交于 2019-12-12 00:28:55
问题 I'm trying to follow the documentation to get started with celery, but running into hard to debug problems with the sample code. I can't tell if I'm hitting two sides of the same problem, or two unique problems. I can make a connection to the SQS queue through the shell, but not with django. I don't know what the relation is of that behavior to the problems importing Celery vs importing task. The "Getting Started" guide here: http://celery.github.com/celery/getting-started/first-steps-with

Avoiding data loss when slow consumers force backpressure in stream processing (spark, aws)

为君一笑 提交于 2019-12-11 17:13:13
问题 I'm new to distributed stream processing (Spark). I've read some tutorials/examples which cover how backpressure results in the producer(s) slowing down in response to overloaded consumers. The classic example given is ingesting and analyzing tweets. When there is an unexpected spike in traffic such that the consumers are unable to handle the load, they apply backpressure and the producer responds by adjusting its rate lower. What I don't really see covered is what approaches are used in

AWS CloudFormation Template - How to configure Lambda with a SQS queue to pick the item from queue

笑着哭i 提交于 2019-12-11 14:59:58
问题 I am pretty new in AWS CloudFormaton template creation. Could not able to find the way to create a Lambda function which will connect with the SQS queue. Any example or resource would help me to learn and implement. 回答1: You need: AWS::SQS::Queue AWS::Lambda::Function AWS::Lambda::EventSourceMapping "DequeueInstanceQueue": { "Type": "AWS::SQS::Queue", "Properties": { "VisibilityTimeout": 301 }, "DeletionPolicy": "Delete" }, "DequeueInstanceMapping": { "Type": "AWS::Lambda::EventSourceMapping"

Getting “peer not authenticated” exception on Amazon SQS

有些话、适合烂在心里 提交于 2019-12-11 13:33:54
问题 When running the following code from a JAR locally: public static List<Message> getMessages(AmazonSQS sqs, String queueUrl) throws IOException { ReceiveMessageRequest receiveMessageRequest = new ReceiveMessageRequest(queueUrl).withWaitTimeSeconds(5); List<Message> messages = sqs.receiveMessage(receiveMessageRequest).getMessages(); return messages; } It works just fine. But when trying to run it remotely on an EC2 instance, I get the following exception: Apr 15, 2014 8:30:48 PM com.amazonaws

Polling Interval for JMS MessageListener with SQS Provider

夙愿已清 提交于 2019-12-11 10:37:31
问题 I am using amazon sqs queues to consume message asynchronously. I have started consuming the messages using amazon sqs JMS api's. So I have implemented MessageListener and doing my work once message arrive in onMessage method. Since amazon sqs is charged based on request made to the server I would like to restrict polling interval to 1 minute or may be more since I do not need message immediately. I there anyway we can configure polling interval in JMS or I should use amazon sqs API to handle

Spring Cloud AWS SQS SendTo annotation with property placeholder

老子叫甜甜 提交于 2019-12-11 08:37:15
问题 This issue suggests that the @SendTo annotation supports property placeholders, but I can't get it to work. Here's some simplified code snippets of what I'm trying to do (easier than trying to explain with words). I'm on spring-cloud-aws version 1.2.1. This works: @Component public class InputQueueListener { @Value("${replyQueueProperty}") private String replyQueue; @Autowired private QueueMessagingTemplate messagingTemplate; @SqsListener(value = "${inputQueueProperty}", deletionPolicy =

Swagger like documentation for queueing and messaging like mqtt or sqs [closed]

假装没事ソ 提交于 2019-12-11 08:02:01
问题 Closed. This question is off-topic. It is not currently accepting answers. Want to improve this question? Update the question so it's on-topic for Stack Overflow. Closed 3 months ago . Is there any solution like swagger for documenting queueing and messaging like mqtt or sqs? 回答1: Check out AsyncAPI. GitHub repo: https://github.com/asyncapi/asyncapi AsyncAPI is an open source initiative that seeks to improve the current state of Event-Driven Architectures (EDA). Our long-term goal is to make

Provider configuration set number of messages to prefectch

一世执手 提交于 2019-12-11 07:47:46
问题 I am new to spring boot jms and i saw something like provider configuration - > set Number of messages to prefetch function, i could not find much about it and i don't have much clear idea as to what does it mean? I have my concurrency set to 5-100. So does it mean, everytime a new consumer is spawned, this will get the number of messages, lets say if i set it 5, will it get 5 messages from queue at once? Without using spring jms we use to make a receive call which can fetch upto 10 message,

Time period between duplicate messages

佐手、 提交于 2019-12-11 07:12:51
问题 According to the documentation for SQS (emphasis mine): Amazon SQS stores copies of your messages on multiple servers for redundancy and high availability. On rare occasions, one of the servers storing a copy of a message might be unavailable when you receive or delete the message. If that occurs, the copy of the message will not be deleted on that unavailable server, and you might get that message copy again when you receive messages . Because of this, you must design your application to be