amazon-sqs

Amazon SQS more than 2GB data

两盒软妹~` 提交于 2019-12-10 12:25:00
问题 Working on SQS to queue some of my uploads coming from the client. I am getting below error: com.amazonaws.services.sqs.model.AmazonSQSException: One or more parameters are invalid. Reason: Message must be shorter than 262144 bytes. (Service: AmazonSQS; Status Code: 400; Error Code: InvalidParameterValue; Request ID: I am using extended client library. Here are the code that I use to send message: MessageAttributeValue msgAttr = new MessageAttributeValue(); byte [] byteArr=attachment.getBytes

How to pool AWS SQS with AWS Lambda

风格不统一 提交于 2019-12-10 11:34:24
问题 At the moment I'm are pooling AWS SQS from our back-end and doing business logic once payload is received. I would like to move this to AWS Lambda and start automating business logic via SQS/SNS. As I can not subscribe to AWS SQS events, what is the best practice in implementing SQS pooling with Lambda (node.js)? 回答1: SQS doesn't really work well with Lambda since you cannot automatically trigger Lambda functions from SQS queues messages. I would rather remove the SQS/SNS logic and go for a

boto3 sessions and aws_session_token management

微笑、不失礼 提交于 2019-12-10 10:15:44
问题 I am developing python software which deals with AWS SQS queues. It uses boto3 , mostly boto3.session.Session . I have seen here that we can pass an aws_session_token to the Session constructor. When running my code outside of Amazon, I need to periodically refresh this aws_session_token since it is only valid for an hour. So I need to reinstantiate a boto3.Session on my own. I am just wondering how things work inside AWS. Do I need to manually refresh my sessions by getting a new aws_session

Triggering a SWF Workflow based on SQS messages

穿精又带淫゛_ 提交于 2019-12-09 18:29:40
问题 Preamble: I'm trying to put together a proposal for what I assume to be a very common use-case, and I'd like to use Amazon's SWF and SQS to accomplish my goals. There may be other services that will better match what I'm trying to do, so if you have suggestions please feel free to throw them out. Problem: The need at its most basic is for a client (mobile device, web server, etc.) to post a message that will be processed asynchronously without a response to the client - very basic. The

Slow Performance of Amazon SQS compared with RabbitMQ

爷,独闯天下 提交于 2019-12-09 10:16:55
问题 I wanted to integrate a message queuing middle tier in my web application. I have tested Rabbitmq as well as Amazon SQS but find Amazon SQS is slow. I am getting 80 req/sec in Amazon SQS where I am getting 2000 req/sec in Rabbitmq. I am asking this Question because I am more interested Amazon SQS since I am using all the services of Amazon for my web app. Can anybody please tell me why this is so slow? Or if anybody has any good benchmark of Amazon SQS can you please share? Any help will be

Best practices for using Amazon SQS - Polling the queue

一个人想着一个人 提交于 2019-12-09 05:23:17
问题 I'm designing a service for sending out emails for our eCommerce site (order confirmations, alerts etc...) The plan is to have a "SendEmail" method, that generates a chunk of XML representing the email to be sent, and sticks it on an Amazon SQS queue. My web app(s) and other applications will use this to "send" emails. I then require a way of checking the queue, and physically sending out the email messages. (I know how I'm going to be dispatching emails) I'm curious as to what the best way

Amazon SQS Java SDK - cannot receive message attributes

心已入冬 提交于 2019-12-09 05:02:49
问题 After posting a message with an attribute to SQS with the following code before sending it (and checking in SQS console to see if everything is posted correctly)... messageRequest.addMessageAttributesEntry( "attributeTest", new MessageAttributeValue() .withDataType("String") .withStringValue("attributeTest 123")); I cannot retrieve any of the message attributes in the message. All I see, as a result, is "0 attributes.". Reinspecting the message in Amazon SQS console, the message - and the

Amazon SQS SenderId inconsitency

微笑、不失礼 提交于 2019-12-08 09:37:50
问题 Scenario is the following: I have an Amazon SQS Queue that several AWS accounts are writing to. I am giving write permissions to a specific list of Amazon Account IDs. Each time a get a message from this Queue, I am also asking for the "SenderId" attribute. That's how I can identify which customer placed the Job in the queue, so I know who to bill. My problem is: For customer A with AWS account id 12345, the SenderId is 12345. It looks like a account ID. For customer B with AWS account id

My worker tier returns 400 Error, CloudWatch Error?

空扰寡人 提交于 2019-12-07 18:44:56
问题 I have a worker tier Elastic Beanstalk Application connected to a SQS. Over, and over and over again I keep getting HTTP 400 Error returned in my log, but I don't get any error messages at all in the log. [14/Apr/2014:18:03:26 +0000] "POST /customer-registered HTTP/1.1" 400 192 "-" "aws-sqsd" I dont get any errors at error_log, the only error I can find in my log is the following, which is located at /var/log/aws-sqsd/default.log: 2014-04-14T18:02:58Z error: AWS::CloudWatch::Errors:

Is it possible for S3 notifications to SQS to fail?

ⅰ亾dé卋堺 提交于 2019-12-07 12:44:57
问题 I have setup an S3 bucket to publish messages for each PUT and POST actions. Files get uploaded to that bucket using CLI. It does work fine but out of 4 files pushed sequentially, only one triggers a message. I am not sure that this has always happened but it is happening consistently now. Note that it does not happen when I upload file manually (i.e. I always get a message per file). I have made sure that there is no downstream system processing the messages (as a confirmation, I still see