How do I return a message back to SQS from lambda trigger

随声附和 提交于 2020-01-14 05:33:08

问题


I have lambda trigger that reads messages from SQS queue. In some conditions, the message may not be ready for processing so I'd like to put the message back in queue for 1min and try again. Currently, I am create another copy of this customer record and posting this new copy in the queue. Is there a reason/way for me to keep the original record in queue as opposed to creating a new one

def postToQueue(customer):

    if 'attemptCount' in customer.keys():
        attemptCount = int(customer["attemptCount"]) + 1
    else:
        attemptCount = 2
    customer["attemptCount"] = attemptCount

    # Get the service resource
    sqs = boto3.resource('sqs')

    # Get the queue
    queue = sqs.get_queue_by_name(QueueName='testCustomerQueue')
response = queue.send_message(MessageBody=json.dumps(customer), DelaySeconds=60)

    print('customer postback: ', customer)
    print ('response from writing ot the queue is: ', response)

#main function
for record in event['Records']:
    if 'body' in record.keys():
        customer = json.loads(record['body'])
        print("attempting to process customer", customer, " at: ", datetime.datetime.now())
        if (not ifReadyToProcess(customer)):
            postToQueue(customer)
        else:
            processCustomer(customer)

回答1:


This is not an ideal setup for SQS triggering Lambda functions.

My testing shows that messages sent to SQS will immediately trigger the Lambda function, even if a Delay setting is provided. Therefore, putting a message back onto the SQS queue will cause Lambda to fire again straight after.

To avoid a situation where Lambda is continually checking whether a message is ready for processing, I would recommend:

  • Use Amazon CloudWatch Events to trigger a Lambda function on a schedule (eg every 2 minutes)
  • The Lambda function should pull messages from the queue and check if they are ready to process.
    • If they are ready, then process them and delete them
    • If they are not ready, then push them back onto the queue with a Delay setting and delete the original message

Note that this is different to having SQS directly trigger Lambda. Instead, the Lambda function should call ReceiveMessages() to obtain the message(s) itself, which allows the Delay function to add some time between checks.

Another option: Instead of re-inserting a message into the queue, you could simply take advantage of the Default Visibility Timeout setting by not deleting the message. A message that is read from the queue, but not deleted, will automatically "reappear" on the queue. You could use this as the "retry" time period. However, this means you will need to handle Dead Letter processing yourself (eg if a message fails to be processed after n tries).



来源:https://stackoverflow.com/questions/53807007/how-do-i-return-a-message-back-to-sqs-from-lambda-trigger

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!