Do Delay Queue messages count as “In Flight” in SQS?

Question

I'm working on a project in which I intend to use an Amazon SQS Delay Queue.

I'm having a bit of trouble understanding exactly what is meant by "inflight" messages.

There is a note in the documentation that says:

Note

There is a 120,000 limit for the number of inflight messages per queue. Messages are inflight after they have been received by the queue, but have not yet been deleted from the queue. If you reach the 120,000 limit, you will receive an OverLimit error message from Amazon SQS. To help avoid reaching the limit, you should delete the messages from the queue after they have been processed. You can also increase the number of queues you use to process the messages.

But I'm not entirely sure what is considered being "received by the queue." In the flow diagram, where is a message "received" by the queue?

I will have a lot of messages purposely delayed (in the first blue bar of the diagram) but not many in the "visibility timeout" phase (second blue bar).

Do both blue bars count as "inflight" messages? Or is a message only "inflight" after it has been "received" by a ReceiveMessage request (to the right of "Message returned" at the bottom of the chart)?

Answer 1

Messages that are dropped into a Delay Queue are not immediately counted as "In Flight", since they are invisible to any consumers. They aren't counted as "Available" or "In Flight", they just aren't visible to you.

You can verify this with a simple experiment on the AWS SQS console:

• Create a delay queue, and have two tabs open to the SQS console.
• Drop several messages into the delay queue by right clicking the queue and selecting "Send a Message" -- you should notice that these messages aren't counted as "Available" or "In Flight" until the delay expires, at which point they become "Available."

As the documentation says, messages are "In Flight" after they have been received (while they are being handled by some application) but before they have been deleted. If they are not deleted when the application has finished with them, they return to an "Available" state.

You can verify this with a second experiment on the AWS SQS console:

• Open two tabs to the AWS SQS console. Create some messages in your delay queue and wait for them to become available.
• In your first tab, right click the queue and select "View/Delete Messages". In this experiment, the "View/Delete Messages" dialog is analogous to your application -- it is handling the queue message while it is viewing it. Set the "Poll queue for messages" text field to an adequately long duration and start polling for messages. If your messages are available, they should appear immediately.
• In your second tab, refresh the SQS console. You should notice that these messages are now "In Flight", as the messages have been received while you are viewing them in the other tab.

Since your "In Flight" messages are messages that are presently being handled by your application, the queue message depth in that column shouldn't realistically grow very large. Delete your messages after handling them (or retry->appropriately handle then delete/move to another queue to process on an exception) and you should be okay.