Removing successful cases from SQS batch. Is it wrong?

Question

I have been working on a lambda that is triggered by a SQS with a batch (let's say of 5) and has automatic retry enabled.

The most common problem with this kind of setup that I have seen is that, if we have 5 events, and event number 1 and 2 are completed successful but event 3 fails then during the retry the 5 events are triggered again. When the lambda has side effect (sending email/writing to DB/etc) you don't want to trigger the successful events again as you don't want to have duplicated values.

The most common answer I have seen is to have a DB where you check before executing the event. Something like:

export const mySqsHandler: SQSHandler = async (event) => {
  for (const record of event.Records) {
    if (!db.has(record)) {
      await doStuffThatCanFail(record);
    }
  }
};

But this sounds over complicated.

Now, after looking for a while, I have seen this answer and I thought of coding the following example:

export const mySqsHandler: SQSHandler = async (event) => {
  const entries : DeleteMessageBatchRequestEntry[] = [];
  try {
    for (const record of event.Records) {
      await doStuffThatCanFail(record);
      // Add the successful message to a list
      entries.push({ Id:record.messageId, ReceiptHandle:record.receiptHandle });
    }
  } catch (error) {
    console.error('Failed', error);
    // delete all the message that succeed
    sqs.deleteMessageBatch({ QueueUrl: "url'', Entries:entries })
    // throw error to force the lambda to fail
    throw error;
  }
};

This example seems extremely easy and effective, so my question is, why haven't I seen it before? I have looked into SQS a lot and only one person suggested this.

Is there any big problem I'm missing? Will this even work?

Answer 1

The reason all the messages are put back in the queue when the lambda fails is the assumption is that there is a problem with the lambda itself. Say the network failed, or the machine crashed, or the lambda has a bug etc.

In these cases removing the messages from the queue would result in lost data. So unless a success is returned, via calling the delete method they are put back on the queue.

In this context:

• Success means "I have done my job on these messages"
• Failure means "I have not been able to process these messages, resend"

But that is not the type of error you seem to be talking about.

If you have some process that can fail your lambda needs to handle that failure while still returning a success.

You have two options.

1. The failure is something that retrying can fix.

   You should fix whatever this is

   The lambda should internally retry, requeue should NOT be used as a retry method

2. The failure is something that will always fail for this message

   There should be a defined process for dealing with this problem

   The Lambda should post the message to a new queue for that failure process

Either way the lambda completes successfully. No messages are requeued and all the messages are confirmed processed via the delete method.

Additionally, when dealing with queue systems, all your handlers should be written with the assumption that they may experience duplicate messages at some point. ie they should be idempotent.

This allows for error cases where a message is sent but the success message is not received, leading to the message being sent more than once.