AWS Lambda retries despite returning value

问题

I have found out that my and my colleagues lambda doesn't return data we anticipate. After what I have already found, we used deprecated invokeAsync which returnes only statusCode. I have upgraded aws-sdk to the newest version and change the invocation code to use invoke with

InvocationType: 'RequestResponse'

as that should give us the returned value.

Lambdas code itself looks good, its an async function without arguments. Im not using the callback here (3rd argument of handler), as we are doing some async stuff and it would require a small refactor to not use async/await, also i have read that just returning value is ok as well. Earlier lambda was returning array, but after some investigation and googling I have changed it to

return {
      statusCode: 200,
      body: JSON.stringify(result)
    }

where result is the mentioned array, as this was recurring pattern in search results.

Overall result is in our service where we do the invocation, we get timeout, lambda logs that it is returning value, but then retries itself again and again.

Dumping code

invocation in our service:

const lambda = new AWS.Lambda({ httpOptions: { timeout: 600000 } })
const results = await new Promise((resolve, reject) => {
    lambda.invoke(
      {
        FunctionName: `lambda-name`,
        InvocationType: 'RequestResponse',
        Payload: '""'
      },
      (err, data) => {
        if (err) {
          reject(err)
        } else {
          resolve(data)
        }
      }
    )
})

lambda handler

import { parallelLimit } from 'async'

const PARALLEL_FETCHERS_NUMBER = 2

export async function runTasksInParallel (
  tasks,
  limit,
) {
  return new Promise(
    (
      resolve,
      reject,
    ) => {
      parallelLimit(tasks, limit, (error, results) => {
        if (error === null) {
          resolve(results)
        } else {
          reject(error)
        }
      })
    },
  )
}


async function connectToDB(logger) {
  const MONGO_URL = process.env.MONGO_URL || 'mongodb://localhost:27017/'
  let db

  logger.debug('Connecting to mongo')
  try {
    db = await MongoClient.connect(
      MONGO_URL,
      {
        sslValidate: false,
      }
    )
  } catch (error) {
    logger.error('Could not connect to Mongo', { error })
    throw error
  }

  logger.debug('Connected')

  return db
}

async function fetchAndStore(
  list,
  sources,
  logger
) {
  const results = []
  const tasks = sources.map((source) => async (callback) => {
    const { fetchAdapter, storageAdapter } = source
    try {
      const { entries, timestamp } = await fetchAdapter.fetchLatest(list)
      await storageAdapter.storeMany(timestamp, entries)
      results.push(['fetch.success', 1, [ `fetcher:${fetchAdapter.name}` ] ])
    } catch (error) {
      const errorMessage = `Failed to fetch and store from adapter ${fetchAdapter.name}`
      const { message, name, stack } = error
      logger.error(errorMessage, { error: { message, name, stack } })
      results.push(['fetch.error', 1, [ `fetcher:${fetchAdapter.name}` ] ])
    }
    callback && callback(null)
  })

  await runTasksInParallel(tasks, PARALLEL_FETCHERS_NUMBER)

  return results
}

export async function handle() {
  const logger = createLambdaLogger() // just a wrapper for console.log to match interface in service
  logger.debug('Starting fetching data')

  const db: Db = await connectToDB(logger)

  const primaryCollection = db.collection(PRIMARY_COLLECTION)
  const itemsColletion = db.collection(ITEMS_COLLECTION)
  const secondaryCollection = db.collection(SECONDARY_PRICING_COLLECTION)

  const primaryStorageAdapter = new StorageAdapter(
    Promise.resolve(primaryCollection),
    logger
  )
  const itemsStorageAdapter = new ItemsStorageAdapter(
    Promise.resolve(itemsColletion),
    logger
  )
  const secondaryStorageAdapter = new StorageAdapter(
    Promise.resolve(secondaryCollection),
    logger
  )

  const primaryFetchAdapter = new PrimaryFetchAdapter(getFetcherCredentials('PRIMARY'), logger)
  const secondaryFetchAdapter = new SecondaryFetchAdapter(getFetcherCredentials('SECONDARY'), logger)

  const sources = [
    { fetchAdapter: primaryFetchAdapter, storageAdapter: primaryStorageAdapter },
    { fetchAdapter: secondaryFetchAdapter, storageAdapter: secondaryStorageAdapter },
  ]

  try {
    const list = await itemsStorageAdapter.getItems()
    logger.debug(`Total items to fetch ${list.length}`)
    const result = await fetchAndStore(list, sources, logger)
    logger.debug('Returning: ', { result })

    return {
      statusCode: 200,
      body: JSON.stringify(result)
    }
  } catch (error) {
    const errorMessage = 'failed to do task'
    const { message, name, stack } = error
    logger.error(errorMessage, { error: { message, name, stack } })

    return {
      statusCode: 500,
      body: JSON.stringify(new Error(errorMessage))
    }
  } finally {
    await db.close()
  }
}

Edit: I have changed the way i'm invoking the lambda to this below. Basically I tried to listen for every possible event that can say me something. Result is that i get a error event, with response message being Converting circular structure to JSON. Thing is I don't really know about which structure this is, as the value I'm returning is a two elements array of 3 elements (strings and number) arrays.

const lambda = new AWS.Lambda({
        httpOptions: { timeout: 660000 },
        maxRetries: 0
      })
      const request = lambda.invoke(
        {
          FunctionName: `${config.get('env')}-credit-rubric-pricing-fetching-lambda`,
          InvocationType: 'RequestResponse',
          Payload: '""'
        },
        (err, data) => {
          if (err) {
            reject(err)
          } else {
            resolve(data)
          }
        }
      )
      const results = await request
        .on('send', () => {
          logger.debug('Request sent to lambda')
        })
        .on('retry', response => {
          logger.debug('Retrying.', { response })
        })
        .on('extractError', response => {
          logger.debug('ExtractError', { response })
        })
        .on('extractData', response => {
          logger.debug('ExtractData', { response })
        })
        .on('error', response => {
          logger.debug('error', { response })
        })
        .on('succes', response => {
          logger.debug('success', { response })
        })
        .on('complete', response => {
          logger.debug('complete', { response })
        })
        .on('httpHeaders', (statusCode, headers, response, statusMessage) => {
          logger.debug('httpHeaders', { statusCode, headers, response, statusMessage })
        })
        .on('httpData', (chunk, response) => {
          logger.debug('httpData', { response, chunk })
        })
        .on('httpError', (error, response) => {
          logger.debug('httpError', { response, error })
        })
        .on('httpDone', response => {
          logger.debug('httpDone', { response })
        })
        .promise()

来源：https://stackoverflow.com/questions/61729392/aws-lambda-retries-despite-returning-value