How to handle errors with boto3?

Question

I am trying to figure how to do proper error handling with boto3.

I am trying to create an IAM user:

def create_user(username, iam_conn):
    try:
           


        

Answer 1

Use the response contained within the exception. Here is an example:

import boto3
from botocore.exceptions import ClientError

try:
    iam = boto3.client('iam')
    user = iam.create_user(UserName='fred')
    print("Created user: %s" % user)
except ClientError as e:
    if e.response['Error']['Code'] == 'EntityAlreadyExists':
        print("User already exists")
    else:
        print("Unexpected error: %s" % e)

The response dict in the exception will contain the following:

• ['Error']['Code'] e.g. 'EntityAlreadyExists' or 'ValidationError'
• ['ResponseMetadata']['HTTPStatusCode'] e.g. 400
• ['ResponseMetadata']['RequestId'] e.g. 'd2b06652-88d7-11e5-99d0-812348583a35'
• ['Error']['Message'] e.g. "An error occurred (EntityAlreadyExists) ..."
• ['Error']['Type'] e.g. 'Sender'

For more information see:

• boto3 error handling
• botocore error handling

[Updated: 2018-03-07]

The AWS Python SDK has begun to expose service exceptions on clients (though not on resources) that you can explicitly catch, so it is now possible to write that code like this:

import botocore
import boto3

try:
    iam = boto3.client('iam')
    user = iam.create_user(UserName='fred')
    print("Created user: %s" % user)
except iam.exceptions.EntityAlreadyExistsException:
    print("User already exists")
except botocore.exceptions.ParamValidationError as e:
    print("Parameter validation error: %s" % e)
except botocore.exceptions.ClientError as e:
    print("Unexpected error: %s" % e)

Unfortunately, there is currently no documentation for these exceptions but you can get a list of them as follows:

import botocore
import boto3
dir(botocore.exceptions)

Note that you must import both botocore and boto3. If you only import botocore then you will find that botocore has no attribute named exceptions. This is because the exceptions are dynamically populated into botocore by boto3.

Answer 2

Or a comparison on the class name e.g.

except ClientError as e:
    if 'EntityAlreadyExistsException' == e.__class__.__name__:
        # handle specific error

Because they are dynamically created you can never import the class and catch it using real Python.

Answer 3

You need to do something when it fails to handle the issue. Right now you are returning the actual exception. For example, if its not a problem that the user exists already and you want to use it as a get_or_create function maybe you handle the issue by returning the existing user object.

try:
    user = iam_conn.create_user(UserName=username)
    return user
except botocore.exceptions.ClientError as e:

    #this exception could actually be other things other than exists, so you want to evaluate it further in your real code.
    if e.message.startswith(
        'enough of the exception message to identify it as the one you want')

        print('that user already exists.')
        user = iam_conn.get_user(UserName=username)
        return user

    elif e.message.some_other_condition:

         #something else
    else:
         #unhandled ClientError
         raise(e)
except SomeOtherExceptionTypeYouCareAbout as e:
    #handle it

# any unhandled exception will raise here at this point.
# if you want a general handler

except Exception as e:
    #handle it.

That said, maybe it is a problem for your app, in which case you want to want to put the exception handler around the code that called your create user function and let the calling function determine how to deal with it, for example, by asking the user to input another username, or whatever makes sense for your application.

Answer 4

Just an update to the 'no exceptions on resources' problem as pointed to by @jarmod (do please feel free to update your answer if below seems applicable)

I have tested the below code and it runs fine. It uses 'resources' for doing things, but catches the client.exceptions - although it 'looks' somewhat wrong... it tests good, the exception classes are showing and matching when looked into using debugger at exception time...

It may not be applicable to all resources and clients, but works for data folders (aka s3 buckets).

lab_session = boto3.Session() 
c = lab_session.client('s3') #this client is only for exception catching

try:
    b = s3.Bucket(bucket)
    b.delete()
except c.exceptions.NoSuchBucket as e:
    #ignoring no such bucket exceptions
    logger.debug("Failed deleting bucket. Continuing. {}".format(e))
except Exception as e:
    #logging all the others as warning
    logger.warning("Failed deleting bucket. Continuing. {}".format(e))

Hope this helps...

Answer 5

Following @armod's update about exceptions being added right on client objects. I'll show how you can see all exceptions defined for your client class.

Exceptions are generated dynamically when you create your client with session.create_client() or boto3.client(). Internally it calls method botocore.errorfactory.ClientExceptionsFactory._create_client_exceptions() and fills client.exceptions field with constructed exception classes.

All class names are available in client.exceptions._code_to_exception dictionary, so you can list all types with following snippet:

client = boto3.client('s3')

for ex_code in client.exceptions._code_to_exception:
    print(ex_code)

Hope it helps.

Answer 6

If you are calling the sign_up API (AWS Cognito) using Python3, you can use the following code.

def registerUser(userObj):
    ''' Registers the user to AWS Cognito.
    '''

    # Mobile number is not a mandatory field. 
    if(len(userObj['user_mob_no']) == 0):
        mobilenumber = ''
    else:
        mobilenumber = userObj['user_country_code']+userObj['user_mob_no']

    secretKey = bytes(settings.SOCIAL_AUTH_COGNITO_SECRET, 'latin-1')
    clientId = settings.SOCIAL_AUTH_COGNITO_KEY 

    digest = hmac.new(secretKey,
                msg=(userObj['user_name'] + clientId).encode('utf-8'),
                digestmod=hashlib.sha256
                ).digest()
    signature = base64.b64encode(digest).decode()

    client = boto3.client('cognito-idp', region_name='eu-west-1' ) 

    try:
        response = client.sign_up(
                    ClientId=clientId,
                    Username=userObj['user_name'],
                    Password=userObj['password1'],
                    SecretHash=signature,
                    UserAttributes=[
                        {
                            'Name': 'given_name',
                            'Value': userObj['given_name']
                        },
                        {
                            'Name': 'family_name',
                            'Value': userObj['family_name']
                        },
                        {
                            'Name': 'email',
                            'Value': userObj['user_email']
                        },
                        {
                            'Name': 'phone_number',
                            'Value': mobilenumber
                        }
                    ],
                    ValidationData=[
                        {
                            'Name': 'email',
                            'Value': userObj['user_email']
                        },
                    ]
                    ,
                    AnalyticsMetadata={
                        'AnalyticsEndpointId': 'string'
                    },
                    UserContextData={
                        'EncodedData': 'string'
                    }
                )
    except ClientError as error:
        return {"errorcode": error.response['Error']['Code'],
            "errormessage" : error.response['Error']['Message'] }
    except Exception as e:
        return {"errorcode": "Something went wrong. Try later or contact the admin" }
    return {"success": "User registered successfully. "}

error.response['Error']['Code'] will be InvalidPasswordException, UsernameExistsException etc. So in the main function or where you are calling the function, you can write the logic to provide a meaningful message to the user.

An example for the response (error.response):

{
  "Error": {
    "Message": "Password did not conform with policy: Password must have symbol characters",
    "Code": "InvalidPasswordException"
  },
  "ResponseMetadata": {
    "RequestId": "c8a591d5-8c51-4af9-8fad-b38b270c3ca2",
    "HTTPStatusCode": 400,
    "HTTPHeaders": {
      "date": "Wed, 17 Jul 2019 09:38:32 GMT",
      "content-type": "application/x-amz-json-1.1",
      "content-length": "124",
      "connection": "keep-alive",
      "x-amzn-requestid": "c8a591d5-8c51-4af9-8fad-b38b270c3ca2",
      "x-amzn-errortype": "InvalidPasswordException:",
      "x-amzn-errormessage": "Password did not conform with policy: Password must have symbol characters"
    },
    "RetryAttempts": 0
  }
}

For further reference : https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/cognito-idp.html#CognitoIdentityProvider.Client.sign_up