How to insert from csv to dynamodb

问题

Dynamo db have one table employees primary key as id

data.csv is below which uploaded in the csvdynamo bucket

bucket_name = csvdynamo

id,name,co
20,AB,PC
21,CD,PC
22,EF,MC
23,GH,MC

Need to insert above csv into dynamodb

psuedo code

for emp in employees:
    emp_data= emp.split(',')
    print (emp_data)

    try:
        table.put_item(
            Item = {
                "emp_id": int(emp_data[0]),
                "Name": emp_data[1],
                "Company": emp_data[2]
            }
        )
    except Exception as e:
          pass

回答1:

Here is an example of a lambda function which works, as I verified it using my own function, csv file and dynamodb. I think the code is self-explanatory. It should be a good start towards your end use-case.

          import boto3
          import json
          import os

          bucket_name = os.environ['BUCKET_NAME']
          csv_key = os.environ['CSV_KEY_NAME'] # csvdynamo.csv
          table_name = os.environ['DDB_TABLE_NAME']
          
          # temprorary file to store csv downloaded from s3
          tmp_csv_file = '/tmp/' + csv_key

          s3 = boto3.resource('s3')
          db_table = boto3.resource('dynamodb').Table(table_name)

          def save_to_dynamodb(id, name, co):

            return db_table.put_item(
                Item={
                  'emp_id': int(id),
                  'Name': name,
                  'Company': co
                })

          def lambda_handler(event, context):

              s3.meta.client.download_file(
                            bucket_name, 
                            csv_key, 
                            tmp_csv_file)

              with open(tmp_csv_file, 'r') as f:

                next(f) # skip header

                for line in f:

                  id, name, co = line.rstrip().split(',')

                  result = save_to_dynamodb(id, name, co)

                  print(result)

              return {'statusCode': 200}

回答2:

DynamoDB needs to be told the type of field being inserted.

From put_item(), the format for Item is:

    Item={
        'string': {
            'S': 'string',
            'N': 'string',
            'B': b'bytes',
            'SS': [
                'string',
            ],
            'NS': [
                'string',
            ],
            'BS': [
                b'bytes',
            ],
            'M': {
                'string': {'... recursive ...'}
            },
            'L': [
                {'... recursive ...'},
            ],
            'NULL': True|False,
            'BOOL': True|False
        }
    },

Therefore, you would need something like:

        table.put_item(
            Item = {
                "emp_id":  {'N': emp_data[0]},
                "Name":    {'S': emp_data[1]},
                "Company": {'S': emp_data[2]}
            }
        )

来源：https://stackoverflow.com/questions/62650527/how-to-insert-from-csv-to-dynamodb

标签

python

amazon-s3

aws-lambda