Access database UPSERT with pyodbc

问题

I am using pyodbc to update an Access database.

I need the functionality of an UPSERT.

ON DUPLICATE KEY UPDATE doesn't exist in Access SQL, and REPLACE is not an option since I want to keep other fields.

There are a lot of suggestions out there how to solve that, so this is the solution which I put together:

for table_name in data_source:
    table = data_source[table_name]

    for y in table:
        if table_name == "whatever":
            SQL_UPDATE = "UPDATE {} set [Part Name] = '{}', [value] = {}, [code] = {}, [tolerance] = {} WHERE [Unique Part Number]='{}'".\
                format(table_name,y['PartName'],y['Value'],y['keycode'],y['Tolerance'], y['PartNum'])
            SQL_INSERT = "INSERT INTO {} ([Part Name],[Unique Part Number], [value], [code], [tolerance]) VALUES ('{}','{}','{}',{},{},{});".\
                format(table_name,y['PartName'],y['PartNum'],y['Value'],y['keycode'],y['Tolerance'])
        elsif ....
                9 more tables....

       res = cursor.execute(SQL_UPDATE)
       if res.rowcount == 0:
         cursor.execute(SQL_INSERT)

Well I have to say, I am not a Python expert, and I didn't manage to understand the fundamental concept nor the Magic of SQL, so I can just Google things together here.

I don't like my above solution because it is very hard to read and difficult to maintain (I have to to this for ~10 different tables). The other point is that I have to use 2 queries because I didn't manage to understand and run any other UPSERT approach I found.

Does anyone have a recommendation for me how to do this in a smarter, better maintainable way?

回答1:

As noted in this question and others, Access SQL does not have an "upsert" statement, so you will need to use a combination of UPDATE and INSERT. However, you can improve your current implementation by

using proper parameters for your query, and
using Python string manipulation to build the SQL command text.

For example, to upsert into a table named [Donor]

Donor ID  Last Name  First Name
--------  ---------  ----------
       1  Thompson   Gord

You can start with a list of the field names. The trick here is to put the key field(s) at the end, so the INSERT and UPDATE statements will refer to the fields in the same order (i.e., the UPDATE will refer to the ID field last because it will be in the WHERE clause).

data_fields = ['Last Name', 'First Name']
key_fields = ['Donor ID']

The parameter values will be the same for both the UPDATE and INSERT cases, e.g.

params = ('Elk', 'Anne', 2)

The UPDATE statement can be constructed like this

update_set = ','.join(['[' + x + ']=?' for x in data_fields])
update_where = ' AND '.join(['[' + x + ']=?' for x in key_fields])
sql_update = "UPDATE [Donor] SET " + update_set + " WHERE " + update_where
print(sql_update)

which shows us

UPDATE [Donor] SET [Last Name]=?,[First Name]=? WHERE [Donor ID]=?

Similarly, the INSERT statement can be constructed like this

insert_fields = ','.join(['[' + x + ']' for x in (data_fields + key_fields)])
insert_placeholders = ','.join(['?' for x in (data_fields + key_fields)])
sql_insert = "INSERT INTO [Donor] (" + insert_fields + ") VALUES (" + insert_placeholders + ")"
print(sql_insert)

which prints

INSERT INTO [Donor] ([Last Name],[First Name],[Donor ID]) VALUES (?,?,?)

So, to perform our upsert, all we need to do is

crsr.execute(sql_update, params)
if crsr.rowcount > 0:
    print('Existing row updated.')
else:
    crsr.execute(sql_insert, params)
    print('New row inserted.')
crsr.commit()

回答2:

Consider using parameterized queries from prepared statements that uses ? placeholders. The str.format is still needed for identifiers such as table and field names. Then unpack dictionary items with zip(*dict.items()) to pass as parameters in the cursor's execute call: cursor.execute(query, params).

for table_name in data_source:
    table = data_source[table_name]

    for y in table:
        keys, values = zip(*y.items())    # UNPACK DICTIONARY INTO TWO TUPLES

        if table_name == "whatever":
            SQL_UPDATE = "UPDATE {} set [Part Name] = ?, [value] = ?, [code] = ?," + \
                       " [tolerance] = ? WHERE [Unique Part Number]= ?".format(table_name)

            SQL_INSERT = "INSERT INTO {} ([Part Name], [Unique Part Number], [value]," + \
                     " [code], [tolerance]) VALUES (?, ?, ?, ?, ?);".format(table_name)

            res = cursor.execute(SQL_UPDATE, values)
            if res.rowcount == 0:
                cursor.execute(SQL_INSERT, values)
       ...

来源：https://stackoverflow.com/questions/43899189/access-database-upsert-with-pyodbc

标签

python

ms-access

pyodbc

upsert