SQLAlchemy INSERT IGNORE

Question

How can I insert multiple data records into table ignoring duplicates. I am using SQLAlchemy.

Thank you!

Answer 1

To always replace INSERT by INSERT OR IGNORE, you can use a compiler extension:

from sqlalchemy.ext.compiler import compiles
from sqlalchemy.sql.expression import Insert

@compiles(Insert)
def _prefix_insert_with_ignore(insert, compiler, **kw):
    return compiler.visit_insert(insert.prefix_with('OR IGNORE'), **kw)

Or to do this only temporarily, call the compiles decorator manually and use deregister once you're done:

from sqlalchemy.ext.compiler import compiles, deregister
from sqlalchemy.sql.expression import Insert

def _prefix_insert_with_ignore(insert, compiler, **kw):
    return compiler.visit_insert(insert.prefix_with('OR IGNORE'), **kw)

compiles(Insert)(_prefix_insert_with_replace)
try:
    # do some inserts...
finally:
    deregister(Insert)

This does feel hacky because it's still a global change, but as long as you don't use threads and make sure everything is properly committed before the deregister call, it's probably okay.