ADF copy data activity - check for duplicate records before inserting into SQL db
问题 I have a very simple ADF pipeline to copy data from local mongoDB (self-hosted integration environment) to Azure SQL database. My pipleline is able to copy the data from mongoDB and insert into SQL db. Currently if I run the pipeline it inserts duplicate data if run multiple times. I have made _id column as unique in SQL database and now running pipeline throws and error because of SQL constraint wont letting it insert the record. How do I check for duplicate _id before inserting into SQL db?