How to detect duplicate data?

Question

I have got a simple contacts database but I\'m having problems with users entering in duplicate data. I have implemented a simple data comparison but unfortunately the duplicate

Answer 1

I imagine that this problem is well understood but what occurs to me on first reading is:

• compare fields individually
• count those that match (for a possibly loose definition of match, and possibly weighing the fields differently)
• present for human intervention any cases which pass some threshold

Use your existing database to get a good first guess for the threshold, and correct as you accumulate experience.

You may prefer a fairly strong bias toward false positives, at least at first.