Removing duplicate records using MapReduce

Question

I\'m using MongoDB and need to remove duplicate records. I have a listing collection that looks like so: (simplified)

[
  { \"MlsId\": \"12345\"\" },
  { \"M         


        

Answer 1

You can use aggregation operation to remove duplicates. Unwind, introduce a dummy $group and $sum stage and ignore the counts in your next stage. Something like this,

db.myCollection.aggregate([
 {
     $unwind: '$list'
 },
 {
    $group:{
   '_id':
       {
         'listing_id':'$_id', 'MlsId':'$list.MlsId'
       },
          'count':
       {
          '$sum':1
       }
      }
},
{
      $group:
       {
        '_id':'$_id.listing_id',
        'list':
         {
          '$addToSet':
           {
            'MlsId':'$_id.MlsId'
           }
         }
       }
}
]);