Merge Mongodb collection and a Python Dict

问题

I have a collection that look something like that :

{
  {"name": "aaa", "value": 100},
  {"name": "bbb", "value": 50},
  {"name": "ccc", "value": 200},
}

and imagine I have a dict like this one :

{
  {"name": "aaa", "value": 40},
  {"name": "ccc", "value": -100},
  {"name": "ddd", "value": 200},
}

I would like to merge the dict into the collections in a way it add the documents in which the name does not appear in the collection, and merge the 2 value between documents that had a name equal. The collection should end up like this :

{
  {"name": "aaa", "value": 140},
  {"name": "bbb", "value": 50},
  {"name": "ccc", "value": 100},
  {"name": "ddd", "value": 200},
}

I've checked the replace_many, insert_many and the update_many but it looks like it's not possible to make them behave like what I want to do.

For the moment I'm doing a basic insert_many of all the document that are not present and a replace_one for each document that is present but I would like to do something more optimized.

thank you in advance

回答1:

You can try below code :

import pymongo
from pymongo import UpdateOne
import json
import sys

inputArray = [
    {"name": "aaa", "value": 40},
    {"name": "ccc", "value": -100},
    {"name": "ddd", "value": 200},
]

bulkArr = []

# Iterate over all dicts in a list & form the query,
# Here `upsert=True` helps to insert a new doc into collection if there is no match on `{"name": x['name']}`,
# If there is a match doc will be updated, You can use `$set` or any-other operator depends on your use case.
for x in inputArray:
    bulkArr.append(
        UpdateOne({"name": x['name']}, {'$inc': {'value': x['value']}}, upsert=True))

# print('[%s]' % ', '.join(map(str, bulkArr)))

try:
    result = db.collection.bulk_write(bulkArr) # Ensure db connection is established by this point
    # Opt for below - if you want to proceed on all dictionaries to be updated, even though an error occured in between for one dict
    # result = db.collection.bulk_write(bulkArr, ordered=False)
    print(result.bulk_api_result)
except:
    e = sys.exc_info()[0]
    print("An exception occurred ::", e) ## Get the ids failed if any & do re-try

Ref : bulk-write-operations

回答2:

You need to use update_one() with upsert=True with the $inc operator:

from pymongo import MongoClient
from bson.json_util import dumps

db = MongoClient()['mydatabase']

item1 = [
  {"name": "aaa", "value": 100},
  {"name": "bbb", "value": 50},
  {"name": "ccc", "value": 200}
]

item2 = [
  {"name": "aaa", "value": 40},
  {"name": "ccc", "value": -100},
  {"name": "ddd", "value": 200}
]

for item in [item1, item2]:
    for record in item:
        db.mycollection.update_one({'name': record.get('name')}, {'$inc': {'value': record.get('value')}}, upsert=True)

print(dumps(db.mycollection.find({}, {'_id': 0}), indent=4))

gives:

[
    {
        "name": "aaa",
        "value": 140
    },
    {
        "name": "bbb",
        "value": 50
    },
    {
        "name": "ccc",
        "value": 100
    },
    {
        "name": "ddd",
        "value": 200
    }
]

来源：https://stackoverflow.com/questions/62221575/merge-mongodb-collection-and-a-python-dict

标签

python

mongodb

pymongo