iCloud + CoreData - how to avoid pre-filled data duplication?

Question

I have a problem with an iCloud shoebox application and hope some-one can help me (I've spent many hours fighting it in vain).

The App: - A simple library style application - containing set of categories (Cat1 .. CatN) each containing items (Item1...ItemM). I used Apple's iPhoneCoreDataRecipes to set up iCloud CoreData stack.

Everything works almost perfect with iCloud except - there should be a number of pre-filled empty categories the user can start using once he has opened the app for the first time (he can also be offline at that time). And here's the devil.

Here's what I do - Once my persistentStoreCoordinator is setup I send notification

dispatch_async(dispatch_get_main_queue(), ^{
    [[NSNotificationCenter defaultCenter]
        postNotificationName: @"RefetchAllDatabaseData"
                      object: self
                    userInfo: nil];
    });

which is received by my MasterViewController. When the notification is received MasterViewController checks the number of categories in the storage. If the number of available categories equals 0 - the pre-filled categories are inserted.

FYI - I use NSMergeByPropertyObjectTrumpMergePolicy for my ManagedObjectContext

The problem: This works well for the 1st device. But for the 2nd device the default categories from iCloud are often received later than persistentStoreCoordinator has been setup (and default categories inserted by 2nd device). In the end I have 2 sets of categories with the same names on both devices.

Any ideas how this can be solved?

Tried solutions: I tried 2 strategies to solve this. Both start in the same way. After I call

[moc mergeChangesFromContextDidSaveNotification: note];

I call

[self materializeKeysWithUserInfo: note.userInfo forContext: moc];

many thanks to Jose Ines Cantu Arrambide from https://devforums.apple.com/thread/126670?start=400&tstart=0 for his reference code - In essence

materializeKeysWithUserInfo:forContext:

get managedObjectIds from note.userInfo and retrieves corresponding objects from ManagedObjectContext putting them into a dictionary.

Strategy 1:

• All my categories have creation time-stamps.
• On insert from iCloud, get pairs of categories with same name if any
• Select older duplicate categories
• move their items to newer duplicate categories
• delete older duplicate categories

These strategy effectively removes duplicates on both devices even before they appear in the UI BUT

1) the items from 1st device are getting lost on the 2nd device - when they come to the 2nd device their parent category is absent and their category field equal nil so I don't know where to put them.

2) in some short time the items that got lost on the 2nd device are also getting lost on the first due to conflicts.

3) some items originating from the 2nd device are also lost due to conflicts.

I tried to prefer older categories against newer but it didn't give any effect

Strategy 2:

• All my categories have creation time-stamps.
• All categories have obsolete boolean field set to NO on creation
• On insert from iCloud, get pairs of categories with same name if any
• Select older duplicate categories
• move their items to newer duplicate categories
• mark older categories with obsolete = YES

These strategy almost always removes duplicates on both devices even before they appear in the UI BUT

the majority (or all) of the items from both devices are getting lost due to a bunch of conflicts on categories and items.

Some concluding thoughts:

It looks like these strategy doesn't work as we start simultaneously changing content ob both devices whereas iCloud is not suitable for such pattern.

In my tests I had both devices running simultaneously. I cannot neglect a case when a happy user who has just bought his 2nd iDevice installs my app on the 2nd device (with tre 1st device running the app) and get lost all his items in the matter of minutes.

Any ideas how this situation can be solved? Do you think iCloud + CoreData is ready for production?

Strategy 3

I've tried to put a pre-filled database (copying it from bundle) to the appropriate path. It worked out partly - I have no more pre-filled categories duplication BUT the items added to the pre-filled categories do not synchronize across the devices.

iCloud is not aware of the data that exists in the database prior to iCloud setup - my 2nd device receives items, inserted on the 1st device in pre-filled categories, with category = nil.

Items in additionally categories (as well as categories themselves) inserted into the storage after iCloud setup do synchronize properly.

Answer 1

Strategy 1 with some modifications appeared to be a working solutions (with some flaws though).

Legend:

• 1st device - started online without any content in the iCloud
• 2nd device - started later than first and OFFLINE. Then it gets online after some items added

So here's the updated strategy:

• All my categories have creation time-stamps

• The categories cannot be renamed (only added or deleted - this is crucial)

• All my items have a string categoryName field which gets its value upon item creation and updated whenever item is moved to a different category - this redundant information helps to achieve success;

On insertion of new Categories:

• On insert from iCloud, I get pairs of categories with same name if any

• Select newer duplicate categories (they will most probably have less items than old ones so we will have less dance in iCloud)

• Move their items if any to older duplicate categories

• Delete newer duplicate categories

On insertion of new Items - if the item belongs to deleted category:

• CoreData tries to merge it and fails as there's no parent category any more (lots of errors in console). It promisses to insert it later.

• After some short time it does merge and insert the item into storage but with NIL category

• Here we pick our item up, find out it's parent category from categoryName and put it to the correct category

VOILA! - no duplicates & everybody happy

A couple of notes:

1. I get a dance of items belonging to the 2nd device (those that will come with nil category to the 1st device) on both devices. After a couple of minutes everything is stabilized
2. No items is lost though
3. The dance happens only on the first iCloud sync of the 2nd (or any other subsequent device)
4. If the 2nd device is started online for the first time the chance that duplicate categories case appears is about 25% only - tested on 3G connection - so dance should not affect the majority of users