Evaluation of a strategy to sync Core Data with Dropbox

Question

This question is about using Dropbox to sync an sqlite Core Data store between multiple iOS devices. Consider this arrangement:

1. An app utilizes a Core Data store, call it local.sql, saved in the app's own NSDocumentDirectory

2. The app uses the Dropbox Sync API to observe a certain file in the user's Dropbox, say, user/myapp/synced.sql

3. The app observes NSManagedObjectContextDidSaveNotification, and on every save it copies local.sql to user/myapp/synced.sql, thereby replacing the latter.

4. When the Dropbox API notifies us that synced.sql changed, we do the opposite of part 3 more or less: tear down the Core Data stack, replace local.sql with synced.sql, and recreate the Core Data stack. The user sees "Syncing" or "Loading" in the UI in the meantime.

Questions:

A. Is this arrangement hugely inefficient, to the extent where it should be completely avoided? What if we can guarantee the database is not large in size?

B. Is this arrangement conducive to file corruption? More than syncing via deltas/changelogs? If so, will you please explain in detail why?

Answer 1

A. Is this arrangement hugely inefficient, to the extent where it should be completely avoided? What if we can guarantee the database is not large in size?

Irrelevant, because:

B. Is this arrangement conducive to file corruption? More than syncing via deltas/changelogs? If so, will you please explain in detail why?

Yes, very much so. Virtually guaranteed. I suggest reviewing How to Corrupt An SQLite Database File. Offhand you're likely to commit at least two of the problems described in section 1, including copying the file while a transaction is active and deleting (or failing to copy, or making a useless copy of) the journal file(s). In any serious testing, your scheme is likely to fall apart almost immediately.

If that's not bad enough, consider the scenario where two devices save changes simultaneously. What then? If you're lucky you'll just get one of Dropbox's notorious "conflicted copy" duplicates of the file, which "only" means losing some data. If not, you're into full database corruption again.

And of course, tearing down the Core Data stack to sync is an enormous inconvenience to the user.

If you'd like to consider syncing Core Data via Dropbox I suggest one of the following:

1. Ensembles, which can sync over Dropbox (while avoiding the problems described above) or iCloud (while avoiding the problems of iOS's built-in Core Data/iCloud sync).
2. TICoreDataSync, which uses Dropbox file sync but which avoids putting a SQLite file in the file store.
3. ParcelKit, which uses Dropbox's new data store API. (Note that this is quite new and that the data store API itself is still beta).