Importing with MagicalRecord + AFNetworking

Question

I'm using AFNetworking and MagicalRecord (the current develop branch) and I'm trying to figure out how to import a lot of objects which are dependent on each other. Each resource/entity has multiple pages worth of downloads. I have a class managing the downloads for a given entity and saving them using MagicalDataImport (which has been amazing).

I believe my issue is that the imports aren't happening on the same thread. So I think what is happening is:

• In one thread, EntityA is getting saved properly and propagated to the parent entity.
• Then in another thread, EntityB is being saved, and along with it it's relationship to EntityA is built. That means a blank (fault?) object is being created. Then when it gets propagated to the parent entity, I believe EntityA is overwriting the EntityA that is there. Thus I'm left with some objects that don't have all of the attributes.

At least, I think that is what is happening. What I'm seeing via the UI is actually that the relationships between entities aren't always built correctly.

My end goal is to get the entire download/import process to be done in the background, not effecting the UI at all.

Here is my AFJSONRequest:

AFJSONRequestOperation *operation = [AFJSONRequestOperation
     JSONRequestOperationWithRequest:request
     success:^(NSURLRequest *request, NSHTTPURLResponse *response, id JSON)
     {
         [self saveResources:[JSON objectForKey:@"data"]];
     }
     failure:^(NSURLRequest *request, NSHTTPURLResponse *response, NSError *error, id JSON)
     {
         DLog(@"%@",error.userInfo);
         [self.webService command:self didFail:error.localizedDescription];
     }];

[operation setQueuePriority:self.priority];

And it calls saveResources::

- (void)saveResources:(NSArray*)resources {
    BOOL stopDownloads = [self stopDownloadsBasedOnDate:resources];
    if ([resources count] > 0 && !stopDownloads){
        self.offset = @([offset intValue] + [resources count]);
        [self send];
    }

    [MagicalRecord saveWithBlock:^(NSManagedObjectContext *blockLocalContext) {
        [self.classRef MR_importFromArray:resources inContext:blockLocalContext];
    } completion:^(BOOL success, NSError *error) {
        if (error){
            // ... handle errors
        }
        else {
            // ... handle callbacks
        }
    }];
}

This kicks off another download ([self send]) and then saves the objects.

I know by default AFNetworking calls the callback in the main queue, and I've tried setting the SuccessCallbackQueue/FailureCallbackQueue to my background thread, but that doesn't seem to solve all the issues, I still have some relationships going to faulted objects, though I think I do need to do that to keep everything going in a background thread.

Is there anything else I need to call in order to properly propagate these changes to the main context? Or is there a different way I need to set this up in order to make sure that all the objects are saved correctly and the relationships are properly built?

Update I've rewritten the issue to try to give more clarification to the issues.

Update

If you need more code I created a gist with (I believe) everything.

Answer 1

I ended up having this exact same issue a few days ago. My issue was I had received a customer record from my API with AFNetworking. That customer could have pets, but at this point I didn't have the petTypes to correspond to the customers pet record.

What I did to resolve this was create a transformable attribute with an NSArray which would temporarly store my pets until my petTypes were imported. Upon the importation of petTypes I then triggered an NSNotificationCenter postNotification (or you can just do the pet import in the completion).

I enumerated through the temporary transformable attribute that stored my pet records and then associated the with the petType

Also I see you are doing your import inside of a save handler. This is not needed. Doing your MR_importFromArray will save automatically. If you are not using an MR_import method then you would use the saveToPersistentStore methods.

One thing is I don't see where you are associating the relationships. Is EntityB's relationship to EntityA being sent over via JSON with the EntityA objecting being in EntityB?

If so then this is where the relationship is getting messed up as it is creating / overwriting the existing EntityA for the one provided in EntityB. My recommendation would be to do something like this.

NSArray *petFactors = [responseObject valueForKeyPath:@"details.items"];
NSManagedObjectContext *currentContext = [NSManagedObjectContext MR_context];
Pets *pet = [Pets MR_findFirstByAttribute:@"id" withValue:petId inContext:currentContext];

pet.petFactors = nil;
for (id factor in petFactors) {
    [pet addPetFactorsObject:[PetFactors MR_findFirstByAttribute:@"id" withValue:[factor valueForKey:@"factorId"]]];
}

[currentContext MR_saveToPersistentStoreWithCompletion:^(BOOL success, NSError *error) {
    if (success) {
        NSLog(@"SAVED PET FACTORS");
        [[NSNotificationCenter defaultCenter] postNotificationName:kPetFactorsSavedSuccessfully object:nil];
    } else {
        NSLog(@"Error: %@", error);
    }
}];

Answer 2

I'm putting this as an answer, though I'm not 100% sure if this is your issue or not. I think the issue stems from your localContext. Here is a sample web request method from an app we wrote that uses data importing, you may be able to use it as an example to get yours working.

Note that the AFNetworking performs its completion block on the main thread, then the MagicalRecord saveInBackground method switches back to a background thread to do the importing and processing, then the final MR completion block performs the handler block on the main thread again. The localContext that's used to import is created/managed by the saveInBackground method. Once that method is complete the context is saved and merged with the app's main context and all the data can then be accessed.

- (void)listWithCompletionHandler:(void (^)(BOOL success))handler{
    [[MyAPIClient sharedClient] getPath:@"list.json" parameters:nil success:^(AFHTTPRequestOperation *operation, id responseObject){
        NSString *statusString = [responseObject objectForKey:@"status"];

        // Handle an error response
        if(![statusString isKindOfClass:[NSString class]] || ![statusString isEqualToString:@"success"]){
            // Request failure
            NSLog(@"List Request Error: %@", statusString);
            NSLog(@"%@", [responseObject objectForKey:@"message"]);
            if(handler)
                handler(NO);
            return;
        }

        NSArray *itemsArray = [responseObject objectForKey:@"items"];

        [MagicalRecord saveInBackgroundWithBlock:^(NSManagedObjectContext *localContext){
            // Load into internal database
            NSArray *fetchedItems = [Item importFromArray:itemsArray inContext:localContext];

            NSLog(@"Loaded %d Items", [fetchedItems count]);
        } completion:^{
            if(handler)
                handler(YES);
        }];
    } failure:^(AFHTTPRequestOperation *operation, NSError *error){
        NSLog(@"Fail: %@", error);
        if(handler)
            handler(NO);
    }];
}