Downloading a large file in iOS app

Question

I'm trying to clean up some existing code that downloads a large file from a server in chunks, checks the checksum on each 50 packets, then stitches them together. I'm having some trouble to see if it's the most efficient way as right now it crashes some time because of memory issues. If I don't have the checksum, it does not seem to crash, but I would prefer if I could check my files first.

@property (nonatomic, retain) NSMutableData * ReceivedData;

- (void)connectionDidFinishLoading:(NSURLConnection *)connection {

NSData *sequencePacketData = [[NSData alloc] initWithData:self.ReceivedData]; 
        [self ProcessPacket:sequencePacketData];
        [sequencePacketData release];
        [[NSNotificationCenter defaultCenter] postNotificationName:DownloadNotification object:self];

}

- (void)ProcessPacket:(NSData *)sequencePacketData {
  // find the directory I need to write to and the name of the file
NSString *currentChecksum = [WebServiceManager MD5CheckSumForNSData:sequencePacketData];
    BOOL checkSumValid = [dmgr ValidateChecksum:currentChecksum againstFileName:self.CurrentFileName];
    self.IsSuccessful = checkSumValid;

    if (!checkSumValid) {
        // log error msg
        return;
    }

    if (success)
    {
        NSFileHandle *handle = [NSFileHandle fileHandleForUpdatingAtPath:sequencePath];
        [handle seekToEndOfFile];
        [handle writeData:sequencePacketData];
        [handle closeFile];
    }
    else
    {
        [sequencePacketData writeToFile:sequencePath atomically:YES];
    }

    // When file is completely downloaded, check the checksum of the entire file:
BOOL completeFileCheckSum;
    if ([packetFile isEqualToString:@"50.bin"]) {
        NSData *temData = [NSData dataWithContentsOfFile:sequencePath];
        currentChecksum = [WebServiceManager MD5CheckSumForNSData:temData];
        completeFileCheckSum = [dmgr ValidateChecksum:currentChecksum againstFileName:fileName];
        NSLog(@"Checksum for whole file is valid: %i", completeFileCheckSum);
        if (!completeFileCheckSum) {
            NSError *err;
            [fileManager removeItemAtPath:sequencePath error:&err];

            // log error
            return;
        }
    }
}

+ (NSString*)MD5CheckSumForNSData:(NSData *) input
{
    // Create byte array of unsigned chars
    unsigned char md5Buffer[CC_MD5_DIGEST_LENGTH];

    // Create 16 byte MD5 hash value, store in buffer
    CC_MD5(input.bytes, input.length, md5Buffer);

    // Convert unsigned char buffer to NSString of hex values
    NSMutableString *output = [NSMutableString stringWithCapacity:CC_MD5_DIGEST_LENGTH * 2];
    for(int i = 0; i < CC_MD5_DIGEST_LENGTH; i++) 
        [output appendFormat:@"%02x",md5Buffer[i]];
    return output;
}

The check checksum againstFile method just grabs the checksum from a temp file and compares it.

I read about NSAutoReleasePools and how that can help if you are loading a bunch of images and need to clear memory and such, but I wasn't sure if that really applies here and if that, or anything else can help in downloading a large file (a little less than 1 GB). Thanks!

Answer 1

Keeping that much data in memory at once is definitely going to be a problem. Fortunately, you don't need to—you can write the data to disk as it comes off the wire, and you can keep a running checksum.

Trade your ReceivedData for a couple of new ivars:

NSFileHandle* filehandle;
MD5_CTX md5sum;

MD5_CTX is in OpenSSL, which …still isn't on iOS? Gah. Okay, you can find the MD5 source online, e.g. here: http://people.csail.mit.edu/rivest/Md5.c (I'd originally suggested adding OpenSSL to your project, but that's a lot of extra junk that you don't need. But if you happen to already be using OpenSSL, it includes the MD5 functions.)

- (void)connection:(NSURLConnection*)connection didReceiveResponse:(NSURLResponse*)response
{
    MD5Init(&md5sum);

    filehandle = [[NSFileHandle filehandleForWritingAtPath:path] retain];
}

- (void)connection:(NSURLConnection*)connection didReceiveData:(NSData*)data
{
    MD5Update(&md5sum, [data bytes], [data length]);

    [filehandle writeData:data];
}

- (void)connectionDidFinishLoading:(NSURLConnection*)connection
{
    MD5Final(&md5sum);
    // MD5 sum is in md5sum.digest[]

    [filehandle closeFile];

    // verify MD5 sum, etc..
}

At the end, your file will be on disk, you'll have its MD5 sum, and you'll barely be using any memory at all.