Do loops and convenience methods cause memory peaks with ARC?

Question

I'm working with ARC and seeing some strange behavior when modifying strings in a loop.

In my situation, I'm looping using NSXMLParser delegate callbacks, but I see the same exact behavior and symptoms using a demo project and sample code which simply modifies some NSString objects.

You can download the demo project from GitHub, just uncomment one of the four method calls in the main view controller's viewDidLoad method to test the different behaviors.

For simplicity's sake, here's a simple loop which I've stuck into an empty single-view application. I pasted this code directly into the viewDidLoad method. It runs before the view appears, so the screen is black until the loop finishes.

NSString *text;

for (NSInteger i = 0; i < 600000000; i++) {

    NSString *newText = [text stringByAppendingString:@" Hello"];

    if (text) {
        text = newText;
    }else{
        text = @"";
    }
}

The following code also keeps eating memory until the loop completes:

NSString *text;

for (NSInteger i = 0; i < 600000000; i++) {

    if (text) {
        text = [text stringByAppendingString:@" Hello"];
    }else{
        text = @"";
    }
}

Here's what these two loops loop like in Instruments, with the Allocations tool running:

See? Gradual and steady memory usage, until a whole bunch of memory warnings and then the app dies, naturally.

Next, I've tried something a little different. I used an instance of NSMutableString, like so:

NSMutableString *text;

for (NSInteger i = 0; i < 600000000; i++) {

    if (text) {
        [text appendString:@" Hello"];
    }else{
        text = [@"" mutableCopy];
    }
}

This code seems to perform a lot better, but still crashes. Here's what that looks like:

Next, I tried this on a smaller dataset, to see if either loop can survive the build up long enough to finish. Here's the NSString version:

NSString *text;

for (NSInteger i = 0; i < 1000000; i++) {

    if (text) {
        text = [text stringByAppendingString:@" Hello"];
    }else{
        text = @"";
    }
}

It crashes as well, and the resultant memory graph looks similar to the first one generated using this code:

Using NSMutableString, the same million-iteration loop not only succeeds, but it does in a lot less time. Here's the code:

NSMutableString *text;

for (NSInteger i = 0; i < 1000000; i++) {

    if (text) {
        [text appendString:@" Hello"];
    }else{
        text = [@"" mutableCopy];
    }
}

And have a look at the memory usage graph:

The short spike in the beginning is the memory usage incurred by the loop. Remember when I noted that seemingly irrelevant fact that the screen is black during the processing of the loop, because I run it in viewDidLoad? Immediately after that spike, the view appears. So it appears that not only do NSMutableStrings handle memory more efficiently in this scenario, but they're also much faster. Fascinating.

Now, back to my actual scenario... I'm using NSXMLParser to parse out the results of an API call. I've created Objective-C objects to match my XML response structure. So, consider for example, an XML response looking something like this:

<person>
<firstname>John</firstname>
<lastname>Doe</lastname>
</person>

My object would look like this:

@interface Person : NSObject

@property (nonatomic, strong) NSString *firstName;
@property (nonatomic, strong) NSString *lastName;

@end

Now, in my NSXMLParser delegate, I'd go ahead and loop through my XML, and I'd keep track of the current element (I don't need a full hierarchy representation since my data is rather flat, it's a dump of a MSSQL database as XML) and then in the foundCharacters method, I'd run something like this:

- (void)parser:(NSXMLParser *)parser foundCharacters:(NSString *)string{
  if((currentProperty is EqualToString:@"firstname"]){
    self.workingPerson.firstname = [self.workingPerson.firstname stringByAppendingString:string]; 
  }
}

This code is much like the first code. I'm effectively looping through the XML using NSXMLParser, so if I were to log all of my method calls, I'd see something like this:

parserDidStartDocument: parser:didStartElement:namespaceURI:qualifiedName:attributes: parser:foundCharacters: parser:didStartElement:namespaceURI:qualifiedName: parser:didStartElement:namespaceURI:qualifiedName:attributes: parser:foundCharacters: parser:didStartElement:namespaceURI:qualifiedName: parser:didStartElement:namespaceURI:qualifiedName:attributes: parser:foundCharacters: parser:didStartElement:namespaceURI:qualifiedName: parserDidEndDocument:

See the pattern? It's a loop. Note that it's possible to have multiple consecutive calls to parser:foundCharacters: as well, which is why we append the property to previous values.

To wrap it up, there are two problems here. First of all, memory build up in any sort of loop seems to crash the app. Second, using NSMutableString with properties is not so elegant, and I'm not even sure that it's working as intended.

In general, is there a way to overcome this memory buildup while looping through strings using ARC? Is there something specific to NSXMLParser that I can do?

Edit:

Initial tests indicate that even using a second @autoreleasepool{...} doesn't seem to fix the issue.

The objects have to go somewhere in memory while thwy exist, and they're still there until the end of the runloop, when the autorelease pools can drain.

This doesn't fix anything in the strings situation as far as NSXMLParser goes, it might, because the loop is spread across method calls - need to test further.

(Note that I call this a memory peak, because in theory, ARC will clean up memory at some point, just not until after it peaks out. Nothing is actually leaking, but it's having the same effect.)

Edit 2:

Sticking the autorelease pool inside of the loop has some interesting effects. It seems to almost mitigate the buildup when appending to an NSString object:

NSString *text;

for (NSInteger i = 0; i < 600000000; i++) {

        @autoreleasepool {
            if (text) {
                text = [text stringByAppendingString:@" Hello"];
            }else{
                text = [@"" mutableCopy];
            }
        }
    }

The Allocations trace looks like so:

I do notice a gradual buildup of memory over time, but it's to the tune of about a 150 kilobytes, not the 350 megabytes seen earlier. However, this code, using NSMutableString behaves the same as it did without the autorelease pool:

NSMutableString *text;

for (NSInteger i = 0; i < 600000000; i++) {

        @autoreleasepool {
            if (text) {
                [text appendString:@" Hello"];
            }else{
                text = [@"" mutableCopy];
            }
        }
    }

And the Allocations trace:

It would appear that NSMutableString is apparently immune to the autorelease pool. I'm not sure why, but at first guess, I'd tie this in with what we saw earlier, that NSMutableString can handle about a million iterations on its own, whereas NSString cannot.

So, what's the correct way of resolving this?

Answer 1

You are polluting the autorelease pool with tons and tons of autoreleased objects.

Surround the internal part of the loop with an autorelease pool:

for (...) {
    @autoreleasepool {
        ... your test code here ....
    }
}

Answer 2

While you're hunting memory-related bugs, you should note that @"" and @" Hello" will be immortal objects. You can think about it as a const, but for objects. There will be one, and only one, instance of this object in memory the entire time.

As @bbum pointed out, and you verified, the @autoreleasepool is the correct way to deal with this in a loop.

In your example with the @autoreleasepool and NSMutableString, the pool doesn't really do much. The only mortal object inside the loop is your mutableCopy of @"", but that will only be used once. The other case is just an objc_msgSend to a persisting object (the NSMutableString), which only references an immortal object and a selector.

I can only assume the memory build up is inside Apple's implementation of NSMutableString, although I can wonder why you'd see it inside the @autoreleasepool and not when it's absent.