How to unit test asynchronous APIs?

Question

I have installed Google Toolbox for Mac into Xcode and followed the instructions to set up unit testing found here.

It all works great, and I can test my synchronous

Answer 1

To elaborate on @St3fan's solution, you can try this after initiating the request:

- (BOOL)waitForCompletion:(NSTimeInterval)timeoutSecs
{
    NSDate *timeoutDate = [NSDate dateWithTimeIntervalSinceNow:timeoutSecs];

    do
    {
        [[NSRunLoop currentRunLoop] runMode:NSDefaultRunLoopMode beforeDate:timeoutDate];
        if ([timeoutDate timeIntervalSinceNow] < 0.0)
        {
            break;
        }
    }
    while (!done);

    return done;
}

Another way:

//block the thread in 0.1 second increment, until one of callbacks is received.
    NSRunLoop *theRL = [NSRunLoop currentRunLoop];

    //setup timeout
    float waitIncrement = 0.1f;
    int timeoutCounter  = (int)(30 / waitIncrement); //30 sec timeout
    BOOL controlConditionReached = NO;


    // Begin a run loop terminated when the downloadComplete it set to true
    while (controlConditionReached == NO)
    {

        [theRL runMode:NSDefaultRunLoopMode beforeDate:[NSDate dateWithTimeIntervalSinceNow:waitIncrement]];
        //control condition is set in one of your async operation delegate methods or blocks
        controlConditionReached = self.downloadComplete || self.downloadFailed ;

        //if there's no response - timeout after some time
        if(--timeoutCounter <= 0)
        {
            break;
        }
    }

Answer 2

Looks like Xcode 6 will solve the issue. https://developer.apple.com/library/prerelease/ios/documentation/DeveloperTools/Conceptual/testing_with_xcode/testing_3_writing_test_classes/testing_3_writing_test_classes.html

Answer 3

My answer is that unit testing, conceptually, is not suitable for testing asynch operations. An asynch operation, such as a request to the server and the handling of the response, happens not in one unit but in two units.

To relate the response to the request you must either somehow block execution between the two units, or maintain global data. If you block execution then your program is not executing normally, and if you maintain global data you have added extraneous functionality that may itself contain errors. Either solution violates the whole idea of unit testing and requires you to insert special testing code into your application; and then after your unit testing, you will still have to turn off your testing code and do old-fashioned "manual" testing. The time and effort spent on unit testing is then at least partly wasted.

Answer 4

This is tricky. I think you will need to setup a runloop in your test and also the ability to specify that runloop to your async code. Otherwise the callbacks won't happen since they are executed on a runloop.

I guess you could just run the runloop for s short duration in a loop. And let the callback set some shared status variable. Or maybe even simply ask the callback to terminate the runloop. That way you you know the test is over. You should be able to check for timeouts by stoppng the loop after a certain time. If that happens then a timeout ocurred.

I've never done this but I will have to soon I think. Please do share your results :-)

Answer 5

St3fan, you are a genius. Thanks a lot!

This is how I did it using your suggestion.

'Downloader' defines a protocol with a method DownloadDidComplete that fires on completion. There's a BOOL member variable 'downloadComplete' that is used to terminate the run loop.

-(void) testDownloader {
 downloadComplete = NO;
 Downloader* downloader = [[Downloader alloc] init] delegate:self];

 // ... irrelevant downloader setup code removed ...

 NSRunLoop *theRL = [NSRunLoop currentRunLoop];

 // Begin a run loop terminated when the downloadComplete it set to true
 while (!downloadComplete && [theRL runMode:NSDefaultRunLoopMode beforeDate:[NSDate distantFuture]]);

}


-(void) DownloaderDidComplete:(Downloader*) downloader withErrors:(int) errors {
    downloadComplete = YES;

    STAssertNotEquals(errors, 0, @"There were errors downloading!");
}

The run-loop could potentially run forever of course.. I'll improve that later!

Answer 6

I find it very convenient to use https://github.com/premosystems/XCAsyncTestCase

It adds three very handy methods to XCTestCase

@interface XCTestCase (AsyncTesting)

- (void)waitForStatus:(XCTAsyncTestCaseStatus)status timeout:(NSTimeInterval)timeout;
- (void)waitForTimeout:(NSTimeInterval)timeout;
- (void)notify:(XCTAsyncTestCaseStatus)status;

@end

that allow very clean tests. An example from the project itself:

- (void)testAsyncWithDelegate
{
    NSURLRequest *request = [NSURLRequest requestWithURL:[NSURL URLWithString:@"http://www.google.com"]];
    [NSURLConnection connectionWithRequest:request delegate:self];
    [self waitForStatus:XCTAsyncTestCaseStatusSucceeded timeout:10.0];
}

- (void)connectionDidFinishLoading:(NSURLConnection *)connection
{
    NSLog(@"Request Finished!");
    [self notify:XCTAsyncTestCaseStatusSucceeded];
}

- (void)connection:(NSURLConnection *)connection didFailWithError:(NSError *)error
{
    NSLog(@"Request failed with error: %@", error);
    [self notify:XCTAsyncTestCaseStatusFailed];
}