multiple testPerformance with XCTestCase objective-C

Question

I have 2 performance test methods in test class. If i run them separately they pass. If i run hole class methods they fail with message:

**** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'API violation - multiple calls made to -[XCTestExpectation fulfill].'*

Is there any way to include couple performance tests in 1 class?

here is sample code:

- (void)testPerformanceAuthenticateWithLogin {
    XCTestExpectation *authenticateExpectation = [self expectationWithDescription:@"Authenticate With Login"];
    __block int userID = 0;
    [self measureBlock:^{
        [AuthenticationService authenticateWithLogin:email password:password success:^(AuthenticationResponse *result) {
            XCTAssert(result.isAuthenticated);
            userID = result.userID;
            [authenticateExpectation fulfill];
        } failure:^(NSError *error) {
            XCTAssertNil(error);
        }];
    }];
    [self waitForExpectationsWithTimeout:3 handler:^(NSError *error) {
        XCTAssertNil(error);
        [AuthenticationService logoutWithServicePersonID:userID success:nil failure:nil];
    }];
}

- (void)testPerformanceGetServicePersonByID {
    XCTestExpectation *getServicePersonExpectation = [self expectationWithDescription:@"get Service Person By ID"];
    __block int userID = 0;
    [AuthenticationService authenticateWithLogin:email password:password success:^(AuthenticationResponse *result) {
        userID = result.userID;
        [self loginSuccess:result];
        [self measureBlock:^{
            [ServicePersonService getServicePersonByIDWithServicePersonID:userID success:^(ServicePersonDTO *result) {
                XCTAssertNotNil(result);
                [getServicePersonExpectation fulfill];
            } failure:^(NSError *error) {
                XCTAssertNil(error);
            }];
        }];
    } failure:^(NSError *error) {
        XCTAssertNil(error);
    }];

    [self waitForExpectationsWithTimeout:3 handler:^(NSError *error) {
        XCTAssertNil(error);
        [AuthenticationService logoutWithServicePersonID:userID success:nil failure:nil];
    }];
}

Answer 1

The measureBlock actually runs the code inside the block multiple times to determine an average value.

If I understand correctly you want to measure how long does the authentication take. If that's the case:

• you could put the expectation inside the measure block so every time a measure iteration is done, the test iteration will wait for the expectation to be fulfilled before executing the block again
• you could bump up the expectation timeout a bit just in case.

I've done something like this (which worked for me):

- (void)testPerformanceAuthenticateWithLogin {
    [self measureBlock:^{
        __block int userID = 0;
        XCTestExpectation *authenticateExpectation = [self expectationWithDescription:@"Authenticate With Login"];

        [AuthenticationService authenticateWithLogin:email password:password success:^(AuthenticationResponse *result) {
            XCTAssert(result.isAuthenticated);
            userID = result.userID;
            [authenticateExpectation fulfill];
        } failure:^(NSError *error) {
            XCTAssertNil(error);
        }];

        [self waitForExpectationsWithTimeout:10 handler:^(NSError *error) {
            XCTAssertNil(error);
            [AuthenticationService logoutWithServicePersonID:userID success:nil failure:nil];
        }];
    }];
}