We're required to use the API of an external partner. The API is in a good shape and we got access to a sandbox environment we can use for automatic testing.
We already test every single call of the external API using unit tests but are unsure regarding best practices for integration tests when it comes to complex operations on the side of the external partner.
Example: Every user of our service also got a user object at our external partner. When performing external API call X on this user object, we expect object Y to appear inside collection Z of this user (which we have to query using a different call).
What are best practices for testing cases like this?
Mock the external API as much as possible and rely on the unit tests to do their job? Advantages: Tests run fast and independent from an internet connection. Disadvantages: Mistakes is in our mocks could lead to false positives.
Integrate the external API sandbox and run every integration test against it. Advantages: Close to real life API interactions. Disadvantages: Tests can only be run with an open internet connection and take more time.
Use a hybrid of mocked and sandbox data, set a boolean to switch between the internal (=mocked) and external (=sandbox) environment when required. Advantages: Reliable tests. Disadvantages: Could be a pain to set up.
Other best practices?
Thanks!
Related: How are integration tests written for interacting with external API? However, the answer "You don't. You have to actually trust that the actual API actually works." is not sufficient in our opinion.
[EDIT] We fear that integration testing only against our assumptions how the external API should work (even if they are based on unit tests) – and not against the actual API – will leave us with false positives. What we'd need is a test that verifies that our assumptions (mocks) are actually correct – not only in the context of unit tests but also in the context of complex operations with several steps.
Validation might be a good example: What if we mess up the integration code and send malformed data or data that does not make any sense in the context we send it in because we missed a step? Our mock API, which does not validate (or only in very limited range) would still return valid data instead of passing the error we would receive from the real API.
I believe there should be 2 level of verifications we need to do when we interface with an external API:
- API verification: verify that the API works according to its specs and/or our understanding
- App functionality verification: verify that our business logic works according to the expectation to the API that passes API verification
In our case, we use a mock API together with real and mock API verification.
- Mock API allows us to isolate any runtime errors/exceptions to app functionality only, so we don't blame any external party for issues
- The same API verification is executed against both real and mock APIs, to make sure that the real one works the way we expect, as well as the mock one should mimic the real one correctly
If along the way, external API changes, API verification may turn red, triggering changes in mock API. Changes in mock API may make app verification turn red, triggering changes in app implementation. This way you never miss any gap between external API and app implementation (ideally).
Another extra benefit of having a mock API + API verification is that your developers can use it as a documentation/specification of how the API is supposed to work.
来源：https://stackoverflow.com/questions/28069535/integration-testing-mock-external-api-vs-use-external-api-sandbox