Properly cleanup / teardown after instrumentation test only when app is 100% finished

问题

I have a bunch of end-to-end instrumentation tests (relying on Espresso) that start our launcher activity, then navigate throughout our app (eventually creating several activities). At the end of each test our @After annotated tear down method performs some cleanup.

The problem we have is that after the test finishes (successful or failed assertion) the app is still "running", so some of the cleanup is actually causing the app to crash. This is either resulting in false positives if the assert was successful, or hiding the test failure (we only see the crash not the failed assertion).

Here's an example:

import android.app.Instrumentation;
import android.content.Intent;
import android.preference.PreferenceManager;
import android.support.test.InstrumentationRegistry;
import android.support.test.rule.ActivityTestRule;

import com.example.SplashActivity;

import org.junit.After;
import org.junit.Before;
import org.junit.Rule;
import org.junit.Test;

import static android.support.test.InstrumentationRegistry.getInstrumentation;

public class ExampleTest {

    @Rule
    public ActivityTestRule<SplashActivity> splashActivityTestRule
            = new ActivityTestRule<>(SplashActivity.class, true, false);

    Instrumentation.ActivityMonitor splashActivityMonitor;

    @Before
    public void setUp() {
        splashActivityMonitor = new Instrumentation.ActivityMonitor(SplashActivity.class.getName(), null, false);
        getInstrumentation().addMonitor(splashActivityMonitor);
    }

    @Test
    public void someTest() throws Exception {
        // ... other test-specific setup before starting splash activity

        // start first activity
        splashActivityTestRule.launchActivity(new Intent());

        // a bunch of espresso steps that result in several other activities
        // ... creating and adding Instrumentation.ActivityMonitor for each one

        // assert something
    }

    @After
    public void tearDown() {
        // clear shared prefs to prepare for next test
        PreferenceManager.getDefaultSharedPreferences(InstrumentationRegistry.getTargetContext())
                .edit()
                .clear()
                .apply();

        // At this point the app is still running. Maybe a UI is still loading that was not relevant to the test, 
        // or some mock web request is in flight. But at some point after the final assert in our test, the app needs
        // to get something from shared prefs, which we just cleared, so the app crashes.
    }
}

As you can see, the app is still running during the tear down method. Any changes we make to the state of the app here may cause the app to crash.

So how can I assert that the app is dead and gone before doing this cleanup?

Some possible (but ugly) solutions I've come up with:

After the final assert, continue to navigate back to some neutral point in the app (i.e use espresso to logout and return to splash screen). This should work but it'll add a lot of other steps to every test. Also I'm not sure this will work if the assert fails.

Or perform some sort of app kill in the teardown:

public void tearDown() {
    // finish all tasks before cleaning up
    ActivityManager activityManager =
            (ActivityManager) InstrumentationRegistry.getTargetContext().getSystemService(Context.ACTIVITY_SERVICE);

    List<ActivityManager.AppTask> appTasks = activityManager.getAppTasks();
    for (ActivityManager.AppTask appTask : appTasks) {
        appTask.finishAndRemoveTask();
    }

    // clear shared prefs to prepare for next test
    PreferenceManager.getDefaultSharedPreferences(InstrumentationRegistry.getTargetContext())
            .edit()
            .clear()
            .apply();
}

Update:

I know I can use ActivityTestRule.afterActivityFinished() docs but I don't think this will work for multiple actvities.

回答1:

The problem you described can be solved by using AndroidTestOrchestrator. From the official Android documentation:

When using AndroidJUnitRunner version 1.0 or higher, you have access to a tool called Android Test Orchestrator, which allows you to run each of your app's tests within its own invocation of Instrumentation.

Application under test will be cleaned after each test run automatically.

build.gradle file sample with AndroidTestOrchestrator enabled:

Using AndroidTestOrchestrator with Android support library - github link
Using AndroidTestOrchestrator with AndroidX Test library - github link

Official Android documentation - link.

回答2:

a) one could either set default preferences instead of blanking them -

b) or use the shared resource itself, to prevent the race condition:

public static final String PREFERENCE_KEY_TEST_COUNT = "testCount";
public static final int MAX_TEST_COUNT = 6;

@After
public void tearDown() {
    SharedPreferences prefs = PreferenceManager.getDefaultSharedPreferences(InstrumentationRegistry.getTargetContext()):
    if(prefs.getPreference(PREFERENCE_KEY_TEST_COUNT, 0) >= MAX_TEST_COUNT) {
        prefs.edit().clear().apply();
    } else {
        int testCount = prefs.getPreference(PREFERENCE_KEY_TEST_COUNT, 0) + 1;
        prefs.edit().putInt(PREFERENCE_KEY_TEST_COUNT, testCount).apply();
    }
}

c) or Test Suite and a custom Runner could be used to control the tests; this example explains it.

来源：https://stackoverflow.com/questions/54079434/properly-cleanup-teardown-after-instrumentation-test-only-when-app-is-100-fin

标签

android

android-espresso