Obviously I need the correct import statment to solve this problem. According to the docs for AndroidJUnit4, this should be
import android.support.test.runn
Make sure your app in debug build variant. Go to Build > Select Build Variant... and the following should show up: