It should first display Toast then after 2 seconds, it will finish your activity.
Toast.makeText(YourActivity.this, "MESSAGE", Toast.LENGTH_SHORT).show();
new Handler().postDelayed(new Runnable() {
@Override
public void run() {
YourActivity.this.finish();
}
}, 2000);