(I changed the question a bit, because the problem is a bit clearer now)
I have 4 buttons on my application, and when a user clickes certain button I change that but
I seem to remember having issues when creating too many ColorFilters before. It doesn't sound for certain like that's what's at fault here, since it's happening right away. Still, what you might try is having the filter as a class variable, and then using it within the if/else block. Also, as Trev mentioned, since you're just wanting to remove the green filter, you can just pass null to setColorFilter and avoid making the transparent filter, so you'd end up with something like this:
//in main class
PorterDuffColorFilter greenFilter =
new PorterDuffColorFilter(Color.GREEN, PorterDuff.Mode.SRC_ATOP);
//in CheckAnswer()
Drawable d = findViewById(R.id.ans2).getBackground();
if(answer.equals("1") d.setColorFilter(greenFilter)
else d.setColorFilter(null);