I have an exception I never thought I would see. A class cast exception of the adapter when removing a footer view from a ListView (sic).
java.lang.ClassCa
Adding to the other answers, if you're adding/removing footers dynamically (such as if they reach the bottom of your list and then you're adding a footer view) the easiest thing is to Override setAdapter in your ListView and add a new View object as the footer there, this will ensure the adapter is wrapped in the HeaderViewListAdapter:
@Override
public void setAdapter(ListAdapter adapter) {
addFooterView(new View(getContext()));
super.setAdapter(adapter);
}