I have run into that problem as well. Sometimes moving fragments works, sometimes you have to create a new instance. It seems that moving fragments does not work, if the fragment keeps being in the "isRemoving" state.
Being removed also seems to be prevented by having a fragment in the backstack.