This question is a more special case of the problem described (and solved) in this question.
I have two methods, stopAndRemove(ServerObject server) and a close() met
You might find this article about ConcurrentModificationException has some advice in this area.