This is straight from the Java Docs:
This class and its iterator implement all of the optional methods of the Collection and Iterator interfaces.
Well, as the Javadoc says, that's how it's been implemented. The priority queue probably uses a binary heap as the underlying data structure. When you remove items, the heap is reordered to preserve the heap property.
Secondly, it's unwise to tie in a specific implementation (forcing a sorted order). With the current implementation, you are free to traverse it in any order and use any implementation.