Javadoc of the done()
method of SwingWorker:
Executed on the Event Dispatch Thread after the doInBackground method is
From the Java docs: cancel(boolean mayInterruptIfRunning) "mayInterruptIfRunning - true if the thread executing this task should be interrupted; otherwise, in-progress tasks are allowed to complete"
If you call cancel(true) instead of cancel(false) that seems to behave as you are expecting.
I have not seen done() called off the EDT using EventQueue.isDispatchThread()