I am using a JFormattedTextField for to format integer input for a Swing app. The default behavior for invalid input is to quietly (i.e. no error message) reset
I'd have a look at JLayer (Java 7) or JXLayer (Java 6 & below)
This would allow you to decorate the field or form with a verity of different states.
As to when.
I'd start by using some kind of highlighter, Dan suggest using icons, which is a great idea, coupled with JLayer/JXLayer this would reduce the overall effect on the form.
I'd wait until the user tries to submit the form before displaying a dialog & forcing focus back to the invalid fields.
The basic idea would be to allow the user the freedom to move about the form without to much restriction (cause not all people think in a linear manner) but encourage them to correct any possible errors they are producing along the way without having to wait till they submit the form to find the errors (think about the worse web form you've ever used and don't do that)
Obviously, if they try and submit the form with errors, you'll want to display and error message and highlight the first offending field
For examples of JLayer/JXLayer have a look at Validation overlays using JXLayer & How to decorate components