What if you let user place caret e.g. to let him select and copy some text?
I would add a DocumentFilter (or override insertString() method of the Document) and in all cases perform insert in the doc.getLength() position and resetting the caret to the doc.getLength() position after the insert.