I can't be satisfied by the answers calling for a English-only solution based on manual formats. I've been looking for a proper solution for a while now and I finally found it.
You should be using RuleBasedNumberFormat. It works perfectly and it's respectful of the Locale.