I just did my first install of PostgreSQL 9.1 on Ubuntu 10.04.
note: I have done it a few times on Windows with an installer without issues.
After a bit of effor
Creating the database via initdb with the encoding option (-E or --encoding=) did the trick for me on Mac Os X Snow Leopard:
initdb -D -E UTF8
If you init the database this way it will create the database templates with the correct encoding also.
Refer to the initdb documentation here and the encoding documentation here to choose the correct encoding.