Facelets relies on XML namespaces to work with XHTML. How are HTML 4, and as far as I know, HTML 5 do not support namespaces. Also HTML 5 has some new elements that are not
I've read, that this should be possible, but I did not do it myself, yet. Maybe you should just use HTML 5 inside the xHTML wrapper code. I will see, if I can find the source of information I've again.
[EDIT] Seems like, there has been some work at MyFaces to support HTML5 rendering during Google's summer of code. I don't know if it should be used in a productive way, yet.
Please give us a feedback, if you get it to work. [/EDIT]