This is more of a HTTP spec question than a Sitecore question.
I have a 404 error page set up in Sitecore:
As well as what Ruud has suggested, there was a blog post by Kern Herskind that uses a Sitecore/IIS approach without having to write any additional code.