The demigod programmer from Microsoft has an answer: http://blogs.msdn.com/oldnewthing/archive/2004/03/10/87247.aspx
And maybe it's more about compiler design. They make the compiler simpler by generalizing the compiling process, it doesn't have to look at operator of operands, so it lumped bitwise operations in the same category as arithmetic operators. Thereby, subjected to type widening
Link dead, archive here:
https://web.archive.org/web/20140118171646/http://blogs.msdn.com/b/oldnewthing/archive/2004/03/10/87247.aspx