There are already questions here on Stackoverflow asking why basic_fstream doesn\'t work. The answers say that char_traits
I was able to reproduce a bad_cast on my gcc (4.7.2 on AIX).
The reason you got it is that gcc library implementors optimized basic_filebuf::xsgetn (which is called from basic_istream::read) to call plain C fread to read from the file if your stream's locale is non-converting (that is, you're not trying to read a UTF-8 or maybe GB18030 file into a UTF-32 string or something), which is absolutely the right thing to do. To find out if it is non-converting, it checks codecvt::always_noconv on the codecvt facet of the locale imbued in your stream... which doesn't exist.
You can reproduce the exception by executing
std::cout << std::use_facet<
std::codecvt<std::uint8_t, char, std::mbstate_t>
>(stream.getloc()).always_noconv() << '\n';
I don't have access to Visual Studio to see why it works there (do they just call basic_filebuf::sgetc() for every char in basic_fstream::read()?), but to use basic_filestream in any case, you need to provide a codecvt facet for your combination of internal and external types (uint8_t and char, in this case).
EDIT: You're almost there, the last missing piece is the line
stream.imbue(std::locale(stream.getloc(),
new std::codecvt<uint8_t, char, std::mbstate_t>));
anywhere before stream.read or, alternatively, imbue the global: std::locale::global(std::locale(std::locale(), new std::codecvt<uint8_t, char, std::mbstate_t>)); anywhere before you construct the basic_ifstream