byte-order-mark

I'm having a problem with UTF8 encoding in my asp.net mvc 2 application in C#. I'm trying let user download a simple text file from a string. I am trying to get bytes array with the following line: var x = Encoding.UTF8.GetBytes(csvString); but when I return it for download using: return File(x, ..., ...); I get a file which is without BOM so I don't get Croatian characters shown up correctly. This is because my bytes array does not include BOM after encoding. I triend inserting those bytes manually and then it shows up correctly, but that's not the best way to do it. I also tried creating

In .NET, I'm trying to use Encoding.UTF8.GetString method, which takes a byte array and converts it to a string . It looks like this method ignores the BOM (Byte Order Mark) , which might be a part of a legitimate binary representation of a UTF8 string, and takes it as a character. I know I can use a TextReader to digest the BOM as needed, but I thought that the GetString method should be some kind of a macro that makes our code shorter. Am I missing something? Is this like so intentionally? Here's a reproduction code: static void Main(string[] args) { string s1 = "abc"; byte[] abcWithBom;

I'm consuming a data feed that has recently added a Unicode BOM header (U+FEFF), and my rake task is now messed up by it. I can skip the first 3 bytes with file.gets[3..-1] but is there a more elegant way to read files in Ruby which can handle this correctly, whether a BOM is present or not? With ruby 1.9.2 you can use the mode r:bom|utf-8 text_without_bom = nil #define the variable outside the block to keep the data File.open('file.txt', "r:bom|utf-8"){|file| text_without_bom = file.read } or text_without_bom = File.read('file.txt', encoding: 'bom|utf-8') or text_without_bom = File.read('file