How to add encoding information to the response stream in ASP.NET?

Question

I have following piece of code:

public void ProcessRequest (HttpContext context) 
{
    context.Response.ContentType = "text/rtf; charset=UTF-8";
    context.Response.Charset = "UTF-8";
    context.Response.ContentEncoding = System.Text.Encoding.UTF8;
    context.Response.AddHeader("Content-disposition", "attachment;filename=lista_obecnosci.csv");
    context.Response.Write("ąęćżźń󳥌ŻŹĆŃŁÓĘ");
}

When I try to open generated csv file, I get following behavior:

• In Notepad2 - everything is fine.
• In Word - conversion wizard opens and asks to convert the text. It suggest UTF-8, which is somehow ok.
• In Excel - I get real mess. None of those Polish characters can be displayed.

I wanted to write those special encoding-information characters in front of my string, i.e.

context.Response.Write((char)0xef);
context.Response.Write((char)0xbb);
context.Response.Write((char)0xbf);

but that won't do any good. The response stream is treating that as normal data and converts it to something different.

I'd appreciate help on this one.

Answer 1

What you call "encoding-information" is actually a BOM. I suspect each of those "characters" is getting encoded separately. To write the BOM manually, you have to write it as three bytes, not three characters. I'm not familiar with the .NET I/O classes, but there should be a method available to you that takes a byte or byte[] parameter and writes them directly to the file.

By the way, the UTF-8 BOM is optional; in fact, its use is discouraged by the Unicode Consortium. If you don't have a specific reason for using it, save yourself some hassle and leave it out.

EDIT: I just remembered you can also write the actual BOM character, '\uFEFF', and let the encoder handle it:

context.Response.Write('\uFEFF');

Answer 2

I ran into the same problem, and this was my solution:

context.Response.BinaryWrite(System.Text.Encoding.UTF8.GetPreamble());
context.Response.Write("ąęćżźń󳥌ŻŹĆŃŁÓĘ");

Answer 3

I think the problem is with Excel based on Microsoft Excel mangles Diacritics in .csv files. To prove this, copy your sample output string of ąęćżźń󳥌ŻŹĆŃŁÓĘ and paste into a test file using your favorite editor, and save as a UTF-8 encoded .csv file. Open in Excel and see the same issues.

Answer 4

The answer from Alan Moore translated to VB:

Context.Response.Write(""c)