HttpUtility.HtmlEncode doesn't encode everything

Question

I am interacting with a web server using a desktop client program in C# and .Net 3.5. I am using Fiddler to see what traffic the web browser sends, and emulate that. Sadly t

Answer 1

Rich Strahl just posted a blog post, Html and Uri String Encoding without System.Web, where he has some custom code that encodes the upper range of characters, too.

/// <summary>
/// HTML-encodes a string and returns the encoded string.
/// </summary>
/// <param name="text">The text string to encode. </param>
/// <returns>The HTML-encoded text.</returns>
public static string HtmlEncode(string text)
{
    if (text == null)
        return null;

    StringBuilder sb = new StringBuilder(text.Length);

    int len = text.Length;
    for (int i = 0; i < len; i++)
    {
        switch (text[i])
        {

            case '<':
                sb.Append("&lt;");
                break;
            case '>':
                sb.Append("&gt;");
                break;
            case '"':
                sb.Append("&quot;");
                break;
            case '&':
                sb.Append("&amp;");
                break;
            default:
                if (text[i] > 159)
                {
                    // decimal numeric entity
                    sb.Append("&#");
                    sb.Append(((int)text[i]).ToString(CultureInfo.InvariantCulture));
                    sb.Append(";");
                }
                else
                    sb.Append(text[i]);
                break;
        }
    }
    return sb.ToString();
}

Answer 2

The AntiXSS library from Microsoft correctly encodes these characters.

AntiXSS on Codeplex

Nuget package (best way to add as a reference)

Answer 3

The return value type of HtmlEncode is a string, which is of Unicode and hence has not need to encode these characters.

If the encoding of your output stream is not compatible with these characters then use HtmlEncode like this:-

 HttpUtility.HtmlEncode(outgoingString, Response.Output);

HtmlEncode with then escape the characters appropriately.

Answer 4

It seems like HtmlEncode is just for encoding strings that are put into HTML documents, where only / < > & etc. cause problems. For URL's, just replace HtmlEncode with UrlEncode.

Answer 5

You can always replace the unwanted ASCII as follows: When this is encoded without the if statement the result string is "This means I am crying :'&'#39;) For whatever reason 'special characters' are handled and replaced with HTML char.

string text = "This means I am crying :'(";

string encoded = HttpUtility.HtmlEncode(text);
if(encoded.Contains("'"))
{
    encoded = encoded.Replace("'", "'");
}

Answer 6

@bdukes response above will do the job, but we can make it much faster if we assume that most characters will not be in this range. Note the quoted 'Ā' (unicode 0x0100)

/// <summary>.Net 2.0's HttpUtility.HtmlEncode will not properly encode
/// Unicode characters above 0xFF.  This may be fixed in newer 
/// versions.</summary>
public static string HtmlEncode(string s)
{
    // Let .Net 2.0 get right what it gets right.
    s = HttpUtility.HtmlEncode(s);

    // Search for first non-ASCII.  Hopefully none and we can just 
    // return s.
    int num = IndexOfHighChar(s, 0);
    if (num == -1)
        return s;
    int old_num = 0;
    StringBuilder sb = new StringBuilder();
    do {
        sb.Append(s, old_num, num - old_num);
        sb.Append("&#");
        sb.Append(((int)s[num]).ToString(NumberFormatInfo.InvariantInfo));
        sb.Append(';');
        old_num = num + 1;
        num = IndexOfHighChar(s, old_num);
    } while (num != -1);
    sb.Append(s, old_num, s.Length - old_num);
    return sb.ToString();
}

static unsafe int IndexOfHighChar(string s, int start)
{
    int num = s.Length - start;
    fixed (char* str = s) {
        char* chPtr = str + start;
        while (num > 0) {
            char ch = chPtr[0];
            if (ch >= 'Ā')
                return s.Length - num;
            chPtr++;
            num--;
        }
    }
    return -1;
}