Percent-encoded slash (“/”) is decoded before the request dispatch

Question

I have an URL containing several slash characters (/) as a part of the filename (not the URL). But when I send http request, the percent-encoded %2F

Answer 1

Wow, this is quite a conundrum. There is a bug report about this on Microsoft Connect. It seems there is a workaround for ASP.net which won't help you in PowerShell.

Here's where it gets really weird though. I am running PowerShell 4.0. I can reproduce this problem when running in the console host. However, if I run the exact same code in the ISE Host, it works flawlessly.

I don't have any idea how or why. I even remoted into another system not on my network to make sure I didn't somehow change anything weird on my system. Same result. Bottom line:

$a = 'https://www.googleapis.com/download/storage/v1/b/chromium-browser-continuous/o/Win_x64%2F292817%2Fchrome-win32.zip?generation=1409504089694000&alt=media'
Invoke-WebRequest -Uri $a

This works in ISE, does not work in console host. I even tried it with -UseBasicParsing to make sure it wasn't a weird quirk of DOM parsing.

A Dirty Workaround

I took the C# code in Simon Maurier's answer to "How to make System.Uri not to unescape %2f (slash) in path?" and I adapted it for use in PowerShell:

$uriFixerDef = @'
using System;
using System.Reflection;

public class UriFixer
{
    private const int UnEscapeDotsAndSlashes = 0x2000000;
    private const int SimpleUserSyntax = 0x20000;

    public static void LeaveDotsAndSlashesEscaped(Uri uri)
    {
        if (uri == null)
            throw new ArgumentNullException("uri");

        FieldInfo fieldInfo = uri.GetType().GetField("m_Syntax", BindingFlags.Instance | BindingFlags.NonPublic);
        if (fieldInfo == null)
            throw new MissingFieldException("'m_Syntax' field not found");

        object uriParser = fieldInfo.GetValue(uri);
        fieldInfo = typeof(UriParser).GetField("m_Flags", BindingFlags.Instance | BindingFlags.NonPublic);
        if (fieldInfo == null)
            throw new MissingFieldException("'m_Flags' field not found");

        object uriSyntaxFlags = fieldInfo.GetValue(uriParser);

        // Clear the flag that we do not want
        uriSyntaxFlags = (int)uriSyntaxFlags & ~UnEscapeDotsAndSlashes;
        uriSyntaxFlags = (int)uriSyntaxFlags & ~SimpleUserSyntax;
        fieldInfo.SetValue(uriParser, uriSyntaxFlags);
    }
}
'@

Add-Type -TypeDefinition $uriFixerDef

$u = 'https://www.googleapis.com/download/storage/v1/b/chromium-browser-continuous/o/Win_x64%2F292817%2Fchrome-win32.zip?generation=1409504089694000&alt=media'

[UriFixer]::LeaveDotsAndSlashesEscaped($u)

Invoke-WebRequest -Uri $u

I first tested it in ISE and then later found that ISE was working no matter what. So I did in fact try this in a clean console host environment and before invoking the method, I got notfound. After invoking, worked.

As it says in the linked answer, this is an ugly hack, may break in future versions, etc.

I do hope it helps, this is an interesting issue.