Null reference exception when generating a url with UrlHelper.Action method

Question

For some reason, when certain bots visit the site, generating a url with the UrlHelper.Action method raises a null exception from System.Web.HttpServerVarsCollection.Get. I've done some debugging and the call stack originiates with an attempt to get the 'HTTP_X_ORIGINAL_URL' from the HttpContextBase.Request.ServerVariables collection.

If I visit the same address directly from a browser - no problem. The page is server and no error is logged. It only seems to occur when visited by a bot.

Not sure if it's relevant or not, but the site was just migrated to IIS 7.5. Still using .NET 2.0 in Integrated mode.

Looking at the code as reversed by Reflector the only place where a null exception can occur directly in the Get method is the call to this._request.FetchServerVariables. As if the complete request was not set up properly.

Has anyone else faced this problem or discovered a workaround? Why would the request be set up differently when visited by a bot?

UPDATE: Some additional debugging has shown that the HttpServerVarsCollection was Disposed, along with it's parent HttpRequest object. The question now is - how can the Request object returned by HttpContext.Current be exposed before the request is complete?

HttpServerVarsCollection.Get Method

public override string Get(string name)
{
    if (!this._populated)
    {
        string simpleServerVar = this.GetSimpleServerVar(name);
        if (simpleServerVar != null)
        {
            return simpleServerVar;
        }
        this.Populate();
    }
    if (this._iis7workerRequest == null)
    {
        return this.GetServerVar(base.BaseGet(name));
    }
    string serverVar = this.GetServerVar(base.BaseGet(name));
    if (string.IsNullOrEmpty(serverVar))
    {
        // Only place null reference can happen
        serverVar = this._request.FetchServerVariable(name);
    }
    return serverVar;
}

Full stack trace

NullReferenceException: Object reference not set to an instance of an object.]
   System.Web.HttpServerVarsCollection.Get(String name) +8645730
   System.Collections.Specialized.NameValueCollection.get_Item(String name) +7
   System.Web.Mvc.PathHelpers.GenerateClientUrlInternal(HttpContextBase httpContext, String contentPath) in C:\Dev\Site\MVC\Microsoft\src\SystemWebMvc\Mvc\PathHelpers.cs:39
   System.Web.Mvc.PathHelpers.GenerateClientUrl(HttpContextBase httpContext, String contentPath) in C:\Dev\Site\MVC\Microsoft\src\SystemWebMvc\Mvc\PathHelpers.cs:21
   System.Web.Mvc.UrlHelper.GenerateUrl(String routeName, String actionName, String controllerName, RouteValueDictionary routeValues, RouteCollection routeCollection, RequestContext requestContext, Boolean includeImplicitMvcValues) in C:\Dev\Site\MVC\Microsoft\src\SystemWebMvc\Mvc\UrlHelper.cs:136
   System.Web.Mvc.UrlHelper.GenerateUrl(String routeName, String actionName, String controllerName, RouteValueDictionary routeValues) in C:\Dev\Site\MVC\Microsoft\src\SystemWebMvc\Mvc\UrlHelper.cs:101
   System.Web.Mvc.UrlHelper.Action(String actionName, String controllerName, Object routeValues) in C:\Dev\Site\MVC\Microsoft\src\SystemWebMvc\Mvc\UrlHelper.cs:51
   www.CmsExtensions.Document(UrlHelper urlHelper, String path) in C:\Dev\Site\www\Code\CmsExtensions.cs:33
   www.CmsExtensions.Document(UrlHelper urlHelper, Document document) in C:\Dev\Site\www\Code\CmsExtensions.cs:20
   www.<>c__DisplayClass17.<Load>b__c(Document d) in C:\Dev\Site\www\Global.asax.cs:251
   Fringine.Cms.DocumentContentParser.ReplaceDocumentRefs(IResolvingDocumentCache cache, Match match) +258
   Fringine.Cms.<>c__DisplayClass4.<ParseContent>b__2(Match m) +17
   System.Text.RegularExpressions.RegexReplacement.Replace(MatchEvaluator evaluator, Regex regex, String input, Int32 count, Int32 startat) +234
   System.Text.RegularExpressions.Regex.Replace(String input, MatchEvaluator evaluator, Int32 count, Int32 startat) +28
   System.Text.RegularExpressions.Regex.Replace(String input, MatchEvaluator evaluator) +38
   System.Text.RegularExpressions.Regex.Replace(String input, String pattern, MatchEvaluator evaluator, RegexOptions options) +47
   Fringine.Cms.DocumentContentParser.ParseContent(String content, IResolvingDocumentCache cache) +83
   Fringine.Cms.ResolvingDocumentCache.<Parse>b__0(String d) +21
   Fringine.Cms.DocumentCache.GetParsedData(String id, String content, IDocumentService documentService, Func`2 parser) +216
   Fringine.Cms.ResolvingDocumentCache.Parse(String id, String content) +67
   Fringine.Cms.CachedDocument.GetSummary() +966
   Fringine.Cms.CachedDocument.get_Summary() +19
   ASP.views_document_widget_feeddocumentsummary_ascx.__Render__control1(HtmlTextWriter __w, Control parameterContainer) +841
   System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children) +256
   System.Web.UI.Control.RenderChildren(HtmlTextWriter writer) +19
   System.Web.UI.Control.Render(HtmlTextWriter writer) +10
   System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter) +27
   System.Web.UI.Control.RenderControl(HtmlTextWriter writer, ControlAdapter adapter) +99
   System.Web.UI.Control.RenderControl(HtmlTextWriter writer) +25
   System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children) +134
   System.Web.UI.Control.RenderChildren(HtmlTextWriter writer) +19
   System.Web.UI.Page.Render(HtmlTextWriter writer) +29
   System.Web.Mvc.ViewPage.Render(HtmlTextWriter writer) in C:\Dev\Site\MVC\Microsoft\src\SystemWebMvc\Mvc\ViewPage.cs:107
   System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter) +27
   System.Web.UI.Control.RenderControl(HtmlTextWriter writer, ControlAdapter adapter) +99
   System.Web.UI.Control.RenderControl(HtmlTextWriter writer) +25
   System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint) +1266

Answer 1

I had this problem, but it was unrelated to URL rewrite modules.

In my case, I accidentally cached an instance of UrlHelper in a static field, and later requests encountered a disposed instance from earlier requests.

Answer 2

I know this question is fairly old now, but I recently found myself in a very similar situation where my UrlHelper was giving me null reference exception on System.Web.HttpServerVarsCollection.Get as well.

The problem was indeed HTTP_X_ORIGINAL_URL and this particular server variable comes from the URL Rewrite 2.0 Module on IIS.

Interestingly enough this module was installed but not being used. However, just it's presence was enough to cause the problem. Uninstalling it made the error go away.

If you need to do URL re-writes there are other modules out there or you could do it at the application level.

Hope that helps.

Answer 3

I ran into this, in my case was able to solve by switching to a non-extension method usage of UrlHelper. I had one extension method calling another, but passing the UrlHelper to the 2nd method.

This way was broken:

public static string Script(this UrlHelper helper, string fileName)
{
    return Asset(helper, "~/js/", fileName);
}

private static string Asset(this UrlHelper helper, string path, string fileName)
{
    return helper.Content(string.Format("{0}/{1}/{2}",
           path,
           Version,
           fileName));
}

This way works:

public static string Script(this UrlHelper helper, string fileName)
{
    return Asset(helper, "~/js/", fileName);
}

private static string Asset(UrlHelper helper, string path, string fileName)
{
    return helper.Content(string.Format("{0}/{1}/{2}",
           path,
           Version,
           fileName));
}

Note the function signature difference of the Asset method.