How can I modify a POST request using a custom IHttpModule and an HttpRequest filter?

Question

Overview

I want to be able to modify request parameters and content to 3rd party web services (ArcGIS Server). This will be used to create a security layer that exists between any client application and the server application.

I think that I have found a solution but I am current having some difficulties in the implementation.

Potential Solution: Modify Request with a Custom Request Filter

For the solution I implemented a custom request filter loosely based on the sample shown on MSDN. I have 'enhanced' the code so that I can search and replace the necessary content using regular expressions. This involves:

1. Converting the content (stored in a byte array) into a string.
2. Searching the string and performing any necessary modifications.
3. Converting the modified string into a byte array and writing it to the buffer.

An example is shown below:

public override int Read(byte[] buffer, int offset, int count)
{
    int bytesRead = _stream.Read(buffer, offset, count);

    string orgContent = Encoding.UTF8.GetString(buffer, offset, bytesRead);
    string orgContentDecoded = HttpUtility.UrlDecode(orgContent);

    string layersPattern = @"&layers=(show|hide|include|exclude):([0-9]+,?)+";
    Regex layersRegex = new Regex(layersPattern, RegexOptions.IgnoreCase);

    string[] permittedLayers = new string[] { "0" , "1" };
    string replacementLayers = "&layers=show:" + String.Join(",", permittedLayers);
    string newContentDecoded = layersRegex.Replace(orgContentDecoded, replacementLayers);

    string newContent =  newContentDecoded.Replace(",", "%2C").Replace(":", "%3A");

    byte[] newBuffer = Encoding.UTF8.GetBytes(newContent);
    int newByteCountLength = Encoding.UTF8.GetByteCount(newContent);

    Encoding.UTF8.GetBytes(newContent, 0, Encoding.UTF8.GetByteCount(newContent), buffer, 0);

    return bytesRead;
}

This seems to work well so long as the modified content length is not different than the original content length. For instance, if I replace a 1 with a 2 everything works. However, if I replace a 1 with a 10 (thereby increasing the message size by 1) then I receive an error from ArcGIS Server that the format is unsupported.

This has brought two concerns to my attention:

1. The current implementation does not handle chunked requests. That is, if the request sie is large enough Read may be called multiple times for a single request. How should chunking be handled in this scenario?
2. What is the root cause of the error message? Is the problem related to the content length being different than the stream length? How do I correctly modify the content so that changing its length is not an issue?

Any thoughts?

Answer 1

The answer to part two of this question is to return the modified content size, not the size of the original stream. Behold!

// return bytesRead;
return newByteCountLength;

Answer 2

This question and your answer were really useful for me, however the answer is not the whole story if you're trying to insert larger amounts of data into the stream:

Returning the modified content size is only valid if the data you're inserting into the stream will not take the number of bytes read into the buffer over the count value. If you try to insert too much data, you're either going to find that the buffer object isn't large enough to hold the data you're inserting, or that by writing more than count bytes to the buffer, you're overwriting some data at the end of the buffer that was meant to be left alone.

If you need to insert more data than the current buffer object can accommodate then you'll have to buffer the data in a separate byte array and copy it in chunks as the calls to stream.read are made.

Answer 3

As Chris McKeown states, this is not really the best technique for filtering unless you promise to not modify the size of the data. For completeness of this answer, I've posted a sample project which demonstrates how to handle filtering the request and the response using the buffered technique if you are interested.

https://github.com/snives/HttpModuleRewrite

Also for questions relating to troubleshooting HttpModules for filtering this post was also very helpful Is it possible to modify the content of HttpRequest POST in an IIS HttpModule?