Configuring Apache HttpClient to access service through proxy/load-balancer (overriding Host header)

Question

I am having a problem getting the Apache HttpClient to connect to a service external to my virtualised development environment. To access the internet (e.g. api.twitter.com) I need to call a local URL (e.g. api.twitter.com.dev.mycompany.net), which then forwards the request to real host.

The problem is, that to whatever request I send, I get a 404 Not Found response.

I have tried debugging it using wget, and it appears the problem is, that the destination server identifies the desired resource by using both the request URL and the hostname in the Host header. Since the hostname does not match, it is unable to locate the resource.

I have (unsuccessfully) tried to override the Host header by setting the http.virtual-host parameter on the client like this:

HttpClient client = new DefaultHttpClient();
if (envType.isWithProxy()) {
    client.getParams().setParameter(ClientPNames.VIRTUAL_HOST, "api.twitter.com");
}

Technical details:

1. Client is used as an executor in RESTeasy to call the REST API. So "manually" setting the virtual host (as described here) is not an option.

2. Everything is done via HTTPS/SSL - not that I think it makes a difference.

Edit 1: Using a HttpHost instead of a String does not have the desired effect either:

HttpClient client = new DefaultHttpClient();
if (envType.isWithProxy()) {
    HttpHost realHost = new HttpHost("api.twitter.com", port, scheme);
    client.getParams().setParameter(ClientPNames.VIRTUAL_HOST, realHost);
}

Edit 2: Further investigation has revealed, that the parameter needs to be set on the request object. The following is the code v. 4.2-aplha1 of HttpClient setting the virtual host:

HttpRequest orig = request;
RequestWrapper origWrapper = wrapRequest(orig);
origWrapper.setParams(params);
HttpRoute origRoute = determineRoute(target, origWrapper, context);

virtualHost = (HttpHost) orig.getParams().getParameter(
            ClientPNames.VIRTUAL_HOST);

paramsare the parameters passed from the client. But the value for 'virtualHost' is read from the request parameters.

So this changes the nature of the question to: How do I set the VIRTUAL_HOST property on the requests?

Answer 1

ClientPNames.VIRTUAL_HOST is the right parameter for overriding physical host name in HTTP requests. I would just recommend setting this parameter on the request object instead of the client object. If that does not produce the desired effect please post the complete wire / context log of the session (see logging guide for instructions) either here or to the HttpClient user list.

---

Follow-up

OK. Let's take a larger sledge hammer. One can override content of the Host header using an interceptor.

DefaultHttpClient client = new DefaultHttpClient();
client.addRequestInterceptor(new HttpRequestInterceptor() {

    public void process(
            final HttpRequest request, 
            final HttpContext context) throws HttpException, IOException {
        request.setHeader(HTTP.TARGET_HOST, "www.whatever.com");
    }
});

One can make the interceptor clever enough to override the header selectively, only for specific hosts.