How to configure the number of allowed pending requests in Apache async client

Question

I'm developing an HTTP client application, using Apache httpasyncclient version 4.0.2.

I would like to configure the maximum number of pending requests. Initially I assumed this number is the same as the maximum number of connections. I set this to 20 in the following way:

    final CloseableHttpAsyncClient httpclient;

In the constructor:

    final NHttpConnectionFactory<ManagedNHttpClientConnection> connFactory = new ManagedNHttpClientConnectionFactory(new DefaultHttpRequestWriterFactory(), new DefaultHttpResponseParserFactory(), HeapByteBufferAllocator.INSTANCE);
    final IOReactorConfig ioReactorConfig = IOReactorConfig.custom()
            .setIoThreadCount(4)
            .setConnectTimeout(30000)
            .setSoTimeout(30000)
            .build();
    final PoolingNHttpClientConnectionManager connManager = new PoolingNHttpClientConnectionManager(new DefaultConnectingIOReactor(ioReactorConfig), connFactory);
    final int maxConns = 20;
    connManager.setDefaultMaxPerRoute(maxConns);
    connManager.setMaxTotal(maxConns);
    httpclient = HttpAsyncClientBuilder.create().setConnectionManager(connManager).build();
    httpclient.start();

and later:

    final BasicAsyncRequestProducer requestProducer = new BasicAsyncRequestProducer(URIUtils.extractHost(URI.create(serverAddress)), request) {
        @Override
        public void requestCompleted(HttpContext context) {
            pendings.add(callback);
            logMessage(Direction.REQUEST, req);
            handler.onContentWriteCompleted();
        }
    };
    httpclient.execute(requestProducer, HttpAsyncMethods.createConsumer(), new HttpClientContext(), callback);

where callback is where I handle the response.

As a proof of concept it fails. Indeed get do four threads running, but when I try to send 20 messages simultaneously, only 8 are sent immediately, the rest must wait until the server responds to them.

Apache debug messages are indicating that 20 connections have indeed been created. It seems that more configuration must be done.

?

Answer 1

Apache HttpAsyncClient maintains an unbounded request execution queue and does not attempt to limit the number of pending requests. Various applications may or may not want to throttle request rate and there is no easy way to satisfy them all.

One can however fairly easily throttle the number of concurrent requests using a simple semaphore.

final Semaphore semaphore = new Semaphore(maxConcurrencyLevel);
for (int i = 0; i < n; i++) {

    semaphore.acquire();
    this.httpclient.execute(
            new BasicAsyncRequestProducer(target, request),
            new MyResponseConsumer(),
            new FutureCallback<HttpResponse>() {

                @Override
                public void completed(final HttpResponse result) {
                    semaphore.release();
                }

                @Override
                public void failed(final Exception ex) {
                    semaphore.release();
                }

                @Override
                public void cancelled() {
                    semaphore.release();
                }

            });
}