Chunked http decoding in java?

Question

I am decoding http packets. And I faced a problem that chunk problem. When I get a http packet it has a header and body. When transefer-encoding is chunked I don't know what to do ?

Is there a useful API or class for dechunk the data in JAVA ?

And if someone , experienced about http decoding , please show me a way how to do this ?

Answer 1

Use a fullworthy HTTP client like Apache HttpComponents Client or just the Java SE provided java.net.URLConnection (mini tutorial here). Both handles it fully transparently and gives you a "normal" InputStream back. HttpClient in turn also comes with a ChunkedInputStream which you just have to decorate your InputStream with.

If you really insist in homegrowing a library for this, then I'd suggest to create a class like ChunkedInputStream extends InputStream and write logic accordingly. You can find more detail how to parse it in this Wikipedia article.

Answer 2

Apache HttpComponents

Oh, and if we are talking about the client side, HttpUrlConnection does this as well.

Answer 3

If you are looking for a simple API try Jodd Http library (http://jodd.org/doc/http.html). It handles Chunked transfer encoding for you and you get the whole body as a string back.

From the docs:

HttpRequest httpRequest = HttpRequest.get("http://jodd.org");
HttpResponse response = httpRequest.send();

System.out.println(response);

Answer 4

Here is quick-and-dirty alternative that requires no dependency except Oracle JRE:

private static byte[] unchunk(byte[] content) throws IOException {
    ByteArrayInputStream bais = new ByteArrayInputStream(content);
    ChunkedInputStream cis = new ChunkedInputStream(bais, new HttpClient() {}, null);
    return readFully(cis);
}

It uses the same sun.net.www.http.ChunkedInputStream as java.net.HttpURLConnection does behind the scene.

This implementation doesn't provide detailed exceptions (line numbers) on wrong content format.

It works with Java 8 but could fail in with next release. You've been warned.

Could be useful for prototyping though.

You can choose any readFully implementation from Convert InputStream to byte array in Java.