HttpClient fails with Handshake Failed in Android 5.0 Lollipop

Question

DefaultHttpClient in Android 5.0 Lollipop seems to be broken. It can not set the connection to some sites that were successfully set by previous versions of Android.

For example I try to connect to https://uralsg.megafon.ru

//Create httpclient like in https://stackoverflow.com/questions/18523784/ssl-tls-protocols-and-cipher-suites-with-the-androidhttpclient
HttpClient client = new DefaultHttpClient(manager, params);
HttpGet httpGet = new HttpGet("https://uralsg.megafon.ru");
HttpResponse client = httpclient.execute(httpGet);

This code works in Android 2.3-4.4, but fails on Android 5.0 (devices and emulator) with error Connection closed by peer. Of course this is understandable because Android 5.0 tries to connect this old server with TLSv1.2 and modern ciphers and it does not support them.

Ok, using the sample code in SSL/TLS protocols and cipher suites with the AndroidHttpClient we limit the protocol and cipher to TLSv1 and SSL_RSA_WITH_RC4_128_MD5. Now it fails with a different error:

javax.net.ssl.SSLHandshakeException: Handshake failed
caused by 
    error:140943FC:SSL routines:SSL3_READ_BYTES:sslv3 alert bad record mac 
    (external/openssl/ssl/s3_pkt.c:1286 0x7f74c1ef16e0:0x00000003) 
    at com.android.org.conscrypt.NativeCrypto.SSL_do_handshake

And of course this code runs smoothly on Android 2.3-4.4.

I examined the traffic with wireshark:

302 4002.147873000  192.168.156.30  83.149.32.13    TLSv1   138 Client Hello
303 4002.185362000  83.149.32.13    192.168.156.30  TLSv1   133 Server Hello
304 4002.186700000  83.149.32.13    192.168.156.30  TLSv1   1244    Certificate
305 4002.186701000  83.149.32.13    192.168.156.30  TLSv1   63  Server Hello Done
307 4002.188117000  192.168.156.30  83.149.32.13    TLSv1   364 Client Key Exchange, Change Cipher Spec, Encrypted Handshake Message
308 4002.240695000  83.149.32.13    192.168.156.30  TLSv1   61  Alert (Level: Fatal, Description: Bad Record MAC)

You can see that connection was established but server alerted because it probably could not decode encrypted handshake message.

I didn't manage to connect to https://uralsg.megafon.ru using HttpClient on Android 5.0. Stock browser does connect it though. Android 2.3-4.4 connects this site in any way without any difficulties.

Is there any way to make it possible for HttpClient to connect such sites? This is only one example, I am sure there are plenty of legacy servers that couldn't be connected by Android 5.0 and HttpClient.

Answer 1

update: it turned out to be a bug in the back-end, not android 5, though indeed with the cipher in question.

I had the same problem. For me it turned out to be the cipher TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 which was chosen from android 5's (updated) set of default ciphers.

As soon as i removed it from the client list of acceptable ciphers, connections worked again.

The android 5 change log mentions:

• AES-GCM (AEAD) cipher suites are now enabled,

I'm pretty sure this is the culprit. As soon as TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 is preferred (by the server), the connection will fail.

Note that TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 works.

My guess is that either the Android implementation of TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 is buggy, or the one of the server you're talking to.

Solutions:

1. Remove TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 from the available ciphers on the server (no app redeployment needed).
2. Remove TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 from the list of ciphers the client offers (during the CLIENT_HELLO).

You can do that on the client side by implementing your own SSLSocketFactory and calling

sslSocket.setEnabledCipherSuites(String[] suites);

on SSLSocket creation.

edit: note that this isn't necessarily an android bug, it might be that the server implementation is faulty. if your problem is indeed caused by the cipher, please leave a comment on the android bug tracker](https://code.google.com/p/android/issues/detail?id=81603). thank you!

Answer 2

I tried changing the cipherSuites in a custom socket factory, but that did not help. In my case, I had to remove the TLSv1.1 and TLSv1.2 protocols from the socket's EnabledProtocols. It appears that some older servers do not handle the protocol negotiation for the new protocols very well. There are various examples out there for creating a custom socket factory, such as How to override the cipherlist sent to the server by Android when using HttpsURLConnection?, and other ones for Apache sockets. That being done, I just called the following AdjustSocket method to remove the protocols.

private void AdjustSocket(Socket socket)
{
    String[] protocols = ((SSLSocket) socket).getSSLParameters().getProtocols();
    ArrayList<String> protocolList = new ArrayList<String>(Arrays.asList(protocols));

    for (int ii = protocolList.size() - 1; ii >= 0; --ii )
        {
        if ((protocolList.get(ii).contains("TLSv1.1")) || (protocolList.get(ii).contains("TLSv1.2")))
            protocolList.remove(ii);
        }

    protocols = protocolList.toArray(new String[protocolList.size()]);
    ((SSLSocket)socket).setEnabledProtocols(protocols);
}