Javamail NTLM Authentication Failure

Question

Trying to connect to Exchange server using NTLM in JavaMail. I can connect to SMTP, but not IMAP. I can also authenticate via the OS X Mail.app application using the ident

Answer 1

I had the same problem sending email via Exhange SMTP connector. After finding out that javamail does not support NTLMv2 authentication, I worked out a workaround that requires JCIFS library though.

I downloded the javamail api source code (https://java.net/projects/javamail/pages/Home) and edited the class com.sun.mail.auth.Ntlm to add the missing support for NTLMv2 using the JCIFS library support (http://jcifs.samba.org).

The first modification in file Ntlm.java was in method init0, and consisted of adding the missing flag NTLMSSP_NEGOTIATE_NTLM2:

private void init0() {
// ANDREA LUCIANO:
//    turn on the NTLMv2 negotiation flag in TYPE1 messages: 
//    NTLMSSP_NEGOTIATE_NTLM2   (0x00080000) 
//  See also http://davenport.sourceforge.net/ntlm.html#type1MessageExample

    type1 = new byte[256];
    type3 = new byte[256];
    System.arraycopy(new byte[] {'N','T','L','M','S','S','P',0,1}, 0,
            type1, 0, 9);
    type1[12] = (byte) 3;
    type1[13] = (byte) 0xb2;
    type1[14] = (byte) 0x08;  // ANDREA LUCIANO - added
// ...

The second modification was to replace the method generateType3Msg with this:

public String generateType3Msg(String challenge) {
    /* First decode the type2 message */
    byte[] type2 = null;
    try {
       type2 = BASE64DecoderStream.decode(challenge.getBytes("us-ascii"));
       logger.fine("type 2 message: " + toHex(type2)); // ANDREA LUCIANO - added
    } catch (UnsupportedEncodingException ex) {
       // should never happen
       assert false;
    }
    jcifs.ntlmssp.Type2Message t2m;
    try {
          t2m = new jcifs.ntlmssp.Type2Message(type2);
    } catch (IOException ex) {
          logger.log(Level.FINE, "Invalid Type 2 message", ex);
          return "";   // will fail later
    }

    final int type2Flags = t2m.getFlags();
    final int type3Flags = type2Flags & (0xffffffff ^ (jcifs.ntlmssp.NtlmFlags.NTLMSSP_TARGET_TYPE_DOMAIN | jcifs.ntlmssp.NtlmFlags.NTLMSSP_TARGET_TYPE_SERVER));

    jcifs.ntlmssp.Type3Message t3m = new jcifs.ntlmssp.Type3Message(t2m, password, ntdomain, username, hostname, type3Flags);
       return jcifs.util.Base64.encode(t3m.toByteArray());
}

The simpest way I found to patch the library is to compile the class and update the library jar file:

"c:\Program Files\Java\jdk1.5.0_22\bin\javac.exe" -cp jcifs-1.3.17.jar;javax.mail-1.5.2.jar -d . Ntlm.java 
 jar uvf javax.mail-1.5.2.jar com\sun\mail\auth\Ntlm.class

To enable debug as much as possible, I used the following code in the main method of my test class:

    final InputStream inputStream = Main.class.getResourceAsStream("/logging.properties");
    LogManager.getLogManager().readConfiguration(inputStream);

    Properties props = new Properties();

    props.put("mail.debug", "true");
    props.put("mail.debug.auth", "true");

with this logging.properties:

   # Logging
   handlers = java.util.logging.ConsoleHandler

  .level = INFO

  # Console Logging
  java.util.logging.ConsoleHandler.level = INFO

Before applying the patch the test was stuck after sending Type 1 message, because my Exchange server required NTLMv2 authentication. After the patch the athentication was done successfully.

Another solution is to send email via the ##Exchange webservice## aka EWS by using the ##EWS Java API## released and mantained by Microsoft (https://github.com/OfficeDev/ews-java-api), such as in this example:

public class Main {

/**
 * @param args
 */
public static void main(String[] args) throws Exception {

       ExchangeService exchangeService = new ExchangeService(ExchangeVersion.Exchange2007_SP1);

        ExchangeCredentials credentials = new WebCredentials("myusername","mypassword", "mydomain");

        exchangeService.setCredentials(credentials);
        exchangeService.setUrl(new URI("https://myhostname/EWS/Exchange.asmx"));

        exchangeService.setTraceEnabled(true);

        EmailMessage msg;
        msg = new EmailMessage(exchangeService);
        msg.setFrom(new EmailAddress("myuser@myserver.com"));
        msg.setSubject("Test Mail");
        msg.getToRecipients().add("myuser@gmail.com");
        msg.setBody(MessageBody.getMessageBodyFromText("Email sent by Miscrosoft Java EWS API."));
        msg.getAttachments().addFileAttachment("c:\\temp\\myattachement.pdf");
        msg.send();

}

}

But again there is a patch to apply in the inner class NTLM of EwsJCIFSNTLMScheme.java to enable NTLMv2 as indicated in the answer to this post:

How to use LDAP authentication for the Exchange Web Services connection in Java?

That is:

private class NTLM {
/** Character encoding */
public static final String DEFAULT_CHARSET = "ASCII";

/**
* The character was used by 3.x's NTLM to encode the username and
* password. Apparently, this is not needed in when passing username,
* password from NTCredentials to the JCIFS library
*/
private String credentialCharset = DEFAULT_CHARSET;

void setCredentialCharset(String credentialCharset) {
       this.credentialCharset = credentialCharset;
}

private static final int TYPE_1_FLAGS = NtlmFlags.NTLMSSP_NEGOTIATE_NTLM
             | NtlmFlags.NTLMSSP_NEGOTIATE_UNICODE
             | NtlmFlags.NTLMSSP_NEGOTIATE_NTLM2;

private String generateType1Msg(String host, String domain) {
       jcifs.ntlmssp.Type1Message t1m = new jcifs.ntlmssp.Type1Message(
                    TYPE_1_FLAGS, domain, host);
       return jcifs.util.Base64.encode(t1m.toByteArray());
}

private String generateType3Msg(String username, String password,
             String host, String domain, String challenge) {
       jcifs.ntlmssp.Type2Message t2m;
       try {
             t2m = new jcifs.ntlmssp.Type2Message(
                           jcifs.util.Base64.decode(challenge));
       } catch (IOException e) {
             throw new RuntimeException("Invalid Type2 message", e);
       }

       final int type2Flags = t2m.getFlags();
       final int type3Flags = type2Flags
                    & (0xffffffff ^ (NtlmFlags.NTLMSSP_TARGET_TYPE_DOMAIN | NtlmFlags.NTLMSSP_TARGET_TYPE_SERVER));

       jcifs.ntlmssp.Type3Message t3m = new jcifs.ntlmssp.Type3Message(
                    t2m, password, domain, username, host, type3Flags);
       return jcifs.util.Base64.encode(t3m.toByteArray());
}

}

I tried and it worked for me.

Answer 2

I have noticed that - via SMTP - the NTLM Authentication did not work with an older version of javax.mail (up to 1.4.1) but it now works with version 1.4.5. And the user name to be specified was of the form "domain\username". Could be that same effect (difference in versions of javax.mail) would also apply to IMAP.

Answer 3

From what I recall about NTLM, and your NTLM debug messages I can gather the following:

• NTLM was designed for single sign on, thus taking credentials from the windows machine its running on - especially the JDK implementation of NTLM.
  • NTLM has two versions, well three to be exact. NTLM v1, NTLMv2 and another version I can't recall at the moment. NTLM v1 has a security hole which allows you to really use a username and password and connect using the NTLM protocol. In NTLM v2 it was fixed, which forces the implementation to take the password (the hashed pass) from the logged in Windows machine.
  • It seems that the NTLM protocol in your case stops after the first message sent by the Exchange server. Notice it has flags which declares which type of NTLM is used and other such as : Encryption, etc.

I suggest you try following the road in which the credentials (u/p) are taken automatically by the JDK from the client Windows machine.

Answer 4

Try setting the domain "" to the properties of the imap store :

properties.setProperty("mail.imap.auth.ntlm.domain","");

Since in SMTP you are login using LOGIN, the use of the domain is not necessary. But in NTLM the domain is mandatory.

Answer 5

Here is my complete working solution:

Using netbeans, create a new java application project. Put this code in there:

package javaapplication4;
import java.util.Date;
import java.util.Properties;

import javax.mail.Message;
import javax.mail.PasswordAuthentication;
import javax.mail.Session;
import javax.mail.Transport;
import javax.mail.internet.MimeMessage;


public class JavaApplication4 {
    public static void main(String[] args) throws Exception {
           new JavaApplication4().send_email();
    }
    private void send_email() throws Exception
    {
        Properties props = new Properties();
        props.put("mail.smtp.host", "smtp.yourserver.net");
        props.put("mail.from", "yourusername@youremailaddress.com");
        props.put("mail.smtp.starttls.enable", "true");
        props.put("mail.smtp.ssl.enable", "false");
        props.put("mail.smtp.auth", "true");
        props.put("mail.smtp.port", "587");

        Authenticator authenticator = new Authenticator();
        props.setProperty("mail.smtp.submitter", authenticator.getPasswordAuthentication().getUserName());

        Session session = Session.getInstance(props, authenticator);
        MimeMessage msg = new MimeMessage(session);
        msg.setFrom();
        msg.setRecipients(Message.RecipientType.TO, "yourusername@youremailaddress.com");  
            // also tried @gmail.com
        msg.setSubject("JavaMail ssl test");
        msg.setSentDate(new Date());
        msg.setText("Hello, world!\n");

        Transport transport;
        transport = session.getTransport("smtp");
        transport.connect();
        msg.saveChanges(); 
        transport.sendMessage(msg, msg.getAllRecipients());
        transport.close();
    }
    private class Authenticator extends javax.mail.Authenticator {
       private PasswordAuthentication authentication;
       public Authenticator() {
           String username = "yourusername@youremailaddress.com";
           String password = "yourpassword";
           authentication = new PasswordAuthentication(username, password);
       }
       protected PasswordAuthentication getPasswordAuthentication() {
           return authentication;
       }
   }
}

Change the username, passwords, ports, and properties to your specific settings.

You will need to get the javamail-1.4.7 and load the mail.jar from (http://www.oracle.com/technetwork/java/index-138643.html) into the project.

One thing we did which shed light on what our parameters should be is to download Thunderbird mail client which can auto-discover information about the exchange server to make sure all of our settings were right. If you cant convince thunderbird to connect, then this code likewise shouldn't be expected to work.