Hadoop Web Authentication using Kerberos

Question

I configured hadoop using kerberos, everything works fine, I can browse hdfs, submit jobs, etc. But failed http web authentication.

I use hadoop-0.20.2 in cdh3u2, which supports HTTP SPNEGO.

HTTP authentication related configurations in core-site.xml are as follows:

  <!-- HTTP web-consoles Authentication -->
  <property>
    <name>hadoop.http.filter.initializers</name>
    <value>org.apache.hadoop.security.AuthenticationFilterInitializer</value>
  </property>

  <property>
    <name>hadoop.http.authentication.type</name>
    <value>kerberos</value>
  </property>

  <property>
    <name>hadoop.http.authentication.token.validity</name>
    <value>36000</value>
  </property>

  <property>
    <name>hadoop.http.authentication.signature.secret.file</name>
    <value>/home/hadoop/hadoop/conf/http-secret-file</value>
  </property>

  <property>
    <name>hadoop.http.authentication.cookie.domain</name>
    <value></value>
  </property>

  <property>
    <name>hadoop.http.authentication.simple.anonymous.allowed</name>
    <value>false</value>
  </property>

  <property>
    <name>hadoop.http.authentication.kerberos.principal</name>
    <value>HTTP/hz169-91.i.site.com@I.NETEASE.COM</value>
  </property>

  <property>
    <name>hadoop.http.authentication.kerberos.keytab</name>
    <value>/home/hadoop/hadoop/conf/http.keytab</value>
  </property>
</configuration>

During startup, http authentication succeeded.

2011-11-15 15:43:59,106 INFO org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler: Initialized, principal [HTTP/hz169-91.i.site.com@I.NETEASE.COM] from keytab [/home/hadoop/hadoop/conf/http.keytab]

After look into the code, I found out that AuthenticationFilter gets null token during doFilter, so, authentication begins (code below), but authorization in httpservletrequest is null, so, every time I reload my page, one log appears.

2011-11-15 15:47:52,190 WARN org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler: SPNEGO starting

---

// org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler
public AuthenticationToken authenticate(HttpServletRequest request, final HttpServletResponse response)
    throws IOException, AuthenticationException {
    AuthenticationToken token = null;
    String authorization = request.getHeader(KerberosAuthenticator.AUTHORIZATION);
    if (authorization == null || !authorization.startsWith(KerberosAuthenticator.NEGOTIATE)) {
      response.setHeader(KerberosAuthenticator.WWW_AUTHENTICATE, KerberosAuthenticator.NEGOTIATE);
      response.setStatus(HttpServletResponse.SC_UNAUTHORIZED);
      if (authorization == null) {
        LOG.warn("SPNEGO starting");
      } else {
        LOG.warn("'" + KerberosAuthenticator.AUTHORIZATION + "' does not start with '" +
            KerberosAuthenticator.NEGOTIATE + "' :  {}", authorization);
      }

Is there any configuration errors, or just my browser doesn't support SPNEGO. I use Chrome v16 in Ubuntu 11.04.

Does anybody have clues to help me figure it out?

Thanks.

Answer 1

First: Thankyou for posting a complete and working example on how to configure Hadoop web consoles for SPNNEGO - I had trouble finding a good example.

Your example works for me after modifying paths to config files (I created hadoop.http.authentication.signature.secret.file by getting some random bytes from /dev/random, which I'm assuming is the right thing to do, although I can't find any documentation supporting that theory).

Google Chrome does support SPNNEGO from version 6.0.472 and forward. However, it seems that on Linux and OSX you have to pass it a list of servers for which it's OK to enable it as documented here. So, try adding *--auth-server-whitelist="*example.com,*foobar.com,baz" to the cmdline when starting Chrome.

Another way of debugging this would be to use a simpler browser. I would recommend curl, if your curl has GSS-Negotiate support. Check by running curl --version

$ curl --version
curl 7.19.7 (i486-pc-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8k zlib/1.2.3.3 libidn/1.15
Protocols: tftp ftp telnet dict ldap ldaps http file https ftps 
Features: GSS-Negotiate IDN IPv6 Largefile NTLM SSL libz 

If GSS-Negotiate is in the Features list, you can use curl to try to access for example the namenode Web Console:

$ curl -v -u foo --negotiate http://your.namenode.tld:50070

Just press enter when asked for host password.

This should give you a better idea on what's going on between the client and the server.