Hadoop IOException failure to login

Question

I\'m pretty new to Hadoop. However, I\'ve been able to successfully setup hadoop 2.7.3 with Java 7 in the cluster mode on my servers. Everything works totally fine.

But

Answer 1

Faced same error while running spark code from IDE.

Exception in thread "main" java.io.IOException: failure to login
        at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:822)
        at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:774)
        at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:647)
        at org.apache.spark.util.Utils$$anonfun$getCurrentUserName$1.apply(Utils.scala:2464)
        at org.apache.spark.util.Utils$$anonfun$getCurrentUserName$1.apply(Utils.scala:2464)
        at scala.Option.getOrElse(Option.scala:121)
        at org.apache.spark.util.Utils$.getCurrentUserName(Utils.scala:2464)
        at org.apache.spark.SparkContext.<init>(SparkContext.scala:292)
        at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2486)
        at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:930)
        at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:921)
        at scala.Option.getOrElse(Option.scala:121)
        at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:921)
        at SimpleApp$.main(SimpleApp.scala:17)
        at SimpleApp.main(SimpleApp.scala)
Caused by: javax.security.auth.login.LoginException:
java.lang.NullPointerException: invalid null input: name

Resolution added following statement in main code.

UserGroupInformation.setLoginUser(UserGroupInformation.createRemoteUser("vyxx"))

Answer 2

I found that I don't have to take any special measures in code if I ensure that the docker image is properly configured for the jenkins user. The code I use to setup the image for the jenkins user for Debian/Ubuntu based images is:

# Add Jenkins user
groupadd --gid 1000 jenkins
useradd --uid 1000 --gid jenkins --shell /bin/bash --home-dir /var/jenkins_home jenkins
mkdir /var/jenkins_home
chown 1000:1000 /var/jenkins_home
echo 'jenkins ALL=NOPASSWD: ALL' >> /etc/sudoers.d/50-jenkins
echo 'Defaults    env_keep += "DEBIAN_FRONTEND"' >> /etc/sudoers.d/env_keep

Answer 3

I've encountered the same issue when running an hbase client from a docker container with Java 8. It is apparently caused by class com.sun.security.auth.module.UnixLoginModule which uses a native call to get the unix username. In my case, it is not mapped in docker, and the class throws a NullPointerException. It is not a bug in hadoop per se.

To instruct hadoop to bypass the lookup of the OS username, I was able to add the following line of code before all initialization:

UserGroupInformation.setLoginUser(UserGroupInformation.createRemoteUser("hduser"));

In your case, you are running the server, so your options of injecting the code are limited. Instead there are two options:

1. Try the IBM JDK instead
2. Try to debug the OS user setup on the workers ( $ whoami). If it says something like 'cannot find name for user ID XXXX', then check the /etc/passwd setup