Will the hbase kerberos token expired

问题

I have a spark streaming application, for every batch, I need to insert it to the hbase which is protected by kerberos. I found a solution, that is in the driver side I create a connection and obtain a token from that conn and then pass it to the executor. In the executor side, I decode it and get the token, in this way I can insert data to hbase successfully. This seems good, but my concern is that will the token expired? If so , how to solve it please?

My code snippet is

val ugi=UserGroupInformation.loginUserFromKeytabAndReturnUGI(principle,keytabfile);
ugi.doAs(new PrivilegedAction[Unit]() {
  def run(): Unit = {
    conn = ConnectionFactory.createConnection(conf)
    val token = TokenUtil.obtainToken(conn)
    tokenStr = token.encodeToUrlString()
  }
})

in the rdd.foreachpartition,

val token = new Token()
token.decodeFromUrlString(tokenStr)
UserGroupInformation.getCurrentUser.addToken(token)

Although I have searched a lot from Internet about this issue, but I did not found a good solution about this issue. The common answer to this question is

UserGroupInformation.getLoginUser().checkTGTAndReloginFromKeytab();

But as my test, inside this method,

public synchronized void checkTGTAndReloginFromKeytab() throws IOException {
if (!isSecurityEnabled()
    || user.getAuthenticationMethod() != AuthenticationMethod.KERBEROS
    || !isKeytab)
  return;
KerberosTicket tgt = getTGT();
if (tgt != null && Time.now() < getRefreshTime(tgt)) {
  return;
}
reloginFromKeytab();

}

The isKeytab is always false, so it will never execute the following code,I do not understand why this return false. So anybody can help me solve this question? Any help is appreciated!

回答1:

It is caused by the java version. If you want to run a secured Hadoop cluster on JDK 1.7.0_85 or later, then you must run Apache Hadoop 2.7.0 or later.

To see this Jira issue HADOOP-10786

来源：https://stackoverflow.com/questions/44815135/will-the-hbase-kerberos-token-expired

标签

apache-spark

hbase

kerberos