Kerboers认证由于和Master时间不同步导致的错误

刚发现Hbase集群一台RegionServer挂掉了，重启之后仍然不正常，日志中的错误：

Caused by: java.io.IOException: Login failure for hbase/hd0159-sw36.dc.sh-wgq.sdo.com@DC.SH-WGQ.SDO.COM from keytab /home/hdfs/hbase-current/conf/hbase.keytab
        at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:638)
        at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:234)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.hbase.security.User.call(User.java:394)
        at org.apache.hadoop.hbase.security.User.access$600(User.java:54)
        at org.apache.hadoop.hbase.security.User$SecureHadoopUser.login(User.java:359)
        at org.apache.hadoop.hbase.security.User.login(User.java:141)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:352)
        ... 10 more
Caused by: javax.security.auth.login.LoginException: Clock skew too great (37) - PREAUTH_FAILED
        at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:696)
        at com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:542)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at javax.security.auth.login.LoginContext.invoke(LoginContext.java:769)
        at javax.security.auth.login.LoginContext.access$000(LoginContext.java:186)
        at javax.security.auth.login.LoginContext$4.run(LoginContext.java:683)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
        at javax.security.auth.login.LoginContext.login(LoginContext.java:579)
        at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:629)
        ... 20 more

 

主要是这句：

Caused by: javax.security.auth.login.LoginException: Clock skew too great

其实是这台RegionServer的时间和Kerboers服务器的时间偏差太大（大于5分钟），因此kerboers认证失败。

 

于是将这台RegionServer上的时间做了同步，从世界标准时间中心获取时间：

 

ntpdate 0.pool.ntp.org

 

之后重启RegionServer，恢复正常。