概况
我们这边用的是key + password 的 kerbose 认证.
在JAVA程序下也遇到一些关于 kerbose 认证绕不过的坑.
认证的代码
//认证返回一个UserGroupInformation,认证成功的User
public static UserGroupInformation login(){
Configuration conf = new Configuration();
conf.set("eagle.security.authentication", "KERBEROS");
conf.set("eagle.security.krb-realm", "XXX.COM");
conf.set("eagle.security.krb-kdc", KDC);
conf.set("eagle.security.authentication-kbr-type", "PASSWORD");
conf.set("eagle.kerberos.password", pwd);
conf.set("eagle.kerberos.principal", username + "@XXX.COM");
conf.setBoolean("hadoop.security.authorization", true);
conf.set("hadoop.security.authentication", "kerberos");
System.setProperty("java.security.krb5.realm",conf.get("eagle.security.krb-realm"));
System.setProperty("java.security.krb5.kdc", conf.get("eagle.security.krb-kdc"));
UserGroupInformation.setConfiguration(conf);
UserGroupInformation.loginUserFromSubject(
authenticateSubject(conf.get("eagle.kerberos.principal"),
conf.get("eagle.kerberos.password"));
return UserGroupInformation.getLoginUser();
}
public static Subject authenticateSubject(String principal, String password) throws LoginException {
logger.debug("Validating password of principal: " + principal);
loginContext = new LoginContext("eagleKerberosLogin", null,
createCallbackHandler(principal, password),
createConfiguration());
loginContext.login();
logger.debug("Principal " + principal + " authenticated succesfully");
return loginContext.getSubject();
}
private static CallbackHandler createCallbackHandler(final String principal, final String password) {
return new CallbackHandler() {
@Override
public void handle(Callback[] callbacks) throws
IOException,UnsupportedCallbackException {
for (Callback callback : callbacks) {
if (callback instanceof NameCallback) {
NameCallback nameCallback = (NameCallback) callback;
nameCallback.setName(principal);
} else if (callback instanceof PasswordCallback) {
PasswordCallback passwordCallback = (PasswordCallback) callback;
passwordCallback.setPassword(password.toCharArray());
} else {
throw new UnsupportedCallbackException(callback,
"Unsupported callback: " + callback.getClass().getCanonicalName());
}
}
}
};
}
private static Configuration createConfiguration() {
return new Configuration() {
@Override
public AppConfigurationEntry[] getAppConfigurationEntry(String name) {
Map<String, Object> options = new HashMap<String, Object>();
options.put("storeKey", "false");
options.put("useTicketCache", "false");
options.put("renewTGT", "false");
AppConfigurationEntry("com.sun.security.auth.module.Krb5LoginModule",
AppConfigurationEntry.LoginModuleControlFlag.REQUIRED, options);
return new AppConfigurationEntry[] { kerberosLMConfiguration };
}
};
}
//获取认证后的jdbc连接
public static Connection getSecurityConnection(String district,String username)
throws IOException, InterruptedException {
final String url = "jdbc:hive2://x.x.x.x:x111x/default";
UserGroupInformation ugi = KerberosAuthenticator.login(username);
return ugi.doAs(new PrivilegedExceptionAction<Connection>() {
@Override
public Connection run() throws Exception {
Class.forName(HiveConnector.HIVE_DRIVER);
Connection connection = DriverManager.getConnection(url);
return connection;
}
});
}
//http 操作webhdfs
public synchronized String getResponse(final String username, final String spec,
final String method,final boolean followRedirects, final OutputStream out)
throws IOException, AuthenticationException, InterruptedException {
UserGroupInformation ugi = KerberosAuthenticator.login(username);
return ugi.doAs(new PrivilegedExceptionAction<String>() {
@Override
public String run() throws Exception {
BASE64Encoder encoder = new BASE64Encoder();
URL httpfsUrl = "请求链接";
URL reqURL = new URL(httpfsUrl, spec);
HttpCallerInfo httpCallerInfo = new HttpCallerInfo(reqURL);
HttpURLConnection conn =
(HttpURLConnection)reqURL.openConnection();
try {
conn.setInstanceFollowRedirects(followRedirects);
conn.setRequestMethod(method);
if (out == null) {
conn.connect();
checkError(conn);
} else {
conn.setRequestProperty("Content-Type",
"application/octet-stream");
conn.connect();
checkError(conn);
InputStream is = conn.getInputStream();
copy(is, out);
}
return result(conn);
} catch (StandbyException e) {
} finally {
conn.disconnect();
}
}
throw new StandbyException("The operation is not supported in state standby");
}
});
}
遇到的问题
按照上述的代码,其他也是如此类推,就可以认证成功了.在请求的时候,不需要附带任何信息,只需要在UserGroupInformation的doAs函数下执行,正常的执行即可,在本地环境是可以直接执行成功.但是,在发到线上后,却发现无法了以下的一些问题.
问题点一 : 线上的Kerbose认证失败,线下,本地代码跑的Kerbose可以认证成功.
解决 : 这是由于线上的认证的机器有个一个krb5.conf的文件,每次Kerbose认证都会默认的去读这个配置,如果该配置出现问题,这个认证就会失败.即使代码写死不读这个配置,代码用其他配置覆盖,最后还是用了krb5.conf的配置,但是代码的配置也不能出错,出错也一定认证失败.
线上为Centos环境,线下为Windows环境,线上环境却不出这个原因的,这个还有待去查证.问题点二 : 认证成功后,HiveConnection,可以正常使用.但是,HTTP方式的WebHdfs的请求却报错误.报没有认证用户.同样的,同样的代码,在本地环境跑通了,线上环境跑不通.
解决: 通过debug本地环境的整个HTTP请求发现,在认证成功后,的doAs函数中,http请求到namenode上面的时候会先报一个401的权限错误,然后HttpURLConnection 的请求链路,检测到返回的请求是401的时候,会重新再发一次HTTP请求,而这次请求是会带上一些权限上面的信息.具体的详细流程,这里不说.只贴上权限对应的代码.//在上面的getResponse方法中加上,以下的代码 Negotiator negotiator = getNegotiator(httpCallerInfo); byte[] tokens = negotiator.firstToken(); String tokenCode = encoder.encode(tokens); tokenCode = tokenCode.trim(); tokenCode = tokenCode.replaceAll("\r|\n",""); conn.setAuthenticationProperty("Authorization","Negotiate " + tokenCode); static Negotiator getNegotiator(HttpCallerInfo var0) { Constructor var2; try { Class var1 = Class.forName("sun.net.www.protocol.http.spnego.NegotiatorImpl", true,(ClassLoader)null); var2 = var1.getConstructor(new Class[]{HttpCallerInfo.class}); } catch (ClassNotFoundException var5) { return null; } catch (ReflectiveOperationException var6) { throw new AssertionError(var6); } try { return (Negotiator)((Negotiator)var2.newInstance(new Object[]{var0})); } catch (ReflectiveOperationException var7) { Throwable var4 = var7.getCause(); return null; } }
- 而线上的HTTP请求,在访问的时候,却没有这个第二次加上权限的请求步骤,所以导致虽然认证成功了,但是HTTP申请WebHdfs的时候,却提示401权限错误.
最后是通过这种直接添加权限信息的方式,将权限信息带过去.解决第二个问题点
至于线上环境为什么不会第二次请求,这个还有待验证.估计是线上环境的jdk的版本问题.