impala加kerberos后权限问题

impala加kerberos后权限问题

背景

公司测试集群需要配置impala+kerberos，但是测试集群很乱，很多人用，用户还有权限比较混乱，而且是ambari HDP的集群。加了kerberos后查询没有问题，建表的时候不行。impala在124也就是master上，程序在126机slave2上，hive在125上。

问题

Unable to obtain password from user

2022-09-27 11:38:40.225 [main] WARN  org.apache.hadoop.util.NativeCodeLoader.<clinit>:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2022-09-27 11:38:40.313 [main] ERROR com.asiainfo.dacp.kpi.process.kpiscope.impl.KpiScopeCal4Db.createTempWideTable:398 - 创建临时宽表失败：Login failure for impala/slave2.am.com@AM.COM from keytab /root/impala-http.keytab: javax.security.auth.login.LoginException: Unable to obtain password from user

java.io.IOException: Login failure for impala/slave2.am.com@AM.COM from keytab /root/impala-http.keytab: javax.security.auth.login.LoginException: Unable to obtain password from user

解决

1.修改/root/impala-http.keytab的权限

chown impala：impala /root/impala-http.keytab

2.检查/etc/default/impala配置
具体配置如下：

IMPALA_CATALOG_SERVICE_HOST=10.1.x.x
IMPALA_STATE_STORE_HOST=10.1.x.x
IMPALA_STATE_STORE_PORT=24000
IMPALA_BACKEND_PORT=22000
IMPALA_LOG_DIR=/var/log/impala

IMPALA_CATALOG_ARGS=" -log_dir=${IMPALA_LOG_DIR}  -state_store_host=${IMPALA_STATE_STORE_HOST}  -kerberos_reinit_interval=60 -principal=impala/_HOST@AM.COM -keytab_file=/etc/impala/conf/impala-http.keytab"
IMPALA_STATE_STORE_ARGS=" -log_dir=${IMPALA_LOG_DIR} -state_store_port=${IMPALA_STATE_STORE_PORT} -kerberos_reinit_interval=60 -principal=impala/_HOST@AM.COM -keytab_file=/etc/impala/conf/impala-http.keytab"
IMPALA_SERVER_ARGS=" \
    -log_dir=${IMPALA_LOG_DIR} \
    -catalog_service_host=${IMPALA_CATALOG_SERVICE_HOST} \
    -state_store_port=${IMPALA_STATE_STORE_PORT} \
    -use_statestore=true \
    -state_store_host=${IMPALA_STATE_STORE_HOST} \
    -be_port=${IMPALA_BACKEND_PORT} \
    -kudu_master_hosts=10.1.251.124:7051 \
    -kerberos_reinit_interval=60 \
    -principal=impala/_HOST@AM.COM \
    -keytab_file=/etc/impala/conf/impala-http.keytab"

ENABLE_CORE_DUMPS=false

# LIBHDFS_OPTS=-Djava.library.path=/usr/lib/impala/lib
# MYSQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-java.jar
# IMPALA_BIN=/usr/lib/impala/sbin
# IMPALA_HOME=/usr/lib/impala
# HIVE_HOME=/usr/lib/hive
# HBASE_HOME=/usr/lib/hbase
# IMPALA_CONF_DIR=/etc/impala/conf
# HADOOP_CONF_DIR=/etc/impala/conf

重启程序报以下错误：

Error Code: 0, SQL state: TStatus(statusCode:ERROR_STATUS, sqlState:HY000, errorMessage:

2022-09-27 11:42:49.364 [main] ERROR com.asiainfo.dacp.kpi.process.kpiscope.impl.KpiScopeCal4Db.createTempWideTable:398 - 创建临时宽表失败：[Cloudera][ImpalaJDBCDriver](500051) ERROR processing query/statement. Error Code: 0, SQL state: TStatus(statusCode:ERROR_STATUS, sqlState:HY000, errorMessage:
ImpalaRuntimeException: Error making 'createTable' RPC to Hive Metastore: 
CAUSED BY: MetaException: Got exception: org.apache.hadoop.ipc.RemoteException Unauthorized connection for super-user: hive/master.am.com@AM.COM from IP 10.1.251.124
), Query: create  table testkpi.mj_test0826_zb_20220101_bfXx as select 20220101 as op_time,area_code_lv3,channel,sum(kpi_val) as DCD012L00021 from ( select AREA_ID_LV3 as area_code_lv3,CHANNEL_CLASS_CODE as CHANNEL,kpi_val from KPI_TEST where DATE_CD = '20211001' ) scope_sql group by area_code_lv3,channel.
java.sql.SQLException: [Cloudera][ImpalaJDBCDriver](500051) ERROR processing query/statement. Error Code: 0, SQL state: TStatus(statusCode:ERROR_STATUS, sqlState:HY000, errorMessage:
ImpalaRuntimeException: Error making 'createTable' RPC to Hive Metastore: 
CAUSED BY: MetaException: Got exception: org.apache.hadoop.ipc.RemoteException Unauthorized connection for super-user: hive/master.am.com@AM.COM from IP 10.1.251.124

解决

原因：hadoop配置文件 core-site.xml错误， 用于连接的IP地址或主机名没有增加到代理配置中
修改hdfs配置
hadoop.proxyuser.hive.hosts 原来是slave1为了方便我直接用*了。

重启服务，重新执行程序任务正常

执行impala-shell -k

创建测试表没有问题

impala-shell -k
Starting Impala Shell using Kerberos authentication
Using service name 'impala'
Connected to master.am.com:21000
Server version: impalad version 2.7.0-IMPALA_KUDU-cdh5 RELEASE (build 10d4ebec3c23961218e972e74e9d342ffc417af1)
***********************************************************************************
Welcome to the Impala shell. Copyright (c) 2015 Cloudera, Inc. All rights reserved.
(Impala Shell v2.6.0-cdh5.8.0 (8d8652f) built on Tue Jul 12 15:43:17 PDT 2016)

To see a summary of a query's progress that updates in real-time, run 'set
LIVE_PROGRESS=1;'.
***********************************************************************************
[master.am.com:21000] > create table jzy(id int);
Query: create table jzy(id int)

Fetched 0 row(s) in 0.07s