impala+kerberos配置

矛始

已于 2022-07-28 10:37:09 修改

阅读量1.1w

点赞数 1

分类专栏： impala 文章标签： linux centos 运维

于 2019-05-27 17:44:20 首次发布

本文为博主原创文章，未经博主允许不得转载。

本文链接：https://blog.csdn.net/czmacd/article/details/90607737

版权

impala 专栏收录该内容

3 篇文章 0 订阅

订阅专栏

1.环境

centos 6.5

192.168.17.138 master   NameNode,SecondaryNameNode,impala-catalog,impala-state-tore
192.168.17.153 slave1   DateNode,impala-server
192.168.17.156 slave2   DateNode,impala-server

2.安装Kerberos

master作为KDC Server也作为client端

yum install krb5-server -y
yum -y install krb5-workstation  ##krb5-client

slave1,slave2

yum -y install krb5-workstation  ##krb5-client

slave1,slave2节点安装一下依赖

yum install python-devel openssl-devel python-pip cyrus-sasl cyrus-sasl-gssapi cyrus-sasl-devel -y

否则在启动impala-server 时E1017 03:24:43.111243 10855 authentication.cc:178] SASL message (Kerberos (internal)): No worthy mechs found

3.master 下配置/etc/krb5.conf

logging]
 default = FILE:/var/log/krb5libs.log
 kdc = FILE:/var/log/krb5kdc.log
 admin_server = FILE:/var/log/kadmind.log

[libdefaults]
 default_realm = ABC.COM  --随便写
 dns_lookup_realm = false
 dns_lookup_kdc = false
 ticket_lifetime = 24h
 renew_lifetime = 7d
 forwardable = true

[realms]
 ABC.COM = {
  kdc = master
  admin_server = master
 }

[domain_realm]
.abc.com = ABC.COM
abc.com = GTLAND.COM

同步到slave

scp /etc/krb5.conf slave1:/etc/krb5.conf
scp /etc/krb5.conf slave2:/etc/krb5.conf

4.master下配置/var/kerberos/krb5kdc/kdc.conf

[kdcdefaults]
 kdc_ports = 88
 kdc_tcp_ports = 88

[realms]
 ABC.COM = {
  max_life = 24h
  max_renewable_life = 7d
  default_principal_flags = +renewable, +forwardable  
  #master_key_type = aes256-cts
  acl_file = /var/kerberos/krb5kdc/kadm5.acl
  dict_file = /usr/share/dict/words
  admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab
  supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal
 }

5.创建KDC数据库

/usr/sbin/kdb5_util create -s -r ABC.COM

其中需要设置管理员密码，创建完成会在/var/kerberos/krb5kdc/下面生成一系列文件，若重建数据库则需先删除/var/kerberos/krb5kdc下面principal相关文件

6.给数据库管理员添加ACL权限，修改kadm5.acl文件，*代表全部权限

test/test@ABC.COM	*   //在ABC.COM域内指定包含 admin 实例的所有主体

表达式：principal privileges [principal-target]

修改完成后需要重启kadmin服务：service kadmin start

https://docs.oracle.com/cd/E24847_01/html/819-7061/aadmin-9.html#aadmin-14

7.添加数据库管理员

在master主节点运行kadmin.local，该命令可以避免登录直接管理 kdc数据库

kadmin.local:  addprinc test/test@ABC.COM
WARNING: no policy specified for test/test@ABC.COM; defaulting to no policy
Enter password for principal "test/test@ABC.COM": test
Re-enter password for principal "test/test@ABC.COM": test
Principal "test/test@ABC.COM" created.

8.启动服务

service krb5kdc start
service kadmin start
chkconfig krb5kdc on
chkconfig kadmin on

通过/var/log/krb5kdc.log 和 /var/log/kadmind.log查看日志，通过kinit检查Kerberos正常运行

9.验证客户端可以访问KDC

在slave2使用test/test@ABC.COM主体进行登录

[root@slave2 impala]# kadmin -p 'test/test@ABC.COM' -m master -w test
Authenticating as principal test/test@ABC.COM with password.
kadmin:  listprincs
K/M@ABC.COM
test/test@ABC.COM
impala/master@ABC.COM
impala/slave1@ABC.COM
impala/slave2@ABC.COM
kadmin/admin@ABC.COM
kadmin/changepw@ABC.COM
kadmin/master@ABC.COM
krbtgt/ABC.COM@ABC.COM

测试：配置test/test/@ABC.COM主休只允许添加任何实例且主体名为impala的主体(impala/*@ABC.COM)
修改kadm5.acl添加

test/test@ABC.COM	a	impala/*@ABC.COM

然后重启在master重启kadmin服务

到slave2测试权限

[root@slave2 impala]# kadmin -p 'test/test@ABC.COM' -m master -w test
Authenticating as principal test/test@ABC.COM with password.
kadmin:  listprincs
get_principals: Operation requires ``list'' privilege while retrieving list.     //因为主体中有impala/master@ABC.COM等，所以没权限列出
kadmin:  addprinc impala/test@ABC.COM
WARNING: no policy specified for impala/test@ABC.COM; defaulting to no policy
Enter password for principal "impala/test@ABC.COM": 
Re-enter password for principal "impala/test@ABC.COM": 
Principal "impala/test@ABC.COM" created.             //只有添加权限，成功添加
kadmin:  delprinc impala/test@ABC.COM       
Are you sure you want to delete the principal "impala/test@ABC.COM"? (yes/no): yes
delete_principal: Operation requires ``delete'' privilege while deleting principal "impala/test@ABC.COM"      //无删除权限

回到master使用kadmin.local或者在slave使用其它没被限制权限的主体登录查看所有主体：

K/M@ABC.COM
test/test@ABC.COM
impala/master@ABC.COM
impala/slave1@ABC.COM
impala/slave2@ABC.COM
impala/test@ABC.COM
kadmin/admin@ABC.COM
kadmin/changepw@ABC.COM
kadmin/master@ABC.COM
krbtgt/ABC.COM@ABC.COM

impala/test@ABC.COM成功添加

10.生成keytab

cd /var/kerberos/krb5kdc/

kadmin.local -q "addprinc -randkey impala/master@ABC.COM "
kadmin.local -q "addprinc -randkey impala/slave1@ABC.COM "
kadmin.local -q "addprinc -randkey impala/slave2@ABC.COM "

kadmin.local -q "xst  -k impala.keytab  impala/master@ABC.COM "
kadmin.local -q "xst  -k impala.keytab  impala/slave1@ABC.COM "
kadmin.local -q "xst  -k impala.keytab  impala/slave2@ABC.COM "

复制到各个节点

scp impala.keytab master:/etc/impala/conf
scp impala.keytab slave1:/etc/impala/conf
scp impala.keytab slave2:/etc/impala/conf

并设置权限

ssh master "cd /etc/impala/conf/;chown impala:hadoop *.keytab ;chmod 400 *.keytab"
ssh slave1 "cd /etc/impala/conf/;chown impala:hadoop *.keytab ;chmod 400 *.keytab"
ssh slave2 "cd /etc/impala/conf/;chown impala:hadoop *.keytab ;chmod 400 *.keytab"

否则启动impala服务时会报Could not init kerberos: Runtime error: unable to kinit: unable to login from keytab: Permission denied

11.修改/etc/default/impala

IMPALA_CATALOG_SERVICE_HOST=master
IMPALA_STATE_STORE_HOST=master
IMPALA_STATE_STORE_PORT=24000
IMPALA_BACKEND_PORT=22000
IMPALA_LOG_DIR=/var/log/impala

 hostname=`hostname -f |tr "[:upper:]" "[:lower:]"`

IMPALA_CATALOG_ARGS=" -log_dir=${IMPALA_LOG_DIR} -state_store_host=${IMPALA_STATE_STORE_HOST} \
    -kerberos_reinit_interval=60 \
    -principal=impala/${hostname}@GTLAND.COM \
    -keytab_file=/etc/impala/conf/impala.keytab
"

IMPALA_STATE_STORE_ARGS=" -log_dir=${IMPALA_LOG_DIR} -state_store_port=${IMPALA_STATE_STORE_PORT}\
    -statestore_subscriber_timeout_seconds=15 \
    -kerberos_reinit_interval=60 \
    -principal=impala/${hostname}@ABC.COM \
    -keytab_file=/etc/impala/conf/impala.keytab
"
IMPALA_SERVER_ARGS=" \
    -log_dir=${IMPALA_LOG_DIR} \
    -catalog_service_host=${IMPALA_CATALOG_SERVICE_HOST} \
    -state_store_port=${IMPALA_STATE_STORE_PORT} \
    -use_statestore \
    -state_store_host=${IMPALA_STATE_STORE_HOST} \
    -be_port=${IMPALA_BACKEND_PORT} \
    -kerberos_reinit_interval=60 \
     -principal=impala/${hostname}@ABC.COM \
    -keytab_file=/etc/impala/conf/impala.keytab 
"

ENABLE_CORE_DUMPS=false

同步到各节点

12.启动impala各服务

kinit -k -t /etc/impala/conf/impala.keytab impala/<各主机hostname>@ABC.COM
service impala-catalog start
service impala-state-store start
service impala-server start
impala-shell -k

13.jdbc连接impala

		String hiveDriver="org.apache.hive.jdbc.HiveDriver";
		String hiveUrl="jdbc:hive2://slave1:21050/default;principal=impala/slave1@ABC.COM";
		System.setProperty("java.security.krb5.conf","D:/Program Files/Java/jdk1.8.0_91/jre/lib/security/krb5.conf");//简单点就从服务器上拷贝krb5.conf到javahome
		Configuration conf = new Configuration();
		conf.set("hadoop.security.authentication", "Kerberos");
		UserGroupInformation.setConfiguration(conf);
		//认证的用户和对应的keytab文件
		UserGroupInformation.loginUserFromKeytab("impala/master@ABC.COM","C:/Users/Administrator/Desktop/impala.keytab");  //

		Class.forName(hiveDriver);
		Connection con = DriverManager.getConnection(hiveUrl,"","");
		HiveStatement stmt = (HiveStatement) con.createStatement();
		ResultSet rs =stmt.executeQuery("select * from tb_user");
		while(rs.next()){
			System.out.println(rs.getString(1)+"-"+rs.getString(2)+"-"+rs.getString(3));
		}

HDFS配置Kerberos认证:http://blog.javachen.com/2014/11/04/config-kerberos-in-cdh-hdfs.html
Impala配置Kerberos认证:http://blog.javachen.com/2014/11/06/config-kerberos-in-cdh-impala.html
Kerberos 基本安装与配置:https://blog.csdn.net/post_yuan/article/details/54406148
kerberos认证原理—讲的非常细致，易懂:https://blog.csdn.net/wulantian/article/details/42418231
Kerberos常用命令总结:https://blog.csdn.net/sinat_32176947/article/details/79605448

矛始

关注

1
点赞
踩
7

收藏

觉得还不错? 一键收藏
0
评论
impala+kerberos配置

1.环境centos 6.5192.168.17.138 master NameNode,SecondaryNameNode,impala-catalog,impala-state-tore192.168.17.153 slave1 DateNode,impala-server192.168.17.156 slave2 DateNode,impala-server2.安装K...
复制链接

扫一扫