CDH集群启用Sentry Kerberos LDAP

参考链接: Cloudera Security | 5.8.x | Cloudera Documentationhttps://www.cloudera.com/documentation/enterprise/5-8-x/topics/security.html

一、前置准备

1、基础环境说明

操作系统：CentOS 6.8 minimal

CDM版本: 5.12.1

CDH版本：5.12.1

MySQL版本: 5.1.73

JDK: 1.8.0_131

浏览器版本: ChromeStandalone_56以上、IE10

内存:32G以上

  CPU :8core

  网络:千兆以上

集群未启用Kerberos

2、CDH安装

参考《CDH5.14.0集群安装》

二、KDC服务安装及配置

1、安装KDC服务

# yum install krb5-server krb5-libs krb5-auth-dialog krb5-workstation

2、配置KDC服务

1). 修改/etc/krb5.conf

# vi /etc/krb5.conf

[logging]

 default = FILE:/var/log/krb5libs.log

 kdc = FILE:/var/log/krb5kdc.log

 admin_server = FILE:/var/log/kadmind.log

[libdefaults]

 default_realm = HADOOP.COM

 dns_lookup_realm = false

 dns_lookup_kdc = false

 ticket_lifetime = 24h

 renew_lifetime = 7d

 forwardable = true

[realms]

 HADOOP.COM = {

  kdc = whgk02.domain

  admin_server = whgk02.domain

 }

[domain_realm]

 . whgk02.domain = HADOOP.COM

 whgk02.domain = HADOOP.COM

2). 修改/var/kerberos/krb5kdc/kadm5.acl

# vi /var/kerberos/krb5kdc/kadm5.acl

*/admin@HADOOP.COM      *

3). 修改/var/kerberos/krb5kdc/kdc.conf

# vi /var/kerberos/krb5kdc/kdc.conf

[kdcdefaults]

 kdc_ports = 88

 kdc_tcp_ports = 88

[realms]

 HADOOP.COM = {

  #master_key_type = aes256-cts

  max_renewable_life= 7d 0h 0m 0s

  acl_file = /var/kerberos/krb5kdc/kadm5.acl

  dict_file = /usr/share/dict/words

  admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab

  supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal

 }

4). 创建Kerberos数据库

# kdb5_util create –r HADOOP.COM -s

Loading random data

Initializing database '/var/kerberos/krb5kdc/principal' for realm HADOOP.COM',

master key name 'K/M@WHGK.COM'

You will be prompted for the database Master Password.

It is important that you NOT FORGET this password.

Enter KDC database master key:设置密码

Re-enter KDC database master key to verify:确认密码

5). 创建Kerberos的管理账号

# kadmin.local

Authenticating as principal root/admin@HADOOP.COM with password.

kadmin.local:  add  （可以按tab补全命令）

add_policy     add_principal  addpol         addprinc      

kadmin.local:  addprinc admin/admin@HADOOP.COM

WARNING: no policy specified for admin/admin@HADOOP.COM; defaulting to no policy

Enter password for principal "admin/admin@HADOOP.COM": 设置密码

Re-enter password for principal "admin/admin@HADOOP.COM": 确认密码

Principal "admin/admin@HADOOP.COM" created.

kadmin.local:  exit

6). 添加自启动服务并启动服务

# systemctl enable --now krb5kdc.service

# systemctl enable --now kadmin.service

7). 测试Kerberos管理员账号

# kinit admin/admin@HADOOP.COM

Password for admin/admin@HADOOP.COM:

# klist

# klist -e

3、集群所有节点安装Kerberos客户端(包括CM)

# yum -y install krb5-libs krb5-workstation

CM节点安装额外组件

# yum -y install openldap-clients

4、拷贝配置文件

将KDC Server上的krb5.conf文件拷贝到所有Kerberos客户端(即集群所有节点)

# scp /etc/krb5.conf boe02:/etc/

# scp /etc/krb5.conf boe03:/etc/

三、CDH集群启用Kerberos

1、配置集群JDK(如未安装JDK参考install_java.sh)

下载jce_policy-8.zip

链接: http://www.oracle.com/technetwork/java/javase/downloads/index.html

# 7za x jce_policy-8.zip

# cp UnlimitedJCEPolicyJDK8/*.jar /usr/jdk64/jdk1.8.0_131/jre/lib/security/

登陆CM：

主页搜索java_home：

设置java_home：

2、KDC添加Cloudera Manager管理员账号

# kadmin.local

Authenticating as principal admin/admin@HADOOP.COM with password.

kadmin.local:  addprinc cloudera-scm/admin@HADOOP.COM

WARNING: no policy specified for cloudera-scm/HADOOP @WHGK.COM; defaulting to no policy

Enter password for principal "cloudera-scm/admin@ HADOOP.COM":

Re-enter password for principal "cloudera-scm/admin@HADOOP.COM":

Principal "cloudera-scm/admin@HADOOP.COM" created.

kadmin.local:  exit

3、配置启用kerberos

1). 启用Kerberos

进入Cloudera Manager的“管理”-> “安全”界面->启用Kerberos

2).检查信息

确保如下列出的所有检查项都已完成

3).配置KDC信息

配置KDC类型、KDC服务器、KDC Realm、加密类型以及待创建的Service Principal（hdfs，yarn,，hbase，hive等）的更新生命期等，点击“继续”

4).KRB5配置

不建议让Cloudera Manager来管理krb5.conf，点击“继续”

5).输入CM的Kerbers管理员账号

必须和之前创建的账号一致，点击“继续”，等待启用Kerberos完成，点击“继续”

6).Kerberos主体

7).重启集群

重启成功：

至此已成功启用Kerberos。

四、CDH集群启用sentry

1、创建数据库

mysql> create database sentry character set utf8;

mysql> grant all privileges on sentry.* to sentry@'cdh01.domain' identified by '123456';

mysql> flush privileges;

2、添加sentry服务

3、配置组件启用sentry服务

将sentry 用户在linux系统上加入hive用户组

# usermod -a -G hive sentry

# cat /etc/group|grep hive

1).HDFS配置启用sentry

启用ACLS和Sentry权限同步

启用 HDFS 权限检查

2).HIVE配置启用sentry

启用sentry

禁用Hive Impersonation

启用数据库中的存储通知

3).IMPALA配置启用sentry

4).HUE配置启用sentry

重启过时服务

五、LDAP服务安装及配置

1、安装OpenLDAP服务

# yum -y install openldap compat-openldap openldap-clients openldap-servers openldap-servers-sql openldap-devel migrationtools

# rpm -qa |grep openldap

2、配置OpenLDAP

1).配置管理员密码

# slappasswd -s UnionBigData@123.

{SSHA}7hy17cr5d81JmKtj6JEXkCI36nXLwLN6

2).配置slapd.ldif文件

配置文件中多处配置dc=whgk02,dc=domain，由于OpenLDAP的域为whgk02.domain，如果LDAP的域为ldap.whgk02.com则配置为dc=ldap,dc=whgk02,dc=domain，根据自己LDAP的域名进行相应的修改。

# cp /usr/share/openldap-servers/slapd.ldif ~/

# vim slapd.ldif

dn: cn=config

objectClass: olcGlobal

cn: config

olcArgsFile: /var/run/openldap/slapd.args

olcPidFile: /var/run/openldap/slapd.pid

dn: cn=schema,cn=config

objectClass: olcSchemaConfig

cn: schema

include: file:///etc/openldap/schema/corba.ldif

include: file:///etc/openldap/schema/core.ldif

include: file:///etc/openldap/schema/cosine.ldif

include: file:///etc/openldap/schema/duaconf.ldif

include: file:///etc/openldap/schema/dyngroup.ldif

include: file:///etc/openldap/schema/inetorgperson.ldif

include: file:///etc/openldap/schema/java.ldif

include: file:///etc/openldap/schema/misc.ldif

include: file:///etc/openldap/schema/nis.ldif

include: file:///etc/openldap/schema/openldap.ldif

include: file:///etc/openldap/schema/ppolicy.ldif

include: file:///etc/openldap/schema/collective.ldif

dn: olcDatabase=frontend,cn=config

objectClass: olcDatabaseConfig

objectClass: olcFrontendConfig

olcDatabase: frontend

dn: olcDatabase=config,cn=config

objectClass: olcDatabaseConfig

olcDatabase: config

olcAccess: to * by dn.base="gidNumber=0+uidNumber=0,cn=peercred,cn=external,c

 n=auth" manage by * none

dn: olcDatabase=monitor,cn=config

objectClass: olcDatabaseConfig

olcDatabase: monitor

olcAccess: to * by dn.base="gidNumber=0+uidNumber=0,cn=peercred,cn=external,c

 n=auth" read by dn.base="cn=Manager,dc=whgk,dc=com" read by * none

dn: olcDatabase=hdb,cn=config

objectClass: olcDatabaseConfig

objectClass: olcHdbConfig

olcDatabase: hdb

olcSuffix: dc=whgk,dc=com

olcRootDN: cn=Manager,dc=whgk,dc=com

olcRootPW: {SSHA}7hy17cr5d81JmKtj6JEXkCI36nXLwLN6

olcDbDirectory: /var/lib/ldap

olcDbIndex: objectClass eq,pres

olcDbIndex: ou,cn,mail,surname,givenname eq,pres,sub

重新生成OpenLDAP的配置

# rm -rf /etc/openldap/slapd.d/*

# slapadd -F /etc/openldap/slapd.d -n 0 -l /root/slapd.ldif

# slaptest -u -F /etc/openldap/slapd.d

# chown -R ldap. /etc/openldap/slapd.d/

3).安装OpenLDAP的数据库文件并启动

# cp /usr/share/openldap-servers/DB_CONFIG.example /var/lib/ldap/DB_CONFIG

# chown -R ldap. /var/lib/ldap/

# systemctl enable --now slapd

# systemctl status slapd

3、导入根域及管理员账号

1). 创建root.ldif文件

# vim root.ldif

dn: dc=whgk,dc=com

dc: whgk

objectClass: top

objectClass: domain

dn: cn=Manager,dc=whgk,dc=com

objectClass: organizationalRole

cn: Manager

2).导入根域及管理员

# ldapadd -D "cn=Manager,dc=whgk,dc=com" -W -x -f root.ldif

查看是否导入成功

# ldapsearch -h whgk02.domain -b "dc=whgk,dc=com" -D "cn=Manager,dc=whgk,dc=com" –W

4、导入基础文件、账号和用户组

1). 配置migrate_common.ph

将文件中的$DEFAULT_MAIL_DOMAIN和$DEFAULT_BASE修改为自己OpenLDAP的域

# vim /usr/share/migrationtools/migrate_common.ph

2). 导出基础文件

导出基础文件并保留需要的基础域配置

# /usr/share/migrationtools/migrate_base.pl >base.ldif

3).导出操作系统group.ldif文件

导出group文件并保留需要的group配置

# /usr/share/migrationtools/migrate_group.pl /etc/group > group.ldif

4).导出操作系统user.ldif文件

导出user文件并保留需要的user配置

# /usr/share/migrationtools/migrate_passwd.pl /etc/passwd >user.ldif

5).导入文件

# ldapadd -D "cn=Manager,dc=whgk,dc=com" -W -x -f base.ldif

# ldapadd -D "cn=Manager,dc=whgk,dc=com" -W -x -f group.ldif

# ldapadd -D "cn=Manager,dc=whgk,dc=com" -W -x -f user.ldif

查看是否导入成功

# ldapsearch -h whgk02.domain -b "dc=whgk,dc=com" -D "cn=Manager,dc=whgk,dc=com" -W|grep dn

5、OpenLDAP客户端配置

1).安装OpenLDAP客户端

# yum -y install openldap-clients

2).配置OpenLDAP客户端

# vim /etc/openldap/ldap.conf

打开日志

新增一行配置

# vim /etc/rsyslog.conf

local4.*                                                /var/log/ldap.log

# systemctl restart rsyslog.service

# tailf /var/log/ldap.log

3).测试客户端

# ldapsearch -D "cn=Manager,dc=whgk,dc=com" -W |grep dn

六、CDH集群启用LDAP

1、HDFS启用LADP

2、Hive启用LDAP

3、Impala启用LDAP

4、HUE启用LDAP