CDH6.3.1集成ranger2.1，并且接管HDFS、Yarn、Hive组件详细步骤。亲测实现！！！

最新推荐文章于 2024-10-02 00:05:33 发布
sunxunyong
最新推荐文章于 2024-10-02 00:05:33 发布
阅读量290
点赞数 1
文章标签： hdfs hive hadoop
本文链接：https://blog.csdn.net/sunxunyong/article/details/142661172
版权
###############################ranger编译安装##########################################
1、ranger解压编译好的包
mkdir -p /opt/ranger
tar -zxvf ranger-2.1.0-admin.tar.gz -C /data01/ranger/
ln -s /data01/sunxy/ranger/ranger-2.1.0-admin /opt/ranger/rangeradmin
ln -s /data01/sunxy/ranger/ranger-2.1.0-hdfs-plugin /opt/ranger/rangerhdfs
ln -s /data01/sunxy/ranger/ranger-2.1.0-hive-plugin /opt/ranger/rangerhive
ln -s /data01/sunxy/ranger/ranger-2.1.0-usersync /opt/ranger/rangerusersync
ln -s /data01/sunxy/ranger/ranger-2.1.0-yarn-plugin /opt/ranger/rangeryarn


2、初始化数据库
集群信息：http://1XXXXXXX.38:7180/cmf/login   amdin admin
数据库：mysql -h1XXXXXXX.39 -uroot  -pTestCDH2024
#创建数据库:
create database ranger CHARACTER SET utf8 COLLATE utf8_general_ci; 
#创建账户
create user 'root'@'%' identified by 'TestCDH2024';
create user 'ranger'@'%' identified by 'TestCDH2024';
#赋予权限
grant all privileges on *.* to 'root'@'%' with grant option;
grant all privileges on *.* to 'ranger'@'%' with grant option;
grant proxy on ''@'' to 'root'@'%' with grant option;
grant proxy on ''@'' to 'ranger'@'%' with grant option;

本地无权限时，单独赋权到ip！！！
grant all privileges on *.* to 'root'@'1XXXXXXX.38' with grant option;
grant proxy on ''@'' to 'root'@'1XXXXXXX.38' with grant option;
grant all privileges on *.* to 'root'@'1XXXXXXX.39' with grant option;
grant proxy on ''@'' to 'root'@'1XXXXXXX.39' with grant option;
grant all privileges on *.* to 'root'@'1XXXXXXX.40' with grant option;
grant proxy on ''@'' to 'root'@'1XXXXXXX.40' with grant option;
grant all privileges on *.* to 'root'@'IT-ES-Node01' with grant option;
grant proxy on ''@'' to 'root'@'IT-ES-Node01' with grant option;
flush privileges;

ALTER USER 'root'@'1XXXXXXX.38' IDENTIFIED WITH mysql_native_password BY 'TestCDH2024';
ALTER USER 'root'@'1XXXXXXX.39' IDENTIFIED WITH mysql_native_password BY 'TestCDH2024';
ALTER USER 'root'@'1XXXXXXX.40' IDENTIFIED WITH mysql_native_password BY 'TestCDH2024';
ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY 'TestCDH2024';

flush privileges;  



#刷新
flush privileges;
回看权限
select host,user,Grant_priv,Super_priv from mysql.user;


3、修改rangeradmin配置
修改ranger集成CDH的solr
修改vi /opt/ranger/rangeradmin/contrib/solr_for_audit_setup/install.properties
SOLR_USER=root
SOLR_GROUP=root
MAX_AUDIT_RETENTION_DAYS=90
SOLR_INSTALL=false
SOLR_DOWNLOAD_URL=
SOLR_INSTALL_FOLDER=/opt/cloudera/parcels/CDH/lib/solr
SOLR_RANGER_HOME=/opt/cloudera/parcels/CDH/lib/solr/ranger_audit_server
SOLR_RANGER_PORT=8983
SOLR_DEPLOYMENT=solrcloud
SOLR_RANGER_DATA_FOLDER=/opt/cloudera/parcels/CDH/lib/solr/ranger_audit_server/data
SOLR_ZK=1XXXXXXX.38:2181,1XXXXXXX.39:2181,1XXXXXXX.40:2181/solr/configs/ranger_audits
SOLR_HOST_URL=http://`hostname -f`:8983
SOLR_SHARDS=3
SOLR_REPLICATION=2
SOLR_LOG_FOLDER=/opt/logs/solr/ranger_audits
SOLR_RANGER_COLLECTION=ranger_audits
SOLR_MAX_MEM=2g

修改vi /opt/ranger/rangeradmin/contrib/solr_for_audit_setup/conf/solrconfig.xml
1- EEE MMM ppd HH:mm:ss [z ]yyyy 修改为 EEE MMM dd HH:mm:ss [z ]yyyy
2- <lib dir="${solr.install.dir:../../../..}/dist/" regex="solr-dataimporthandler-.*\.jar" /> 改为cdh上solr jar对应的目录： <lib dir="/opt/cloudera/parcels/CDH/lib/solr/" regex="solr-dataimporthandler-.*\.jar" />

CDH页面修改用于存储该Solr服务相关信息的ZooKeeper znode。
由/solr 改为 /ranger_audits

修改ranger-admin的install.propertities
vi /opt/ranger/rangeradmin/install.properties
PYTHON_COMMAND_INVOKER=python
DB_FLAVOR=MYSQL
SQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-javaold.jar
db_root_user=root
db_root_password=TestCDH2024
db_host=1XXXXXXX.39
db_ssl_enabled=false
db_ssl_required=false
db_ssl_verifyServerCertificate=false
db_ssl_auth_type=2-way
javax_net_ssl_keyStore=
javax_net_ssl_keyStorePassword=
javax_net_ssl_trustStore=
javax_net_ssl_trustStorePassword=
db_name=ranger
db_user=root
db_password=TestCDH2024
rangerAdmin_password=TestCDH2024
rangerTagsync_password=TestCDH2024
rangerUsersync_password=TestCDH2024
keyadmin_password=TestCDH2024
audit_store=solr
audit_elasticsearch_urls=
audit_elasticsearch_port=
audit_elasticsearch_protocol=
audit_elasticsearch_user=
audit_elasticsearch_password=
audit_elasticsearch_index=
audit_elasticsearch_bootstrap_enabled=false
audit_solr_urls=http://1XXXXXXX.38:8983/solr/ranger_audits
audit_solr_user=
audit_solr_password=
audit_solr_zookeepers=1XXXXXXX.38:2181,1XXXXXXX.39:2181,1XXXXXXX.40:2181/ranger_audits
audit_solr_collection_name=ranger_audits
audit_solr_config_name=ranger_audits
audit_solr_no_shards=
audit_solr_no_replica=
audit_solr_max_shards_per_node=
audit_solr_acl_user_list_sasl=solr,infra-solr
audit_solr_bootstrap_enabled=true
policymgr_external_url=http://1XXXXXXX.38:6080
policymgr_http_enabled=true
policymgr_https_keystore_file=
policymgr_https_keystore_keyalias=rangeradmin
policymgr_https_keystore_password=
policymgr_supportedcomponents=
unix_user=sunxy
unix_user_pwd=TestCDH2024
unix_group=sunxy
authentication_method=NONE
remoteLoginEnabled=true
authServiceHostName=1XXXXXXX.38
authServicePort=5151
ranger_unixauth_keystore=keystore.jks
ranger_unixauth_keystore_password=password
ranger_unixauth_truststore=cacerts
ranger_unixauth_truststore_password=changeit
xa_ldap_url=
xa_ldap_userDNpattern=
xa_ldap_groupSearchBase=
xa_ldap_groupSearchFilter=
xa_ldap_groupRoleAttribute=
xa_ldap_base_dn=
xa_ldap_bind_dn=
xa_ldap_bind_password=
xa_ldap_referral=
xa_ldap_userSearchFilter=
xa_ldap_ad_domain=
xa_ldap_ad_url=
xa_ldap_ad_base_dn=
xa_ldap_ad_bind_dn=
xa_ldap_ad_bind_password=
xa_ldap_ad_referral=
xa_ldap_ad_userSearchFilter=
spnego_principal=
spnego_keytab=
token_valid=30
cookie_domain=
cookie_path=/
admin_principal=
admin_keytab=
lookup_principal=
lookup_keytab=
hadoop_conf=/etc/hadoop/conf
sso_enabled=false
sso_providerurl=https://127.0.0.1:8443/gateway/knoxsso/api/v1/websso
sso_publickey=
RANGER_ADMIN_LOG_DIR=$PWD
RANGER_PID_DIR_PATH=/var/run/ranger
XAPOLICYMGR_DIR=$PWD
app_home=$PWD/ews/webapp
TMPFILE=$PWD/.fi_tmp
LOGFILE=$PWD/logfile
LOGFILES="$LOGFILE"
JAVA_BIN='java'
JAVA_VERSION_REQUIRED='1.8'
JAVA_ORACLE='Java(TM) SE Runtime Environment'
ranger_admin_max_heap_size=1g
PATCH_RETRY_INTERVAL=120
STALE_PATCH_ENTRY_HOLD_TIME=10
mysql_core_file=db/mysql/optimized/current/ranger_core_db_mysql.sql
mysql_audit_file=db/mysql/xa_audit_db.sql
oracle_core_file=db/oracle/optimized/current/ranger_core_db_oracle.sql
oracle_audit_file=db/oracle/xa_audit_db_oracle.sql
postgres_core_file=db/postgres/optimized/current/ranger_core_db_postgres.sql
postgres_audit_file=db/postgres/xa_audit_db_postgres.sql
sqlserver_core_file=db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql
sqlserver_audit_file=db/sqlserver/xa_audit_db_sqlserver.sql
sqlanywhere_core_file=db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql
sqlanywhere_audit_file=db/sqlanywhere/xa_audit_db_sqlanywhere.sql
cred_keystore_filename=$app_home/WEB-INF/classes/conf/.jceks/rangeradmin.jceks

4、初始化solr
cd /opt/ranger/rangeradmin/contrib/solr_for_audit_setup
./setup.sh

5、初始化admin
cd /opt/ranger/rangeradmin/
./setup.sh

1、报错：com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Cannot delete or update a parent row: a foreign key constraint fails
SET GLOBAL FOREIGN_KEY_CHECKS = 0; 
2、报错：This function has none of DETERMINISTIC, NO SQL, or READS SQL DATA in its declaration and binary logging is enabled (you *might* want to use the less safe log_bin_trust_function_creators variable)
SET GLOBAL log_bin_trust_function_creators = 1;

SQLException : SQL state: 42000 com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Specified key was too long; max key length is 767 bytes ErrorCode: 1071
SET GLOBAL innodb_large_prefix = ON;
SET GLOBAL innodb_file_format=Barracuda;

5、启动ranger-admin
ranger-admin start

页面访问：
http://1XXXXXXX.38:6080  admin:TestCDH2024
返回204
因为编译过程中有jar包冲突，只需删掉/opt/ranger/rangeradmin/ews/webapp/WEB-INF/lib下的相关jar包
rm -f javax.ws.rs-api-2.1.jar  jersey-client-2.6.jar jersey-server-2.27.jar
ranger-admin restart

6、上传相关配置到zookeeper并创建对应collection
查看创建的solrhome的实例 
solrctl instancedir --list 
创建 collection1 实例并将配置文件上传到 zookeeper: 
#conf必须是一个目录，这里的目录是 /opt/ranger/rangeradmin/contrib/solr_for_audit_setup/conf
solrctl instancedir --create ranger_audits /opt/ranger/rangeradmin/contrib/solr_for_audit_setup/conf 
可以通过下面命令查看上传的实体： 
solrctl instancedir --list 
创建 collection 
创建ranger_audits
solrctl collection --create ranger_audits -s 1 -c ranger_audits -r 1 -m 1
验证：登陆zookeeper查看
cd /opt/cloudera/parcels/CDH-6.3.1-1.cdh6.3.1.p0.1470567/lib/zookeeper/bin/
./zkCli.sh -server 1XXXXXXX.38:2181
ls /ranger_audits
[configs, overseer, aliases.json, live_nodes, collections, overseer_elect, security.json, clusterstate.json, solr.xml, autoscaling, autoscaling.json, clusterprops.json]

Ranger时区问题
修改ranger-admin-site.xml配置文件
vi /opt/ranger/rangeradmin/conf/ranger-admin-site.xml
<property>
   <name>ranger.jpa.jdbc.url</name>
   <value>jdbc:log4jdbc:mysql://1XXXXXXX.39/ranger?serverTimezone=Asia/Shanghai</value>
   <description />
</property>

重启Ranger Admin
ranger-admin restart


###############################usersync安装##########################################
安装usersync
解压ranger-2.1.0-usersync.tar.gz文件
tar -zxvf ranger-2.1.0-usersync.tar.gz -C /opt/ranger/

修改/opt/ranger-2.1.0-admin/ranger-2.1.0-usersync/install.properties 文件
vim install.properties
#配置 Ranger-Admin的访问地址
POLICY_MGR_URL = http://1XXXXXXX.38:6080
#配置同步用户的周期（分钟）
SYNC_INTERVAL = 1
#配置usersync进程的操作用户及组
unix_user=sunxy
unix_group=sunxy
#设置Usersync用户的密码，之前安装Ranager-Admin配置过
rangerUsersync_password=TestCDH2024
#配置Hadoop路径
hadoop_conf=/etc/hadoop/conf

执行“setup.sh”脚本进行安装 usersync 模块
进入到解压好的目录“ranger-2.1.0-usersync”下，执行脚本“setup.sh”
cd ranger-2.1.0-usersync
./setup.sh 
... ...
Provider jceks://file/etc/ranger/usersync/conf/rangerusersync.jceks was updated.
[I] Successfully updated password of rangerusersync user

配置ranger-ugsync-site.xml
进入到目录/ranger-2.1.0-usersync/conf下配置ranger-ugsync-site.xml设置开启自动同步用户
<property>
      <name>ranger.usersync.enabled</name>
      <value>true</value>
</property>
启动usersync 模块
#任意目录下执行以下命令，启动usersync模块
ranger-usersync start
注意：如果启动过程中有错误，可以在/ranger-2.1.0-usersync/logs目录下查看日志。
验证是否安装成功：在Ranger控制台可以看到users中同步的用户信息。

报错处理：
1、查看x_portal_user_role表中rangerusersync用户的user_role是否为ROLE_SYS_ADMIN，若不是，则修改。
select user_role from ranger.x_portal_user_role where user_id=(select id from ranger.x_user where user_name='rangerusersync');
use ranger；
select * from x_portal_user_role;
update x_portal_user_role set user_role='ROLE_SYS_ADMIN' where user_id=3;
./ranger-usersync-services.sh  restart
2、/etc/ranger/usersync/conf/ranger-ugsync-site.xml
查看/etc/ranger/usersync/conf/ranger-ugsync-site.xml(install.properties中ranger_base_dir配置)
发现ranger.usersync.enabled为false，修改为true。重启ranger-usersync，再看管理界面，已经同步成功。


###############################hdfs插件安装##########################################
插件必须安装在NameNode节点!!!
解压ranger-2.1.0-hdfs-plugin.tar.gz文件
tar -zxvf ranger-2.1.0-hdfs-plugin.tar.gz -C /opt/ranger/rangerhdfs

在ranger hdfs插件目录下创建hadoop/etc目录
mkdir -p /opt/ranger/rangerhdfs/hadoop/etc

将hadoop安装目录下的jar包和其下lib目录下的jar包链接到/opt/ranger/rangerhdfs/hadoop目录下
cd  /opt/ranger/rangerhdfs/hadoop
ln -s /opt/cloudera/parcels/CDH/lib/hadoop/*.jar ./
ln -s /opt/cloudera/parcels/CDH/lib/hadoop/lib/*.jar ./

将hadoop安装目录下的配置文件目录链接到/opt/ranger/rangerhdfs/hadoop/etc下
cd /opt/ranger/rangerhdfs/hadoop/etc
ln -s /opt/cloudera/parcels/CDH/lib/hadoop/etc/hadoop ./hadoop

将ranger hdfs插件下的文件复制到hadoop的安装目录：
# ll /opt/ranger/rangerhdfs/lib
drwxr-xr-x 2 root root  4096 Apr 22 09:22 ranger-hdfs-plugin-impl
-rw-r--r-- 1 root root 16087 Apr 22 00:23 ranger-hdfs-plugin-shim-2.1.0.jar
-rw-r--r-- 1 root root 17684 Apr 22 00:18 ranger-plugin-classloader-2.1.0.jar
链接到hadoop安装目录的lib目录下(SecondaryNamenode节点也需要这几个文件，否则启动hadoop报错)：
# cd /opt/cloudera/parcels/CDH/lib/hadoop/lib/
ln -s /opt/ranger/rangerhdfs/lib/* ./

配置ranger hdfs插件的/opt/ranger/rangerhdfs/install.properties
#ranger admin的路径
POLICY_MGR_URL=http://1XXXXXXX.38:6080
#hdfs服务名称，后面会匹配到在ranger的web界面创建的hdfs服务
REPOSITORY_NAME=hdfspoc
#本来是hadoop的安装路径，ranger hdfs会去读取hadoop中的依赖以及配置文件，由于是CDH,与apache的安装后的文件路径有些差异，所以将hadoop相关的文件导入到这里
COMPONENT_INSTALL_DIR_NAME=/opt/ranger/rangerhdfs/hadoop
#mysql驱动的路径
SQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-javaold.jar
#Enable audit logs to Solr
XAAUDIT.SOLR.ENABLE=true
XAAUDIT.SOLR.URL=http://1XXXXXXX.38:8983/solr/ranger_audits
XAAUDIT.SOLR.USER=NONE
XAAUDIT.SOLR.PASSWORD=NONE
XAAUDIT.SOLR.ZOOKEEPER=NONE
XAAUDIT.SOLR.FILE_SPOOL_DIR=/var/log/hadoop/hdfs/audit/solr/spool
#Solr Audit Provider
XAAUDIT.SOLR.IS_ENABLED=true
XAAUDIT.SOLR.MAX_QUEUE_SIZE=1
XAAUDIT.SOLR.MAX_FLUSH_INTERVAL_MS=1000
XAAUDIT.SOLR.SOLR_URL=http://1XXXXXXX.38:8983/solr/ranger_audits
#配置操作HDFS插件的用户和所属组
CUSTOM_USER=hdfs
CUSTOM_GROUP=supergroup


启动ranger hdfs初始化
sh enable-hdfs-plugin.sh 
初始化完成后会在/opt/ranger/rangerhdfs/hadoop/etc/hadoop目录下生成3个文件
ll /opt/ranger/rangerhdfs/hadoop/etc/hadoop/ | grep ranger
-rwxr--r-- 1 hdfs hadoop 10710 Sep 30 08:31 ranger-hdfs-audit.xml
-rwxr--r-- 1 hdfs hadoop  3710 Sep 30 08:31 ranger-hdfs-security.xml
-rwxr--r-- 1 hdfs hadoop  1907 Sep 30 08:31 ranger-policymgr-ssl.xml


在CDH页面，编辑hdfs-site.xml的NameNode高级配置-新增配置：
dfs.permissions.enabled=true -- 启用权限
dfs.namenode.acls.enabled=true -- 开启ACL
dfs.permissions=true -- 文件操作时，检查权限
<!-- 以下两项是关于Ranger安全检查配置 -->
dfs.namenode.inode.attributes.provider.class=org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer
dfs.permissions.ContentSummary.subAccess=true
重启hadoop集群，使其生效

Ranger页面增加hdfs服务
servicename必须和install.properties中的REPOSITORY_NAME填写一致! hdfspoc
username 为操作系统的用户 sunxy TestCDH2024
以下内容参考hdfs-site.xml和core-site.xml参考填写
    <name>fs.defaultFS</name>
    <value>hdfs://IT-ES-Node01:8020,IT-ES-Node02:8020</value>
	<name>hadoop.security.authorization</name>
    <value>false</value>
    <name>hadoop.security.authentication</name>
    <value>simple</value>
添加以下配置
tag.download.auth.users  hdfs
policy.download.auth.users hdfs

校验：
policy目录：/etc/ranger/$REPOSITORY_NAME/policycache/
ll /etc/ranger/hdfspoc/policycache/
total 12
-rw-r--r-- 1 hdfs hdfs 6281 Sep 30 11:50 hdfs_hdfspoc.json
-rw-r--r-- 1 hdfs hdfs  105 Sep 30 11:50 hdfs_hdfspoc_roles.json
log日志：
/opt/ranger/rangeradmin/ews/logs

报错：
2024-09-30 11:20:38,340 ERROR org.apache.ranger.authorization.hadoop.config.RangerPluginConfig: Copy ranger config file failed.
java.io.FileNotFoundException: Source '/opt/cloudera/parcels/CDH-6.3.1-1.cdh6.3.1.p0.1470567/lib/hadoop/etc/hadoop/xasecure-audit.xml' does not exist

# vim /opt/cloudera/parcels/CDH-6.3.1-1.cdh6.3.1.p0.1470567/lib/hadoop/etc/hadoop/xasecure-audit.xml
<!-- MYSQL的连接地址及用户名端口号，配置同RANGER ADMIN中INSTALL.PROPERTIES的MYSQL配置相同-->
<property>
  <name>xasecure.audit.jpa.javax.persistence.jdbc.url</name>
  <value>jdbc:mysql://1XXXXXXX.39:3306/ranger</value>
</property>
<property>
  <name>xasecure.audit.jpa.javax.persistence.jdbc.user</name>
  <value>ranger</value>
</property>
<property>
  <name>xasecure.audit.jpa.javax.persistence.jdbc.password</name>
  <value>TestCDH2024</value>
</property>

ranger权限管理不生效：
将HDFS umask从022更改为077 这将防止所有者以外的任何人访问任何新文件或文件夹。（如果不进行修改，用户创建的hdfs文件权限为：644、目录权限为：755，ranger授权形同虚设）
更改umask设置如下：
Ambari - HDFS - 配置参数 - advanced - 高级设置 hdfs-site
将值从022更改为077
重启hdfs组件之后，创建出来的hdfs文件（权限：600）及目录（权限：700）

验证：
bash-4.2$ hdfs dfs -put ttt333  /rangertest/
put: Permission denied: user=test, access=WRITE, inode="/rangertest":sunxy:supergroup:drwxr-xr-x
bash-4.2$ hdfs dfs -put ttt333  /rangertest/
bash-4.2$ 

###############################yarn插件安装##########################################
# cd /opt/ranger/rangeryarn
# vim install.properties

配置ranger admin的地址
POLICY_MGR_URL = http://1XXXXXXX.38:6080
配置yarn的仓库名
REPOSITORY_NAME=yarnpoc
配置hadoop组件的HADOOP_HOME
COMPONENT_INSTALL_DIR_NAME=/opt/cloudera/parcels/CDH/lib/hadoop
SQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-javaold.jar
配置yarn的审计日志
XAAUDIT.SOLR.ENABLE=true
XAAUDIT.SOLR.URL=http://1XXXXXXX.38:8983/solr/ranger_audits
XAAUDIT.SOLR.USER=NONE
XAAUDIT.SOLR.PASSWORD=NONE
XAAUDIT.SOLR.ZOOKEEPER=1XXXXXXX.38:2181,1XXXXXXX.39:2181,1XXXXXXX.40:2181/ranger_audits
XAAUDIT.SOLR.FILE_SPOOL_DIR=/var/log/hadoop/yarn/audit/solr/spool
配置ranger-yarn-plugin的所属用户、用户组
CUSTOM_USER=yarn
CUSTOM_GROUP=hadoop

修改yarn-site.xml配置文件
编辑yarn-site.xml高级配置-新增配置：
yarn.acl.enable=true
yarn.authorization-provider=org.apache.ranger.authorization.yarn.authorizer.RangerYarnAuthorizer
修改必须使用Capacity Scheduler：
yarn.resourcemanager.scheduler.class=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler
搜索：scheduler，在YARN服务的capacity-scheduler.xml中，确认Capacity Scheduler调度器的配置
yarn.scheduler.capacity.root.acl_submit_applications= （配置值为单个空格）
yarn.scheduler.capacity.root.acl_administer_queue= hadoop（hadoop前有单个空格），表示所有队列默认将队列管理权限授权给hadoop组。
确认除了上述root队列ACL配置外没有其他acl_submit_applications或者acl_administer_queue配置，避免YARN ACL影响Ranger鉴权预期结果。
搜索：yarn.scheduler.capacity.root.queues 配置队列，以XML格式查看，导入以下配置后重启yarn。
<configuration>
<property>
    <name>yarn.scheduler.capacity.maximum-applications</name>
    <value>10000</value>
  </property>
  <property>
    <name>yarn.scheduler.capacity.maximum-am-resource-percent</name>
    <value>0.1</value>
  </property>
  <property>
    <name>yarn.scheduler.capacity.resource-calculator</name>
    <value>org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator</value>
  </property>
  <property>
    <name>yarn.scheduler.capacity.root.queues</name>
    <value>default,demo</value>
  </property>
  <property>
    <name>yarn.scheduler.capacity.root.default.capacity</name>
    <value>40</value>
  </property>
  <property>
    <name>yarn.scheduler.capacity.root.demo.capacity</name>
    <value>60</value>
  </property>
  <property>
    <name>yarn.scheduler.capacity.root.default.user-limit-factor</name>
    <value>1</value>
  </property>
  <property>
    <name>yarn.scheduler.capacity.root.demo.user-limit-factor</name>
    <value>1</value>
  </property>
  <property>
    <name>yarn.scheduler.capacity.root.default.maximum-capacity</name>
    <value>100</value>
  </property>
  <property>
    <name>yarn.scheduler.capacity.root.demo.maximum-capacity</name>
    <value>100</value>
  </property>
  <property>
    <name>yarn.scheduler.capacity.root.default.state</name>
    <value>RUNNING</value>
  </property>
  <property>
    <name>yarn.scheduler.capacity.root.demo.state</name>
    <value>RUNNING</value>
  </property>
  <property>
    <name>yarn.scheduler.capacity.root.default.acl_submit_applications</name>
    <value>*</value>
  </property>
  <property>
    <name>yarn.scheduler.capacity.root.demo.acl_submit_applications</name>
    <value>*</value>
  </property>
  <property>
    <name>yarn.scheduler.capacity.root.default.acl_administer_queue</name>
    <value>*</value>
  </property>
  <property>
    <name>yarn.scheduler.capacity.root.demo.acl_administer_queue</name>
    <value>*</value>
  </property>
  <property>
    <name>yarn.scheduler.capacity.root.default.acl_application_max_priority</name>
    <value>*</value>
  </property>
  <property>
    <name>yarn.scheduler.capacity.root.demo.acl_application_max_priority</name>
    <value>*</value>
  </property>
   <property>
     <name>yarn.scheduler.capacity.root.default.maximum-application-lifetime </name>
     <value>-1</value>
   </property>
   <property>
     <name>yarn.scheduler.capacity.root.demo.maximum-application-lifetime </name>
     <value>-1</value>
   </property>
   <property>
     <name>yarn.scheduler.capacity.root.default.default-application-lifetime </name>
     <value>-1</value>
   </property>
   <property>
     <name>yarn.scheduler.capacity.root.demo.default-application-lifetime </name>
     <value>-1</value>
   </property>
  <property>
    <name>yarn.scheduler.capacity.node-locality-delay</name>
    <value>40</value>
  </property>
  <property>
    <name>yarn.scheduler.capacity.rack-locality-additional-delay</name>
    <value>-1</value>
  </property>
  <property>
    <name>yarn.scheduler.capacity.queue-mappings</name>
    <value></value>
  </property>
  <property>
    <name>yarn.scheduler.capacity.queue-mappings-override.enable</name>
    <value>false</value>
  </property>
  <property>
    <name>yarn.scheduler.capacity.per-node-heartbeat.maximum-offswitch-assignments</name>
    <value>1</value>
  </property>
    <property>
    <name>yarn.scheduler.capacity.root.acl_submit_applications</name>
    <value> </value>
  </property>
    <property>
    <name>yarn.scheduler.capacity.root.acl_administer_queue</name>
    <value> hadoop</value>
  </property>
</configuration>  

重启yarn，以上配置生效后可以进入Web UI页面查看，队列设置是否正确。

启动ranger hdfs初始化
# cd /opt/ranger/rangeryarn
# ./enable-yarn-plugin.sh

重启yarn

配置yarn组件服务
service name要与ranger-yarn-plugin的install.properties配置的REPOSITORY_NAME值要一致。
http://1XXXXXXX.38:8088
添加以下配置
tag.download.auth.users  yarn
policy.download.auth.users yarn


###############################hive插件安装##########################################
Hive开启Ranger权限控制后，HiveServer2服务会加载Ranger Hive plugin，仅在您通过HiveServer2提交SQL作业时需要进行权限校验，其他方式访问Hive将不会触发权限校验。
支持权限校验的访问方式
通过Beeline客户端访问HiveServer2。
通过JDBC URL连接HiveServer2。
不支持权限校验的访问方式
通过Hive客户端直接连接Metastore。
通过Hive-Client API直接连接Metastore。

/opt/ranger/rangerhive修改install.properties文件
cd /opt/ranger/rangerhive
vi install.properties
#ranger admin的路径
POLICY_MGR_URL=http://1XXXXXXX.38:6080
#hive服务名称
REPOSITORY_NAME=hivepoc
#hive的安装路径
COMPONENT_INSTALL_DIR_NAME=/opt/cloudera/parcels/CDH/lib/hive
SQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-javaold.jar
#使用solr作为审计
XAAUDIT.SOLR.ENABLE=true
XAAUDIT.SOLR.URL=http://1XXXXXXX.38:8983/solr/ranger_audits
XAAUDIT.SOLR.USER=NONE
XAAUDIT.SOLR.PASSWORD=NONE
XAAUDIT.SOLR.ZOOKEEPER=1XXXXXXX.38:2181,1XXXXXXX.39:2181,1XXXXXXX.40:2181/ranger_audits
XAAUDIT.SOLR.FILE_SPOOL_DIR=/var/log/hive/audit/solr/spool
CUSTOM_USER=hive
CUSTOM_GROUP=hadoop

启动ranger hive初始化
./enable-hive-plugin.sh

在/opt/cloudera/parcels/CDH/lib/hive/conf目录下生成五个文件
-rwxr--r-- 1 hive hadoop 9634 Sep 30 16:06 ranger-hive-audit.xml
-rwxr--r-- 1 hive hadoop 2905 Sep 30 16:06 ranger-hive-security.xml
-rwxr--r-- 1 hive hadoop 1906 Sep 30 16:06 ranger-policymgr-ssl.xml
-rw-r--r-- 1 hive hadoop   69 Sep 30 16:06 ranger-security.xml
-rwxr--r-- 1 hive hadoop 1588 Sep 30 16:06 hiveserver2-site.xml

在/opt/cloudera/parcels/CDH/lib/hive/lib目录下生成两个JAR文件软连接和一个目录软连接
# ll /opt/cloudera/parcels/CDH/lib/hive/lib/ | grep ranger
lrwxrwxrwx 1 root root    50 Sep 30 16:06 ranger-hive-plugin-impl -> /opt/ranger/rangerhive/lib/ranger-hive-plugin-impl
lrwxrwxrwx 1 root root    60 Sep 30 16:06 ranger-hive-plugin-shim-2.1.0.jar -> /opt/ranger/rangerhive/lib/ranger-hive-plugin-shim-2.1.0.jar
lrwxrwxrwx 1 root root    62 Sep 30 16:06 ranger-plugin-classloader-2.1.0.jar -> /opt/ranger/rangerhive/lib/ranger-plugin-classloader-2.1.0.jar

在ranger hive的配置文件目录新建xasecure-audit.xml文件
# vim /opt/ranger/rangerhive/install/conf.templates/enable/xasecure-audit.xml
<!-- MYSQL的连接地址及用户名端口号，配置同RANGER ADMIN中INSTALL.PROPERTIES的MYSQL配置相同-->
<property>
  <name>xasecure.audit.jpa.javax.persistence.jdbc.url</name>
  <value>jdbc:mysql://1XXXXXXX.39:3306/ranger</value>
</property>
<property>
  <name>xasecure.audit.jpa.javax.persistence.jdbc.user</name>
  <value>ranger</value>
</property>
<property>
  <name>xasecure.audit.jpa.javax.persistence.jdbc.password</name>
  <value>TestCDH2024</value>
</property>

修改vi /opt/cloudera/parcels/CDH/lib/hive/conf/hive-env.sh
注释export HIVE_OPTS配置（集群所有机器都要修改）
#export HIVE_OPTS="${HIVE_OPTS} --hiveconf hive.query.redaction.rules=${HIVE_CONF_DIR}/redaction-rules.json --hiveconf hive.exec.query.redactor.hooks=org.cloudera.hadoop.hive.ql.hooks.QueryRedactor"
./dep.sh /opt/cloudera/parcels/CDH/lib/hive/conf/hive-env.sh /opt/cloudera/parcels/CDH/lib/hive/conf/ all

CDH页面在hive-site.xml中添加如下参数，只允许beeline用户的方式访问，达到权限验证效果。
hive.security.authenticator.manager=org.apache.hadoop.hive.ql.security.SessionStateUserAuthenticator
hive.security.authorization.manager=org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizerFactory
hive.security.authorization.enabled=true
hive.conf.restricted.list=hive.security.authorization.enabled,hive.security.authorization.manager,hive.security.authenticator.manager

重启hive服务，hive的策略缓存目录中生成文件
# ll /etc/ranger/hivepoc/policycache/

ranger配置hive服务
service name要与install.properties配置的REPOSITORY_NAME值要一致。hivepoc
jdbc:hive2://1XXXXXXX.38:10000  可以通过
jdbc:hive2://1XXXXXXX.38:2181,1XXXXXXX.39:2181,1XXXXXXX.40:2181/default;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2_zk  没通过！！！

hive无法使用zk连接beeline，需要修改hive-site.xml 的 HiveServer2 高级配置代码段配置：
<property>
   <name>hive.server2.support.dynamic.service.discovery</name>
   <value>true</value>
   <final>true</final>
   <description>配置hiveserver HA</description>
</property>
<property>
   <name>hive.server2.zookeeper.namespace</name>
   <value>hiveserver2_zk</value>
   <final>true</final>
   <description>配置hiveserver HA</description>
</property>
<property>
   <name>hive.zookeeper.quorum</name>
   <value>1XXXXXXX.38:2181,
1XXXXXXX.39:2181,
1XXXXXXX.40:2181</value>
   <final>true</final>
   <description>配置hiveserver HA</description>
</property>
<property>
   <name>hive.zookeeper.client.port</name>
   <value>2181</value>
   <final>true</final>
   <description>配置hiveserver HA</description>
</property>
<property>
   <name>hive.server2.async.exec.threads</name>
   <value>1000</value>
   <description>hive异步执行连接池大小</description>
</property>
<property>
   <name>hive.server2.async.exec.wait.queue.size</name>
   <value>1000</value>
   <description>hive异步执行连接池队列大小</description>
</property>
<property>
   <name>hive.server2.async.exec.shutdown.timeout</name>
   <value>600</value>
   <description>异步线程结束的超时时间</description>
</property>
<property>
   <name>hive.server2.async.exec.keepalive.time</name>
   <value>600</value>
   <description>异步线程的等待超时时间</description>
</property>


查看znode的ns：
cd /opt/cloudera/parcels/CDH-6.3.1-1.cdh6.3.1.p0.1470567/lib/zookeeper/bin/
./zkCli.sh -server 1XXXXXXX.38:2181
> ls /
[cluster, controller, brokers, zookeeper, hadoop-ha, admin, isr_change_notification, log_dir_event_notification, controller_epoch, ranger_audits, consumers, hive_zookeeper_namespace_hive, latest_producer_id_block, config]
ls /hiveserver2_zk
[serverUri=IT-ES-Node01:10000;version=2.1.1-cdh6.3.1;sequence=0000000000]

beeline -u "jdbc:hive2://1XXXXXXX.38:2181,1XXXXXXX.39:2181,1XXXXXXX.40:2181/default;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2_zk" -n sunxy -p TestCDH2024   可以通过！！！

增加参数：
tag.download.auth.users  hive
policy.download.auth.users hive

测试连接报错：点击测试连接后会提示连接失败，具体原因是sunxy用户没有访问hive表的权限，
这是因为到目前为止，我们还未使用Ranger向任何用户赋予任何权限，故此时连接失败为正常现象。
org.apache.ranger.plugin.client.HadoopException: Unable to execute SQL [show databases like "*"]..
Error while compiling statement: FAILED: HiveAccessControlException Permission denied: user [sunxy] does not have [USE] privilege on [*].
Permission denied: user [sunxy] does not have [USE] privilege on [*].

点击Add按钮继续，重新测试即可成功！！！