本文档详细介绍了在CDH 5.8.4环境中，如何编译并安装Ranger 0.5.4-SNAPSHOT，包括解决HDFS编译问题，配置Solr日志审计，以及安装Ranger Admin和Hive Plugin。此外，还涉及到了权限策略配置和Hive的创建、更改权限操作。
组件版本信息：
Hadoop 版本：cdh-5.8.4
Hive 版本: 1.1.0-cdh5.8.4
Ranger 版本: ranger-0.5.4-SNAPSHOT
Solr 版本: 5.2.1
Jdk 版本：1.7.80
前提条件：
cdh-5.8.4 需要修复 HDFS-6826，否则Ranger HDFS Plugin编译无法通过
本地mvn仓库编译安装修改后hdfs依赖包：
[cloudera@quickstart hadoop-2.6.0-cdh5.8.4-src]$ mvn clean install package -Dhttps.protocols=TLSv1,TLSv1.1,TLSv1.2 -Dcdh.mr1.version=2.6.0-mr1-cdh5.8.4 -Dcdh.hadoop.version=2.6.0-cdh5.8.4 -Dcdh.hbase.version=1.2.0-cdh5.8.4 -Dcdh.zookeeper.version=3.4.5-cdh5.8.4 -Phadoop-2 -Dmaven.test.skip=true
编译Ranger：
JDK 版本：
[cloudera@quickstart ~]$ java -version
java version "1.7.0_80"
[cloudera@quickstart ~]$ git clone https://github.com/apache/ranger.git
[cloudera@quickstart ranger]$ git checkout ranger-0.5
POM文件修改下面Hadoop,Hive相关内容:
+ <hadoop-auth.version>2.6.0-cdh5.8.4</hadoop-auth.version>
+ <hadoop-common.version>2.6.0-cdh5.8.4</hadoop-common.version>
+ <hadoop-common-kms.version>2.6.0-cdh5.8.4</hadoop-common-kms.version>
+ <hadoop.version>2.6.0-cdh5.8.4</hadoop.version>
+ <hive.version>1.1.0-cdh5.8.4</hive.version>
[cloudera@quickstart ranger]$ mvn -Drat.skip=true -Dhttps.protocols=TLSv1,TLSv1.1,TLSv1.2 compile install package assembly:assembly
安装Solr记录审计日志：
官网建议要求
JDK 1.7, Apache Solr 5.2
Solr memory and cpu intensive
32GB RAM, 1TB free space
Solr - Standalone
172.17.8.4
[cloudera@quickstart ~] tar -cvf ranger.tar.gz ranger/*
[cloudera@quickstart ~] scp ranger.tar.gz xxxxxxx@172.17.8.4:/home/xxxxxxx/
[xxxxxxx@xxxx-dp-dev05-84 ~]$ tar -xvf ranger.tar.gz
[xxxxxxx@xxxx-dp-dev05-84 ~]$ cd ranger/security-admin/contrib/solr_for_audit_setup/
[xxxxxxx@xxxx-dp-dev05-84 solr_for_audit_setup]$ pwd
/home/xxxxxxx/ranger/security-admin/contrib/solr_for_audit_setup
编辑自动安装配置：
[xxxxxxx@xxxx-dp-dev05-84 solr_for_audit_setup]$ vim install.properties
[xxxxxxx@xxxx-dp-dev05-84 solr_for_audit_setup]$ cat install.properties | grep -v "^#" | grep -v "^$"
注意下面配置项：
JAVA_HOME=/usr/java/jdk1.7.0_80
SOLR_USER=solr
SOLR_INSTALL=true
SOLR_DOWNLOAD_URL=http://archive.apache.org/dist/lucene/solr/5.2.1/solr-5.2.1.tgz
SOLR_INSTALL_FOLDER=/opt/solr
SOLR_RANGER_HOME=/opt/solr/ranger_audit_server
SOLR_RANGER_PORT=6083
SOLR_DEPLOYMENT=standalone
SOLR_RANGER_DATA_FOLDER=/opt/solr/ranger_audit_server/data
SOLR_LOG_FOLDER=/var/log/solr/ranger_audits
SOLR_MAX_MEM=2g
自动安装：
[xxxxxxx@xxxx-dp-dev05-84 solr_for_audit_setup]$ sudo ./setup.sh
Fri Oct 11 17:30:41 CST 2019|INFO|Downloading solr from http://archive.apache.org/dist/lucene/solr/5.2.1/solr-5.2.1.tgz
--2019-10-11 17:30:41-- http://archive.apache.org/dist/lucene/solr/5.2.1/solr-5.2.1.tgz
Resolving archive.apache.org (archive.apache.org)... 163.172.17.199
Connecting to archive.apache.org (archive.apache.org)|163.172.17.199|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 148849766 (142M) [application/x-gzip]
Saving to: ‘solr-5.2.1.tgz’
100%[=========================================================================================>] 148,849,766 3.41MB/s in 4m 49s
2019-10-11 17:35:31 (503 KB/s) - ‘solr-5.2.1.tgz’ saved [148849766/148849766]
Fri Oct 11 17:35:33 CST 2019|INFO|Installed Solr in /opt/solr
Fri Oct 11 17:35:33 CST 2019|INFO|Configuring standalone instance
Fri Oct 11 17:35:33 CST 2019|INFO|Copying Ranger Audit Server configuration to /opt/solr/ranger_audit_server
df: ‘/opt/solr/ranger_audit_server/data’: No such file or directory
Fri Oct 11 17:35:35 CST 2019|INFO|Done configuring Solr for Apache Ranger Audit
Fri Oct 11 17:35:35 CST 2019|INFO|Solr HOME for Ranger Audit is /opt/solr/ranger_audit_server
Fri Oct 11 17:35:35 CST 2019|INFO|Data folder for Audit logs is /opt/solr/ranger_audit_server/data
Fri Oct 11 17:35:35 CST 2019|INFO|To start Solr run /opt/solr/ranger_audit_server/scripts/start_solr.sh
Fri Oct 11 17:35:35 CST 2019|INFO|To stop Solr run /opt/solr/ranger_audit_server/scripts/stop_solr.sh
Fri Oct 11 17:35:35 CST 2019|INFO|After starting Solr for RangerAudit, it will listen at 6083. E.g http://xxxx-dp-dev05-84.jpushoa.com:6083
Fri Oct 11 17:35:35 CST 2019|INFO|Configure Ranger to use the following URL http://xxxx-dp-dev05-84.jpushoa.com:6083/solr/ranger_audits
########## Done ###################
Created file /opt/solr/ranger_audit_server/install_notes.txt with instructions to start and stop
###################################
[xxxxxxx@xxxx-dp-dev05-84 solr_for_audit_setup]$ cat /opt/solr/ranger_audit_server/install_notes.txt
Solr installation notes for Ranger Audits.
Note: Don't edit this file. It will be over written if you run ./setup.sh again.
You have installed Solr in standalone mode.
Note: In production deployment, it is recommended to run in SolrCloud mode with at least 2 nodes and replication factor 2
Start and Stoping Solr:
Login as user solr or root and the run the below commands to start or stop Solr:
To start Solr run: /opt/solr/ranger_audit_server/scripts/start_solr.sh
To stop Solr run: /opt/solr/ranger_audit_server/scripts/stop_solr.sh
After starting Solr for RangerAudit, Solr will listen at 6083. E.g http://xxxx-dp-dev05-84.jpushoa.com:6083
Configure Ranger to use the following URL http://xxxx-dp-dev05-84.jpushoa.com:6083/solr/ranger_audits
Solr HOME for Ranger Audit is /opt/solr/ranger_audit_server
DATA FOLDER: /opt/solr/ranger_audit_server/data
Make sure you have enough disk space for index. In production, it is recommended to have at least 1TB free.
[xxxxxxx@xxxx-dp-dev05-84 solr_for_audit_setup]$ sudo /opt/solr/ranger_audit_server/scripts/start_solr.sh
Running this script as solr...
This account is currently not available.
[xxxxxxx@xxxx-dp-dev05-84 solr_for_audit_setup]$ sudo useradd solr
useradd: user 'solr' already exists
[xxxxxxx@xxxx-dp-dev05-84 solr_for_audit_setup]$ sudo vim /etc/passwd
solr:x:597:600:Solr:/var/lib/solr:/bin/bash
[xxxxxxx@xxxx-dp-dev05-84 solr_for_audit_setup]$ sudo /opt/solr/ranger_audit_server/scripts/start_solr.sh
Running this script as solr...
Started Solr server on port 6083 (pid=21030). Happy searching!
网址：http://172.17.8.4:6083
配置Ranger时，注意修改下面配置项：
Configuring Ranger Admin and Ranger Plugins
Ranger Admin and Ranger Plugins need the URL to Solr collection. Check the install_notes.txt for the appropriate value. The sample URL is:
http://${SOLR_HOST}:6083/solr/ranger_audits (Replace ${SOLR_HOST} with the server were Solr is installed.
For Ranger Admin, configure the following properties in install.properties:
#Source for Audit DB
# * audit_db is solr or db
audit_store=solr
# * audit_solr_url URL to Solr. E.g. http://<solr_host>:6083/solr/ranger_audits
audit_solr_urls=http://localhost:6083/solr/ranger_audits
For all plugins, configure the following properties in install.properties
XAAUDIT.SOLR.ENABLE=true
XAAUDIT.SOLR.URL=http://localhost:6083/solr/ranger_audits
(replace localhost with the Solr host)
安装Ranger Admin(管理权限策略):
ranger admin 安装(172.17.8.94)：
[xxxxxxx@xxxx-dp-dev05-84 target]$ scp ranger-0.5.4-SNAPSHOT-admin.tar.gz 172.17.8.94:/home/xxxxxxx/
登录：172.17.8.94
[xxxxxxx@xxxx-dp-dev06-894 ~]$ tar -xvf ranger-0.5.4-SNAPSHOT-admin.tar.gz
[xxxxxxx@xxxx-dp-dev06-894 ~]$ cd /usr/local/
[xxxxxxx@xxxx-dp-dev06-894 local]$ pwd
/usr/local
[xxxxxxx@xxxx-dp-dev06-894 local]$ sudo mv ~/ranger-0.5.4-SNAPSHOT-admin ./
[xxxxxxx@xxxx-dp-dev06-894 local]$ sudo chown -R root:root ranger-0.5.4-SNAPSHOT-admin/
[xxxxxxx@xxxx-dp-dev06-894 local]$ sudo ln -s ranger-0.5.4-SNAPSHOT-admin ranger-admin
[xxxxxxx@xxxx-dp-dev06-894 local]$ cd /usr/local/ranger-admin/
建立ranger数据库管理员：
CREATE USER 'rangerdba'@'localhost' IDENTIFIED BY 'rangerdba';
GRANT ALL PRIVILEGES ON *.* TO 'rangerdba'@'localhost' WITH GRANT OPTION;
CREATE USER 'rangerdba'@'%' IDENTIFIED BY 'rangerdba';
GRANT ALL PRIVILEGES ON *.* TO 'rangerdba'@'%' WITH GRANT OPTION;
FLUSH PRIVILEGES;
[xxxxxxx@xxxx-dp-dev06-894 ranger-admin]$ sudo mv ~/mysql-connector-java-5.1.37.jar /usr/local/ranger-0.5.4-SNAPSHOT-admin/
[xxxxxxx@xxxx-dp-dev06-894 ranger-admin]$ sudo chown root:root mysql-connector-java-5.1.37.jar
配置文件：
[xxxxxxx@xxxx-dp-dev06-894 ranger-admin]$ sudo cat install.properties | grep -v "^#" | grep -v "^$"
DB_FLAVOR=MYSQL
SQL_CONNECTOR_JAR=/usr/local/ranger-admin/mysql-connector-java-5.1.37.jar
db_root_user=rangerdba
db_root_password=rangerdba
db_host=172.17.8.48:3306
db_name=ranger
db_user=rangeradmin
db_password=******
audit_store=solr
audit_solr_urls=http://172.17.8.4:6083/solr/ranger_audits
policymgr_external_url=http://localhost:6080
dba_script.py
db_setup.py
添加：
os.environ['JAVA_HOME'] = '/usr/java/jdk1.7.0_80'
/usr/local/ranger-0.5.4-SNAPSHOT-admin/db/mysql/create_dbversion_catalog.sql
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Invalid default value for 'updated_at'
SQLException : SQL state: 42000 com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Invalid default value for 'updated_at' ErrorCode: 1067
改为如下(建表语句时间戳默认值为当天时间戳的只能有一个)：
updated_at timestamp null default null,
Drop存储过程失败,执行下面的语句:
mysql_upgrade -hxxxx-dp-dev10-848 -uroot -p
java.sql.SQLException: Index column size too large. The maximum column size is 767 bytes.
建表语句添加：
) row_format=DYNAMIC;
[xxxxxxx@xxxx-dp-dev06-894 ranger-admin]$ sudo ./setup.sh
Installation of Ranger PolicyManager Web Application is completed.
启动服务：
[xxxxxxx@xxxx-dp-dev06-894 ranger-admin]$ sudo service ranger-admin start
http://172.17.8.94:6080
用户名：admin 密码：admin
如下图所示：
安装用户同步服务：
安装Ranger UserSync：
172.17.8.85
[xxxxxxx@xxxx-dp-dev05-84 ~]$ cd ranger/target/
[xxxxxxx@xxxx-dp-dev05-84 target]$ scp ranger-0.5.4-SNAPSHOT-usersync.tar.gz 172.17.8.85:/home/xxxxxxx/
[xxxxxxx@xxxx-dp-dev03-885 ~]$ cd /usr/local/
[xxxxxxx@xxxx-dp-dev03-885 local]$ sudo tar zxvf ~/ranger-0.5.4-SNAPSHOT-usersync.tar.gz
[xxxxxxx@xxxx-dp-dev03-885 local]$ sudo ln -s ranger-0.5.4-SNAPSHOT-usersync ranger-usersync
[xxxxxxx@xxxx-dp-dev03-885 local]$ sudo mkdir -p /var/log/ranger-usersync
[xxxxxxx@xxxx-dp-dev03-885 local]$ sudo useradd ranger
[xxxxxxx@xxxx-dp-dev03-885 local]$ sudo chown ranger:ranger /var/log/ranger-usersync/
修改配置文件：
[xxxxxxx@xxxx-dp-dev03-885 local]$ cd ranger-usersync/
[xxxxxxx@xxxx-dp-dev03-885 ranger-usersync]$ sudo cat install.properties | grep -v "^#" | grep -v "^$"
POLICY_MGR_URL = http://172.17.8.94:6080
SYNC_SOURCE = unix
MIN_UNIX_USER_ID_TO_SYNC = 500
SYNC_INTERVAL = 2
logdir=/var/log/ranger-usersync
[xxxxxxx@xxxx-dp-dev03-885 ranger-usersync]$ sudo su - root
[xxxxxxx@xxxx-dp-dev03-885 ranger-usersync]$ export JAVA_HOME=/usr/java/jdk1.7.0_80
[xxxxxxx@xxxx-dp-dev03-885 ~]$ sudo su - root
[root@xxxx-dp-dev03-885 ranger-usersync]# ./setup.sh
Caused by: java.lang.ClassNotFoundException: org.apache.htrace.core.Tracer$Builder
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 16 more
ERROR: Unable update the JCKSFile(/etc/ranger/usersync/conf/rangerusersync.jceks) for aliasName (usersync.ssl.key.password)
[root@xxxx-dp-dev03-885 ~]# cd /usr/local/ranger-usersync/lib
[root@xxxx-dp-dev03-885 lib]# wget https://repo1.maven.org/maven2/org/apache/htrace/htrace-core4/4.0.1-incubating/htrace-core4-4.0.1-incubating.jar
[root@xxxx-dp-dev03-885 lib]# cd ..
[root@xxxx-dp-dev03-885 ranger-usersync]# ./setup.sh
ranger.usersync.policymgr.password has been successfully created.
Provider jceks://file/etc/ranger/usersync/conf/rangerusersync.jceks has been updated.
[root@xxxx-dp-dev03-885 ranger-usersync]# ./ranger-usersync-services.sh start
Ranger Usersync Service has started successfully.
修改RangerAdmin 的 Authentication Modes：
[root@xxxx-dp-dev06-894 ranger-admin]# service ranger-admin stop
修改配置：
[root@xxxx-dp-dev06-894 ranger-admin]# cat install.properties
#LDAP|ACTIVE_DIRECTORY|UNIX|NONE
authentication_method=UNIX
remoteLoginEnabled=true
authServiceHostName=172.17.8.85
authServicePort=5151
[root@xxxx-dp-dev06-894 ranger-admin]# cat setup.sh | grep "^#"
#setup_unix_user_group
#run_dba_steps
#if [ "$?" == "0" ]
#then
#$PYTHON_COMMAND_INVOKER db_setup.py
#else
# exit 1
#fi
#$PYTHON_COMMAND_INVOKER db_setup.py -javapatch
[root@xxxx-dp-dev06-894 ranger-admin]# export JAVA_HOME=/usr/java/jdk1.7.0_80
[root@xxxx-dp-dev06-894 ranger-admin]# ./setup.sh
Installation of Ranger PolicyManager Web Application is completed.
成功后，Ranger Admin会出现用户组同步信息：
安装HDFS Plugin:
HDFS权限管理(Fedration 2组 Nameservice)：
172.17.8.79
172.17.8.93
172.17.8.85
172.17.8.59
替换hdfs jar包:
[xxxxxxx@xxxx-dp-dev04-859 ~]$ sudo su - root
Last login: Tue Oct 15 10:54:14 CST 2019 on pts/0
[root@xxxx-dp-dev04-859 ~]# cd /opt/cloudera/parcels/CDH/jars/
[root@xxxx-dp-dev04-859 jars]# mv hadoop-hdfs-2.6.0-cdh5.8.4.jar hadoop-hdfs-2.6.0-cdh5.8.4.jar.20191015
[root@xxxx-dp-dev04-859 jars]# cp /home/xxxxxxx/hadoop-hdfs-2.6.0-cdh5.8.4.jar ./
[root@xxxx-dp-dev01-879 xxxxxxx]# cd /opt/cloudera/parcels/CDH/lib
[root@xxxx-dp-dev01-879 lib]# cp /home/xxxxxxx/ranger-0.5.4-SNAPSHOT-hdfs-plugin.tar.gz ./
[root@xxxx-dp-dev01-879 lib]# tar -xvf ranger-0.5.4-SNAPSHOT-hdfs-plugin.tar.gz
[root@xxxx-dp-dev01-879 lib]# ln -s ranger-0.5.4-SNAPSHOT-hdfs-plugin ranger-hdfs-plugin
[root@xxxx-dp-dev01-879 lib]# cd ranger-hdfs-plugin/
[root@xxxx-dp-dev01-879 ranger-hdfs-plugin]# vim install.properties
[root@xxxx-dp-dev01-879 ranger-hdfs-plugin]# cat install.properties | grep -v "^#" | grep -v "^$"
POLICY_MGR_URL=http://172.17.8.94:6080
SQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-java.jar
REPOSITORY_NAME=hadoopdev
XAAUDIT.SOLR.ENABLE=true
XAAUDIT.SOLR.URL=http://172.17.8.4:6083/solr/ranger_audits
XAAUDIT.DB.IS_ENABLED=false
XAAUDIT.DB.FLAVOUR=MYSQL
XAAUDIT.DB.HOSTNAME=localhost # useless
XAAUDIT.DB.DATABASE_NAME=ranger_audit # useless
XAAUDIT.DB.USER_NAME=rangerlogger # useless
XAAUDIT.DB.PASSWORD=****** # useless
[root@xxxx-dp-dev01-879 ranger-hdfs-plugin]# export HADOOP_HOME=/opt/cloudera/parcels/CDH/lib/hadoop
[root@xxxx-dp-dev01-879 ranger-hdfs-plugin]# export JAVA_HOME=/usr/java/jdk1.7.0_80
报错Hadoop配置路径不存在：
[root@xxxx-dp-dev01-879 ranger-hdfs-plugin]# ln -s ../hadoop/etc/hadoop/ ../hadoop/conf
报错：
Caused by: java.lang.ClassNotFoundException: org.apache.htrace.core.Tracer$Builder
[root@xxxx-dp-dev01-879 ranger-hdfs-plugin]# cp ../../jars/htrace-core4-4.0.1-incubating.jar install/lib/
[root@xxxx-dp-dev01-879 ranger-hdfs-plugin]# ./enable-hdfs-plugin.sh
Custom user and group is available, using custom user and group.
+ Tue Oct 15 16:37:29 CST 2019 : hadoop: lib folder=/opt/cloudera/parcels/CDH/lib/hadoop/lib conf folder=/opt/cloudera/parcels/CDH/lib/hadoop/conf
+ Tue Oct 15 16:37:29 CST 2019 : Saving /opt/cloudera/parcels/CDH/lib/hadoop/conf/ranger-hdfs-audit.xml to /opt/cloudera/parcels/CDH/lib/hadoop/conf/.ranger-hdfs-audit.xml.20191015-163729 ...
+ Tue Oct 15 16:37:29 CST 2019 : Saving /opt/cloudera/parcels/CDH/lib/hadoop/conf/ranger-hdfs-security.xml to /opt/cloudera/parcels/CDH/lib/hadoop/conf/.ranger-hdfs-security.xml.20191015-163729 ...
+ Tue Oct 15 16:37:29 CST 2019 : Saving /opt/cloudera/parcels/CDH/lib/hadoop/conf/ranger-policymgr-ssl.xml to /opt/cloudera/parcels/CDH/lib/hadoop/conf/.ranger-policymgr-ssl.xml.20191015-163729 ...
+ Tue Oct 15 16:37:29 CST 2019 : Saving current config file: /opt/cloudera/parcels/CDH/lib/hadoop/conf/hdfs-site.xml to /opt/cloudera/parcels/CDH/lib/hadoop/conf/.hdfs-site.xml.20191015-163729 ...
+ Tue Oct 15 16:37:29 CST 2019 : Saving current config file: /opt/cloudera/parcels/CDH/lib/hadoop/conf/ranger-hdfs-audit.xml to /opt/cloudera/parcels/CDH/lib/hadoop/conf/.ranger-hdfs-audit.xml.20191015-163729 ...
+ Tue Oct 15 16:37:30 CST 2019 : Saving current config file: /opt/cloudera/parcels/CDH/lib/hadoop/conf/ranger-hdfs-security.xml to /opt/cloudera/parcels/CDH/lib/hadoop/conf/.ranger-hdfs-security.xml.20191015-163729 ...
+ Tue Oct 15 16:37:30 CST 2019 : Saving current config file: /opt/cloudera/parcels/CDH/lib/hadoop/conf/ranger-policymgr-ssl.xml to /opt/cloudera/parcels/CDH/lib/hadoop/conf/.ranger-policymgr-ssl.xml.20191015-163729 ...
+ Tue Oct 15 16:37:30 CST 2019 : Saving lib file: /opt/cloudera/parcels/CDH/lib/hadoop/lib/ranger-hdfs-plugin-shim-0.5.4-SNAPSHOT.jar to /opt/cloudera/parcels/CDH/lib/hadoop/lib/.ranger-hdfs-plugin-shim-0.5.4-SNAPSHOT.jar.20191015163730 ...
+ Tue Oct 15 16:37:30 CST 2019 : Saving lib file: /opt/cloudera/parcels/CDH/lib/hadoop/lib/ranger-plugin-classloader-0.5.4-SNAPSHOT.jar to /opt/cloudera/parcels/CDH/lib/hadoop/lib/.ranger-plugin-classloader-0.5.4-SNAPSHOT.jar.20191015163730 ...
+ Tue Oct 15 16:37:31 CST 2019 : Saving current JCE file: /etc/ranger/hadoopdev/cred.jceks to /etc/ranger/hadoopdev/.cred.jceks.20191015163731 ...
+ Tue Oct 15 16:37:33 CST 2019 : Saving current JCE file: /etc/ranger/hadoopdev/cred.jceks to /etc/ranger/hadoopdev/.cred.jceks.20191015163733 ...
Ranger Plugin for hadoop has been enabled. Please restart hadoop to ensure that changes are effective.
[root@xxxx-dp-dev01-879 lib]# vim /usr/lib64/cmf/service/common/cdh-default-hadoop
[root@xxxx-dp-dev01-879 lib]# cat /usr/lib64/cmf/service/common/cdh-default-hadoop | grep "cp"
cp /etc/hadoop/conf/ranger-hdfs-audit.xml $CONF_DIR/
cp /etc/hadoop/conf/ranger-hdfs-security.xml $CONF_DIR/
cp /etc/hadoop/conf/ranger-policymgr-ssl.xml $CONF_DIR/
cp /etc/hadoop/conf/ranger-security.xml $CONF_DIR/
[root@xxxx-dp-dev01-879 lib]# tar -cvf ranger-0.5.4-SNAPSHOT-hdfs-plugin.tar.gz ranger-0.5.4-SNAPSHOT-hdfs-plugin/
[root@xxxx-dp-dev01-879 lib]# mv ranger-0.5.4-SNAPSHOT-hdfs-plugin.tar.gz /tmp/
[root@xxxx-dp-dev01-879 lib]# chown xxxxxxx:users /tmp/ranger-0.5.4-SNAPSHOT-hdfs-plugin.tar.gz
[root@xxxx-dp-dev01-879 lib]# exit
logout
拷贝文件：
[xxxxxxx@xxxx-dp-dev01-879 ~]$ scp /tmp/ranger-0.5.4-SNAPSHOT-hdfs-plugin.tar.gz 172.17.8.93:/home/xxxxxxx/
xxxxxxx@172.17.8.93's password:
ranger-0.5.4-SNAPSHOT-hdfs-plugin.tar.gz 100% 18MB 42.7MB/s 00:00
[xxxxxxx@xxxx-dp-dev01-879 ~]$ scp /tmp/ranger-0.5.4-SNAPSHOT-hdfs-plugin.tar.gz 172.17.8.85:/home/xxxxxxx/
xxxxxxx@172.17.8.85's password:
ranger-0.5.4-SNAPSHOT-hdfs-plugin.tar.gz 100% 18MB 72.1MB/s 00:00
[xxxxxxx@xxxx-dp-dev01-879 ~]$ scp /tmp/ranger-0.5.4-SNAPSHOT-hdfs-plugin.tar.gz 172.17.8.59:/home/xxxxxxx/
xxxxxxx@172.17.8.59's password:
ranger-0.5.4-SNAPSHOT-hdfs-plugin.tar.gz
登录172.17.8.93：
[xxxxxxx@xxxx-dp-dev02-893 ~]$ sudo su - root
Last login: Tue Oct 15 15:58:39 CST 2019 on pts/0
[root@xxxx-dp-dev02-893 ~]# cd /opt/cloudera/parcels/CDH/lib
[root@xxxx-dp-dev02-893 lib]# cp /home/xxxxxxx/ranger-0.5.4-SNAPSHOT-hdfs-plugin.tar.gz ./
[root@xxxx-dp-dev02-893 lib]# tar -xvf ranger-0.5.4-SNAPSHOT-hdfs-plugin.tar.gz
[root@xxxx-dp-dev02-893 lib]# ln -s ranger-0.5.4-SNAPSHOT-hdfs-plugin ranger-hdfs-plugin
[root@xxxx-dp-dev02-893 lib]# cd ranger-hdfs-plugin/
[root@xxxx-dp-dev02-893 ranger-hdfs-plugin]# ln -s ../hadoop/etc/hadoop/ ../hadoop/conf
[root@xxxx-dp-dev02-893 ranger-hdfs-plugin]# export HADOOP_HOME=/opt/cloudera/parcels/CDH/lib/hadoop
[root@xxxx-dp-dev02-893 ranger-hdfs-plugin]# export JAVA_HOME=/usr/java/jdk1.7.0_80
[root@xxxx-dp-dev02-893 ranger-hdfs-plugin]# ./enable-hdfs-plugin.sh
Custom user and group is available, using custom user and group.
+ Tue Oct 15 17:01:35 CST 2019 : hadoop: lib folder=/opt/cloudera/parcels/CDH/lib/hadoop/lib conf folder=/opt/cloudera/parcels/CDH/lib/hadoop/conf
+ Tue Oct 15 17:01:35 CST 2019 : Saving current config file: /opt/cloudera/parcels/CDH/lib/hadoop/conf/hdfs-site.xml to /opt/cloudera/parcels/CDH/lib/hadoop/conf/.hdfs-site.xml.20191015-170135 ...
+ Tue Oct 15 17:01:35 CST 2019 : Saving current config file: /opt/cloudera/parcels/CDH/lib/hadoop/conf/ranger-hdfs-audit.xml to /opt/cloudera/parcels/CDH/lib/hadoop/conf/.ranger-hdfs-audit.xml.20191015-170135 ...
+ Tue Oct 15 17:01:36 CST 2019 : Saving current config file: /opt/cloudera/parcels/CDH/lib/hadoop/conf/ranger-hdfs-security.xml to /opt/cloudera/parcels/CDH/lib/hadoop/conf/.ranger-hdfs-security.xml.20191015-170135 ...
+ Tue Oct 15 17:01:36 CST 2019 : Saving current config file: /opt/cloudera/parcels/CDH/lib/hadoop/conf/ranger-policymgr-ssl.xml to /opt/cloudera/parcels/CDH/lib/hadoop/conf/.ranger-policymgr-ssl.xml.20191015-170135 ...
+ Tue Oct 15 17:01:37 CST 2019 : Saving current JCE file: /etc/ranger/hadoopdev/cred.jceks to /etc/ranger/hadoopdev/.cred.jceks.20191015170137 ...
+ Tue Oct 15 17:01:38 CST 2019 : Saving current JCE file: /etc/ranger/hadoopdev/cred.jceks to /etc/ranger/hadoopdev/.cred.jceks.20191015170138 ...
Ranger Plugin for hadoop has been enabled. Please restart hadoop to ensure that changes are effective.
[root@xxxx-dp-dev02-893 ranger-hdfs-plugin]# vim /usr/lib64/cmf/service/common/cdh-default-hadoop
[root@xxxx-dp-dev02-893 ranger-hdfs-plugin]# cat /usr/lib64/cmf/service/common/cdh-default-hadoop | grep "cp"
cp /etc/hadoop/conf/ranger-hdfs-audit.xml $CONF_DIR/
cp /etc/hadoop/conf/ranger-hdfs-security.xml $CONF_DIR/
cp /etc/hadoop/conf/ranger-policymgr-ssl.xml $CONF_DIR/
cp /etc/hadoop/conf/ranger-security.xml $CONF_DIR/
同样操作：
172.17.8.85
[root@xxxx-dp-dev03-885 ~]# sudo su - root
sudo: sudoers specifies that root is not allowed to sudo
[root@xxxx-dp-dev03-885 ~]# cd /opt/cloudera/parcels/CDH/lib
[root@xxxx-dp-dev03-885 lib]# cp /home/xxxxxxx/ranger-0.5.4-SNAPSHOT-hdfs-plugin.tar.gz ./
[root@xxxx-dp-dev03-885 lib]# tar -xvf ranger-0.5.4-SNAPSHOT-hdfs-plugin.tar.gz
[root@xxxx-dp-dev03-885 lib]# ln -s ranger-0.5.4-SNAPSHOT-hdfs-plugin ranger-hdfs-plugin
[root@xxxx-dp-dev03-885 lib]# cd ranger-hdfs-plugin/
[root@xxxx-dp-dev03-885 ranger-hdfs-plugin]# ln -s ../hadoop/etc/hadoop/ ../hadoop/conf
[root@xxxx-dp-dev03-885 ranger-hdfs-plugin]# export HADOOP_HOME=/opt/cloudera/parcels/CDH/lib/hadoop
[root@xxxx-dp-dev03-885 ranger-hdfs-plugin]# export JAVA_HOME=/usr/java/jdk1.7.0_80
[root@xxxx-dp-dev03-885 ranger-hdfs-plugin]# ./enable-hdfs-plugin.sh
Custom user and group is available, using custom user and group.
+ Tue Oct 15 17:09:25 CST 2019 : hadoop: lib folder=/opt/cloudera/parcels/CDH/lib/hadoop/lib conf folder=/opt/cloudera/parcels/CDH/lib/hadoop/conf
+ Tue Oct 15 17:09:25 CST 2019 : Saving current config file: /opt/cloudera/parcels/CDH/lib/hadoop/conf/hdfs-site.xml to /opt/cloudera/parcels/CDH/lib/hadoop/conf/.hdfs-site.xml.20191015-170925 ...
+ Tue Oct 15 17:09:26 CST 2019 : Saving current config file: /opt/cloudera/parcels/CDH/lib/hadoop/conf/ranger-hdfs-audit.xml to /opt/cloudera/parcels/CDH/lib/hadoop/conf/.ranger-hdfs-audit.xml.20191015-170925 ...
+ Tue Oct 15 17:09:26 CST 2019 : Saving current config file: /opt/cloudera/parcels/CDH/lib/hadoop/conf/ranger-hdfs-security.xml to /opt/cloudera/parcels/CDH/lib/hadoop/conf/.ranger-hdfs-security.xml.20191015-170925 ...
+ Tue Oct 15 17:09:26 CST 2019 : Saving current config file: /opt/cloudera/parcels/CDH/lib/hadoop/conf/ranger-policymgr-ssl.xml to /opt/cloudera/parcels/CDH/lib/hadoop/conf/.ranger-policymgr-ssl.xml.20191015-170925 ...
+ Tue Oct 15 17:09:27 CST 2019 : Saving current JCE file: /etc/ranger/hadoopdev/cred.jceks to /etc/ranger/hadoopdev/.cred.jceks.20191015170927 ...
+ Tue Oct 15 17:09:28 CST 2019 : Saving current JCE file: /etc/ranger/hadoopdev/cred.jceks to /etc/ranger/hadoopdev/.cred.jceks.20191015170928 ...
Ranger Plugin for hadoop has been enabled. Please restart hadoop to ensure that changes are effective.
[root@xxxx-dp-dev03-885 ranger-hdfs-plugin]# vim /usr/lib64/cmf/service/common/cdh-default-hadoop
[root@xxxx-dp-dev03-885 ranger-hdfs-plugin]# cat /usr/lib64/cmf/service/common/cdh-default-hadoop | grep "cp"
cp /etc/hadoop/conf/ranger-hdfs-audit.xml $CONF_DIR/
cp /etc/hadoop/conf/ranger-hdfs-security.xml $CONF_DIR/
cp /etc/hadoop/conf/ranger-policymgr-ssl.xml $CONF_DIR/
cp /etc/hadoop/conf/ranger-security.xml $CONF_DIR/
同样操作：
172.17.8.59
[xxxxxxx@xxxx-dp-dev04-859 ~]$ sudo su - root
Last login: Tue Oct 15 16:01:38 CST 2019 on pts/0
[root@xxxx-dp-dev04-859 ~]# cd /opt/cloudera/parcels/CDH/lib
[root@xxxx-dp-dev04-859 lib]# cp /home/xxxxxxx/ranger-0.5.4-SNAPSHOT-hdfs-plugin.tar.gz ./
[root@xxxx-dp-dev04-859 lib]# tar -xvf ranger-0.5.4-SNAPSHOT-hdfs-plugin.tar.gz
[root@xxxx-dp-dev04-859 lib]# ln -s ranger-0.5.4-SNAPSHOT-hdfs-plugin ranger-hdfs-plugin
[root@xxxx-dp-dev04-859 lib]# cd ranger-hdfs-plugin/
[root@xxxx-dp-dev04-859 ranger-hdfs-plugin]# export HADOOP_HOME=/opt/cloudera/parcels/CDH/lib/hadoop
[root@xxxx-dp-dev04-859 ranger-hdfs-plugin]# export JAVA_HOME=/usr/java/jdk1.7.0_80
[root@xxxx-dp-dev04-859 ranger-hdfs-plugin]# ./enable-hdfs-plugin.sh
Custom user and group is available, using custom user and group.
+ Tue Oct 15 17:13:01 CST 2019 : hadoop: lib folder=/opt/cloudera/parcels/CDH/lib/hadoop/lib conf folder=/opt/cloudera/parcels/CDH/lib/hadoop/conf
+ Tue Oct 15 17:13:01 CST 2019 : Saving current config file: /opt/cloudera/parcels/CDH/lib/hadoop/conf/hdfs-site.xml to /opt/cloudera/parcels/CDH/lib/hadoop/conf/.hdfs-site.xml.20191015-171301 ...
+ Tue Oct 15 17:13:02 CST 2019 : Saving current config file: /opt/cloudera/parcels/CDH/lib/hadoop/conf/ranger-hdfs-audit.xml to /opt/cloudera/parcels/CDH/lib/hadoop/conf/.ranger-hdfs-audit.xml.20191015-171301 ...
+ Tue Oct 15 17:13:02 CST 2019 : Saving current config file: /opt/cloudera/parcels/CDH/lib/hadoop/conf/ranger-hdfs-security.xml to /opt/cloudera/parcels/CDH/lib/hadoop/conf/.ranger-hdfs-security.xml.20191015-171301 ...
+ Tue Oct 15 17:13:02 CST 2019 : Saving current config file: /opt/cloudera/parcels/CDH/lib/hadoop/conf/ranger-policymgr-ssl.xml to /opt/cloudera/parcels/CDH/lib/hadoop/conf/.ranger-policymgr-ssl.xml.20191015-171301 ...
+ Tue Oct 15 17:13:04 CST 2019 : Saving current JCE file: /etc/ranger/hadoopdev/cred.jceks to /etc/ranger/hadoopdev/.cred.jceks.20191015171304 ...
+ Tue Oct 15 17:13:05 CST 2019 : Saving current JCE file: /etc/ranger/hadoopdev/cred.jceks to /etc/ranger/hadoopdev/.cred.jceks.20191015171305 ...
Ranger Plugin for hadoop has been enabled. Please restart hadoop to ensure that changes are effective.
[root@xxxx-dp-dev04-859 ranger-hdfs-plugin]# vim /usr/lib64/cmf/service/common/cdh-default-hadoop
[root@xxxx-dp-dev04-859 ranger-hdfs-plugin]# cat /usr/lib64/cmf/service/common/cdh-default-hadoop | grep "cp"
cp /etc/hadoop/conf/ranger-hdfs-audit.xml $CONF_DIR/
cp /etc/hadoop/conf/ranger-hdfs-security.xml $CONF_DIR/
cp /etc/hadoop/conf/ranger-policymgr-ssl.xml $CONF_DIR/
cp /etc/hadoop/conf/ranger-security.xml $CONF_DIR/
注意Service Name与HDFS Plugin配置要一致：
下面配置应该只对测试连接有用，权限策略实际是缓存namenode本地：
添加HDFS策略：
成功后：
安装Hive Plugin（高可用存在3个实例）：
172.17.8.85
172.17.8.93
172.17.8.59
172.17.8.85:
[xxxxxxx@xxxx-dp-dev03-885 local]$ sudo su - root
Last login: Mon Oct 14 13:44:49 CST 2019 on pts/1
[root@xxxx-dp-dev03-885 local]# cd /opt/cloudera/parcels/CDH/lib
[root@xxxx-dp-dev03-885 local]# cp /home/xxxxxxx/ranger-0.5.4-SNAPSHOT-hive-plugin.tar.gz ./
[root@xxxx-dp-dev03-885 lib]# ln -s ranger-0.5.4-SNAPSHOT-hive-plugin ranger-hive-plugin
[root@xxxx-dp-dev03-885 local]# cd ranger-hive-plugin/
[root@xxxx-dp-dev03-885 ranger-hive-plugin]# vim install.properties
[root@xxxx-dp-dev03-885 ranger-hive-plugin]# cat install.properties | grep -v "^#" | grep -v "^$"
POLICY_MGR_URL=http://172.17.8.94:6080
REPOSITORY_NAME=hivedev
XAAUDIT.SOLR.ENABLE=true
XAAUDIT.SOLR.URL=http://172.17.8.4:6083/solr/ranger_audits
XAAUDIT.SOLR.FILE_SPOOL_DIR=/var/log/hive/audit/solr/spool
CUSTOM_USER=hive
CUSTOM_GROUP=hive
[root@xxxx-dp-dev03-885 ranger-hive-plugin]# export JAVA_HOME=/usr/java/jdk1.7.0_80
[root@xxxx-dp-dev03-885 ranger-hive-plugin]# ./enable-hive-plugin.sh
Custom user and group is available, using custom user and group.
+ Mon Oct 14 14:36:28 CST 2019 : hive: lib folder=/opt/cloudera/parcels/CDH/lib/hive/lib conf folder=/opt/cloudera/parcels/CDH/lib/hive/conf
+ Mon Oct 14 14:36:28 CST 2019 : Saving /opt/cloudera/parcels/CDH/lib/hive/conf/ranger-hive-audit.xml to /opt/cloudera/parcels/CDH/lib/hive/conf/.ranger-hive-audit.xml.20191014-143628 ...
+ Mon Oct 14 14:36:28 CST 2019 : Saving /opt/cloudera/parcels/CDH/lib/hive/conf/ranger-hive-security.xml to /opt/cloudera/parcels/CDH/lib/hive/conf/.ranger-hive-security.xml.20191014-143628 ...
+ Mon Oct 14 14:36:28 CST 2019 : Saving /opt/cloudera/parcels/CDH/lib/hive/conf/ranger-policymgr-ssl.xml to /opt/cloudera/parcels/CDH/lib/hive/conf/.ranger-policymgr-ssl.xml.20191014-143628 ...
+ Mon Oct 14 14:36:28 CST 2019 : Saving current config file: /opt/cloudera/parcels/CDH/lib/hive/conf/hiveserver2-site.xml to /opt/cloudera/parcels/CDH/lib/hive/conf/.hiveserver2-site.xml.20191014-143628 ...
+ Mon Oct 14 14:36:28 CST 2019 : Saving current config file: /opt/cloudera/parcels/CDH/lib/hive/conf/ranger-hive-audit.xml to /opt/cloudera/parcels/CDH/lib/hive/conf/.ranger-hive-audit.xml.20191014-143628 ...
+ Mon Oct 14 14:36:28 CST 2019 : Saving current config file: /opt/cloudera/parcels/CDH/lib/hive/conf/ranger-hive-security.xml to /opt/cloudera/parcels/CDH/lib/hive/conf/.ranger-hive-security.xml.20191014-143628 ...
+ Mon Oct 14 14:36:29 CST 2019 : Saving current config file: /opt/cloudera/parcels/CDH/lib/hive/conf/ranger-policymgr-ssl.xml to /opt/cloudera/parcels/CDH/lib/hive/conf/.ranger-policymgr-ssl.xml.20191014-143628 ...
Unable to store password in non-plain text format. Error: [SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Oct 14, 2019 2:36:29 PM org.apache.hadoop.util.NativeCodeLoader <clinit>
WARNING: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/htrace/core/Tracer$Builder
at org.apache.hadoop.fs.FsTracer.get(FsTracer.java:42)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2696)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:99)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2761)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2743)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:387)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
at org.apache.hadoop.security.alias.JavaKeyStoreProvider.initFileSystem(JavaKeyStoreProvider.java:89)
at org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.<init>(AbstractJavaKeyStoreProvider.java:85)
at org.apache.hadoop.security.alias.JavaKeyStoreProvider.<init>(JavaKeyStoreProvider.java:49)
at org.apache.hadoop.security.alias.JavaKeyStoreProvider.<init>(JavaKeyStoreProvider.java:41)
at org.apache.hadoop.security.alias.JavaKeyStoreProvider$Factory.createProvider(JavaKeyStoreProvider.java:100)
at org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:58)
at org.apache.ranger.credentialapi.CredentialReader.getDecryptedString(CredentialReader.java:58)
at org.apache.ranger.credentialapi.buildks.createCredential(buildks.java:86)
at org.apache.ranger.credentialapi.buildks.main(buildks.java:39)
Caused by: java.lang.ClassNotFoundException: org.apache.htrace.core.Tracer$Builder
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 16 more]
Exiting plugin installation
[root@xxxx-dp-dev03-885 ranger-hive-plugin]# cp /opt/cloudera/parcels/CDH-5.8.4-1.cdh5.8.4.p0.5/jars/htrace-core4-4.0.1-incubating.jar install/lib/
[root@xxxx-dp-dev03-885 ranger-hive-plugin]# ./enable-hive-plugin.sh
Custom user and group is available, using custom user and group.
+ Mon Oct 14 14:39:29 CST 2019 : hive: lib folder=/opt/cloudera/parcels/CDH/lib/hive/lib conf folder=/opt/cloudera/parcels/CDH/lib/hive/conf
+ Mon Oct 14 14:39:29 CST 2019 : Saving /opt/cloudera/parcels/CDH/lib/hive/conf/ranger-hive-audit.xml to /opt/cloudera/parcels/CDH/lib/hive/conf/.ranger-hive-audit.xml.20191014-143929 ...
+ Mon Oct 14 14:39:29 CST 2019 : Saving /opt/cloudera/parcels/CDH/lib/hive/conf/ranger-hive-security.xml to /opt/cloudera/parcels/CDH/lib/hive/conf/.ranger-hive-security.xml.20191014-143929 ...
+ Mon Oct 14 14:39:29 CST 2019 : Saving /opt/cloudera/parcels/CDH/lib/hive/conf/ranger-policymgr-ssl.xml to /opt/cloudera/parcels/CDH/lib/hive/conf/.ranger-policymgr-ssl.xml.20191014-143929 ...
+ Mon Oct 14 14:39:29 CST 2019 : Saving current config file: /opt/cloudera/parcels/CDH/lib/hive/conf/hiveserver2-site.xml to /opt/cloudera/parcels/CDH/lib/hive/conf/.hiveserver2-site.xml.20191014-143929 ...
+ Mon Oct 14 14:39:29 CST 2019 : Saving current config file: /opt/cloudera/parcels/CDH/lib/hive/conf/ranger-hive-audit.xml to /opt/cloudera/parcels/CDH/lib/hive/conf/.ranger-hive-audit.xml.20191014-143929 ...
+ Mon Oct 14 14:39:29 CST 2019 : Saving current config file: /opt/cloudera/parcels/CDH/lib/hive/conf/ranger-hive-security.xml to /opt/cloudera/parcels/CDH/lib/hive/conf/.ranger-hive-security.xml.20191014-143929 ...
+ Mon Oct 14 14:39:29 CST 2019 : Saving current config file: /opt/cloudera/parcels/CDH/lib/hive/conf/ranger-policymgr-ssl.xml to /opt/cloudera/parcels/CDH/lib/hive/conf/.ranger-policymgr-ssl.xml.20191014-143929 ...
+ Mon Oct 14 14:39:30 CST 2019 : Saving lib file: /opt/cloudera/parcels/CDH/lib/hive/lib/ranger-hive-plugin-shim-0.5.4-SNAPSHOT.jar to /opt/cloudera/parcels/CDH/lib/hive/lib/.ranger-hive-plugin-shim-0.5.4-SNAPSHOT.jar.20191014143930 ...
+ Mon Oct 14 14:39:30 CST 2019 : Saving lib file: /opt/cloudera/parcels/CDH/lib/hive/lib/ranger-plugin-classloader-0.5.4-SNAPSHOT.jar to /opt/cloudera/parcels/CDH/lib/hive/lib/.ranger-plugin-classloader-0.5.4-SNAPSHOT.jar.20191014143930 ...
+ Mon Oct 14 14:39:31 CST 2019 : Saving current JCE file: /etc/ranger/hivedev/cred.jceks to /etc/ranger/hivedev/.cred.jceks.20191014143931 ...
+ Mon Oct 14 14:39:32 CST 2019 : Saving current JCE file: /etc/ranger/hivedev/cred.jceks to /etc/ranger/hivedev/.cred.jceks.20191014143932 ...
Ranger Plugin for hive has been enabled. Please restart hive to ensure that changes are effective.
添加下面三行cp配置文件命令：
[xxxxxxx@xxxx-dp-dev03-885 ~]$ sudo vim /usr/lib64/cmf/service/common/cloudera-config.sh
# When created, the final resting place of config files is unknown,
# so it is marked as {{CMF_CONF_DIR}}. We know what this directory will
# be here in this script, so search-replace it.
replace_conf_dir() {
find $CONF_DIR -type f ! -path "$CONF_DIR/logs/*" ! -name "*.log" ! -name "*.keytab" ! -name "*jceks" -exec perl -pi -e "s#{{CMF_CONF_DIR}}#$CONF_DIR#g" {} \;
cp /etc/hive/conf/ranger-hive-audit.xml $CONF_DIR/
cp /etc/hive/conf/ranger-hive-security.xml $CONF_DIR/
cp /etc/hive/conf/ranger-policymgr-ssl.xml $CONF_DIR/
}
Ranger Admin页面添加服务：
注意Hive服务名为：hivedev
172.17.8.93：
[xxxxxxx@xxxx-dp-dev03-885 ~]$ sudo su - root
[root@xxxx-dp-dev03-885 ~]# cd /opt/cloudera/parcels/CDH/lib
[root@xxxx-dp-dev03-885 lib]# tar -cvf ranger-0.5.4-SNAPSHOT-hive-plugin.tar.gz ranger-0.5.4-SNAPSHOT-hive-plugin/*
[root@xxxx-dp-dev03-885 lib]# chown xxxxxxx:users /tmp/ranger-0.5.4-SNAPSHOT-hive-plugin.tar.gz
[root@xxxx-dp-dev03-885 ~]# cd /etc/hive/conf
[root@xxxx-dp-dev03-885 conf]# mkdir /tmp/ranger-hive-conf
[root@xxxx-dp-dev03-885 conf]# cp ranger-hive-audit.xml ranger-hive-security.xml ranger-policymgr-ssl.xml /tmp/ranger-hive-conf/
[root@xxxx-dp-dev03-885 conf]# cd /tmp/
[root@xxxx-dp-dev03-885 tmp]# tar -cvf ranger-hive-conf.tar.gz ranger-hive-conf/
[root@xxxx-dp-dev03-885 tmp]# chown xxxxxxx:users ranger-hive-conf.tar.gz
[xxxxxxx@xxxx-dp-dev03-885 ~]$ scp /tmp/ranger-hive-conf.tar.gz /tmp/ranger-0.5.4-SNAPSHOT-hive-plugin.tar.gz 172.17.8.59:/tmp
xxxxxxx@172.17.8.59's password:
ranger-hive-conf.tar.gz 100% 20KB 13.7MB/s 00:00
ranger-0.5.4-SNAPSHOT-hive-plugin.tar.gz 100% 19MB 38.8MB/s 00:00
[xxxxxxx@xxxx-dp-dev03-885 ~]$ scp /tmp/ranger-hive-conf.tar.gz /tmp/ranger-0.5.4-SNAPSHOT-hive-plugin.tar.gz 172.17.8.93:/tmp
xxxxxxx@172.17.8.93's password:
ranger-hive-conf.tar.gz 100% 20KB 3.4MB/s 00:00
ranger-0.5.4-SNAPSHOT-hive-plugin.tar.gz
[root@xxxx-dp-dev02-893 ~]# cd /opt/cloudera/parcels/CDH/lib/
[root@xxxx-dp-dev02-893 lib]# mv /tmp/ranger-hive-conf.tar.gz ./
[root@xxxx-dp-dev02-893 lib]# chown root:root ranger-hive-conf.tar.gz
[root@xxxx-dp-dev02-893 lib]# tar -xvf ranger-hive-conf.tar.gz
[root@xxxx-dp-dev02-893 lib]# mv /tmp/ranger-0.5.4-SNAPSHOT-hive-plugin.tar.gz ./
[root@xxxx-dp-dev02-893 lib]# chown root:root ranger-0.5.4-SNAPSHOT-hive-plugin.tar.gz
[root@xxxx-dp-dev02-893 lib]# tar -xvf ranger-0.5.4-SNAPSHOT-hive-plugin.tar.gz
[root@xxxx-dp-dev02-893 lib]# ln -s ranger-0.5.4-SNAPSHOT-hive-plugin ranger-hive-plugin
[root@xxxx-dp-dev02-893 lib]# cp ranger-hive-conf/* /etc/hive/conf/
[root@xxxx-dp-dev02-893 lib]# ls -al /etc/hive/conf
lrwxrwxrwx 1 root root 27 Jul 26 14:39 /etc/hive/conf -> /etc/alternatives/hive-conf
[root@xxxx-dp-dev02-893 lib]# ls -al /etc/alternatives/hive-conf/
total 92
drwxr-xr-x 2 root root 4096 Oct 15 10:48 .
drwxr-xr-x 3 root root 44 Jul 26 14:39 ..
-rwxr--r-- 1 root root 9568 Oct 15 10:48 ranger-hive-audit.xml
-rwxr--r-- 1 root root 2905 Oct 15 10:48 ranger-hive-security.xml
-rwxr--r-- 1 root root 2266 Oct 15 10:48 ranger-policymgr-ssl.xml
添加下面三行cp配置文件命令：
[root@xxxx-dp-dev02-893 conf]# vim /usr/lib64/cmf/service/common/cloudera-config.sh
# When created, the final resting place of config files is unknown,
# so it is marked as {{CMF_CONF_DIR}}. We know what this directory will
# be here in this script, so search-replace it.
replace_conf_dir() {
find $CONF_DIR -type f ! -path "$CONF_DIR/logs/*" ! -name "*.log" ! -name "*.keytab" ! -name "*jceks" -exec perl -pi -e "s#{{CMF_CONF_DIR}}#$CONF_DIR#g" {} \;
cp /etc/hive/conf/ranger-hive-audit.xml $CONF_DIR/
cp /etc/hive/conf/ranger-hive-security.xml $CONF_DIR/
cp /etc/hive/conf/ranger-policymgr-ssl.xml $CONF_DIR/
}
[root@xxxx-dp-dev02-893 conf]# cd /opt/cloudera/parcels/CDH/lib
[root@xxxx-dp-dev02-893 lib]# cd ranger-hive-plugin/
[root@xxxx-dp-dev02-893 ranger-hive-plugin]# export JAVA_HOME=/usr/java/jdk1.7.0_80
[root@xxxx-dp-dev02-893 ranger-hive-plugin]# ./enable-hive-plugin.sh
同样配置：
172.17.8.59:
[xxxxxxx@xxxx-dp-dev04-859 ~]$ sudo su - root
Last login: Wed Jun 26 14:06:27 CST 2019
[root@xxxx-dp-dev04-859 ~]# cd /opt/cloudera/parcels/CDH/lib
[root@xxxx-dp-dev04-859 lib]# mv /tmp/ranger-hive-conf.tar.gz ./
[root@xxxx-dp-dev04-859 lib]# mv /tmp/ranger-0.5.4-SNAPSHOT-hive-plugin.tar.gz ./
[root@xxxx-dp-dev04-859 lib]# chown root:root ranger-hive-conf.tar.gz
[root@xxxx-dp-dev04-859 lib]# chown root:root ranger-0.5.4-SNAPSHOT-hive-plugin.tar.gz
[root@xxxx-dp-dev04-859 lib]# tar -xvf ranger-hive-conf.tar.gz
[root@xxxx-dp-dev04-859 lib]# tar -xvf ranger-0.5.4-SNAPSHOT-hive-plugin.tar.gz
[root@xxxx-dp-dev04-859 lib]# ln -s ranger-0.5.4-SNAPSHOT-hive-plugin ranger-hive-plugin
[root@xxxx-dp-dev04-859 lib]# cp ranger-hive-conf/* /etc/hive/conf/
[root@xxxx-dp-dev04-859 lib]# vim /usr/lib64/cmf/service/common/cloudera-config.sh
# When created, the final resting place of config files is unknown,
# so it is marked as {{CMF_CONF_DIR}}. We know what this directory will
# be here in this script, so search-replace it.
replace_conf_dir() {
find $CONF_DIR -type f ! -path "$CONF_DIR/logs/*" ! -name "*.log" ! -name "*.keytab" ! -name "*jceks" -exec perl -pi -e "s#{{CMF_CONF_DIR}}#$CONF_DIR#g" {} \;
cp /etc/hive/conf/ranger-hive-audit.xml $CONF_DIR/
cp /etc/hive/conf/ranger-hive-security.xml $CONF_DIR/
cp /etc/hive/conf/ranger-policymgr-ssl.xml $CONF_DIR/
}
[root@xxxx-dp-dev04-859 lib]# cd /opt/cloudera/parcels/CDH/lib
[root@xxxx-dp-dev04-859 lib]# cd ranger-hive-plugin/
[root@xxxx-dp-dev04-859 ranger-hive-plugin]# export JAVA_HOME=/usr/java/jdk1.7.0_80
[root@xxxx-dp-dev04-859 ranger-hive-plugin]# ./enable-hive-plugin.sh
Cloudera Manger Hive服务页面添加下列参数：
<property>
<name>hive.security.authorization.enabled</name>
<value>true</value>
</property>
<property>
<name>hive.security.authorization.manager</name>
<value>org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizerFactory</value>
</property>
<property>
<name>hive.security.authenticator.manager</name>
<value>org.apache.hadoop.hive.ql.security.SessionStateUserAuthenticator</value>
</property>
<property>
<name>hive.conf.restricted.list</name>
<value>hive.security.authorization.enabled,hive.security.authorization.manager,hive.security.authenticator.manager</value>
</property>
重启HiveServer2服务，确认日志是否报错！
Ranger Admin添加服务(注意Servcie Name也要与Hive Plugin配置一致)：
添加策略：
验证策略（所验证策略并不是上图所配置策略）：
[xxxxxxx@xxxx-dp-dev12-8108 ~]$ beeline
0: jdbc:hive2://xxxx-dp-dev01:2181,xxxx-dp-de> use xxxxx;
Error: Error while compiling statement: FAILED: HiveAccessControlException Permission denied: user [xxxxxxx] does not have [USE] privilege on [xxxxx] (state=42000,code=40000)
0: jdbc:hive2://xxxx-dp-dev01:2181,xxxx-dp-de> select * from xxxxx_account limit 30;
Error: Error while compiling statement: FAILED: HiveAccessControlException Permission denied: user [xxxxxxx] does not have [SELECT] privilege on [xxxxx/xxxxx_account/account_buyer_level,account_contact,account_country,account_id,account_seller_level] (state=42000,code=40000)
0: jdbc:hive2://xxxx-dp-dev01:2181,xxxx-dp-de> select account_id from xxxxx_account limit 30;
INFO : Compiling command(queryId=hive_20191015131414_3b6bff09-5c50-4817-98ff-41857abf6df0): select account_id from xxxxx_account limit 30
INFO : UserName: xxxxxxx
INFO : Semantic Analysis Completed
INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:account_id, type:bigint, comment:null)], properties:null)
INFO : Completed compiling command(queryId=hive_20191015131414_3b6bff09-5c50-4817-98ff-41857abf6df0); Time taken: 7.205 seconds
INFO : Executing command(queryId=hive_20191015131414_3b6bff09-5c50-4817-98ff-41857abf6df0): select account_id from xxxxx_account limit 30
INFO : Completed executing command(queryId=hive_20191015131414_3b6bff09-5c50-4817-98ff-41857abf6df0); Time taken: 0.003 seconds
INFO : OK
10000000
10000001
10000002
10000003
10000004
10000005
Create, Alter权限分离：
0: jdbc:hive2://172.17.8.85:10000/default> create table test(a string);
Error: Error while compiling statement: FAILED: HiveAccessControlException Permission denied: user [xxxxxxx] does not have [CREATE] privilege on [edw/test] (state=42000,code=40000)
0: jdbc:hive2://xxxx-dp-dev01:2181,xxxx-dp-de> create table test_partition(a string) partitioned by(data_date bigint);
0: jdbc:hive2://xxxx-dp-dev01:2181,xxxx-dp-de> alter table test_partition add partition(data_date=20191015);
Error: Error while compiling statement: FAILED: HiveAccessControlException Permission denied: user [xxxxxxx] does not have [ALTER] privilege on [edw/test_partition] (state=42000,code=40000)
添加Alter权限，回收Create权限:
0: jdbc:hive2://xxxx-dp-dev01:2181,xxxx-dp-de> alter table test_partition add partition(data_date=20191015);
Error: Error while compiling statement: FAILED: HiveAccessControlException Permission denied: user [xxxxxxx] does not have [ALTER] privilege on [edw/test_partition] (state=42000,code=40000)
0: jdbc:hive2://xxxx-dp-dev01:2181,xxxx-dp-de> alter table test_partition add partition(data_date=20191015);
INFO : Compiling command(queryId=hive_20191015133737_9b53e08a-18fd-48fc-b52a-8538785b2851): alter table test_partition add partition(data_date=20191015)
INFO : UserName: xxxxxxx
INFO : Semantic Analysis Completed
INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
INFO : Completed compiling command(queryId=hive_20191015133737_9b53e08a-18fd-48fc-b52a-8538785b2851); Time taken: 5.942 seconds
INFO : Executing command(queryId=hive_20191015133737_9b53e08a-18fd-48fc-b52a-8538785b2851): alter table test_partition add partition(data_date=20191015)
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing command(queryId=hive_20191015133737_9b53e08a-18fd-48fc-b52a-8538785b2851); Time taken: 0.322 seconds
INFO : OK
No rows affected (6.347 seconds)
不需要Create权限，也可以drop分区：
Error: Error while compiling statement: FAILED: HiveAccessControlException Permission denied: user [xxxxxxx] does not have [CREATE] privilege on [edw/test_b] (state=42000,code=40000)
0: jdbc:hive2://xxxx-dp-dev01:2181,xxxx-dp-de> alter table test_partition drop partition(data_date=20191015);
INFO : Compiling command(queryId=hive_20191015134141_ad9bf01b-6930-49fb-8d1d-58559b8c1955): alter table test_partition drop partition(data_date=20191015)
INFO : UserName: xxxxxxx
INFO : Semantic Analysis Completed
INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
INFO : Completed compiling command(queryId=hive_20191015134141_ad9bf01b-6930-49fb-8d1d-58559b8c1955); Time taken: 0.593 seconds
INFO : Executing command(queryId=hive_20191015134141_ad9bf01b-6930-49fb-8d1d-58559b8c1955): alter table test_partition drop partition(data_date=20191015)
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Dropped the partition data_date=20191015
INFO : Completed executing command(queryId=hive_20191015134141_ad9bf01b-6930-49fb-8d1d-58559b8c1955); Time taken: 0.699 seconds
INFO : OK
No rows affected (1.375 seconds)
Hive没有查询权限，Hdfs依然有权限：
0: jdbc:hive2://xxxx-dp-dev01:2181,xxxx-dp-de> select * from test;
Error: Error while compiling statement: FAILED: HiveAccessControlException Permission denied: user [xxxxxxx] does not have [SELECT] privilege on [edw/test/a] (state=42000,code=40000)
[xxxxxxx@xxxx-dp-dev12-8108 ~]$ hdfs dfs -get /user/hive/warehouse/edw.db/test/000000_0
[xxxxxxx@xxxx-dp-dev12-8108 ~]$ parquet-tools
No command specified
[xxxxxxx@xxxx-dp-dev12-8108 ~]$ parquet-tools cat 000000_0
扫一扫
682
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
