Ranger 部署教程

最新推荐文章于 2024-09-27 23:21:46 发布
最新推荐文章于 2024-09-27 23:21:46 发布
阅读量758 收藏 10
点赞数 20
分类专栏： 开源大数据平台 DataLight 文章标签： 开源 大数据 java ambari cloudera
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/Boundivore/article/details/142552945
版权

本项目示例过程将以 Ranger 2.4.0 版本为例。

一、下载源码并编译

编译具体过程请参考 Ranger 源码编译文档。

1.1 Ranger 目录清单

编译后，可在对应编译目录下得到如下文件。

[root@node01 RANGER]# ll
total 445844
-rw-r--r-- 1 root root 391066220 Sep 15 16:23 ranger-2.4.0-admin.tar.gz
-rw-r--r-- 1 root root   2144079 Sep 15 16:23 ranger-2.4.0-atlas-plugin.tar.gz
-rw-r--r-- 1 root root   3415231 Sep 15 16:23 ranger-2.4.0-elasticsearch-plugin.tar.gz
-rw-r--r-- 1 root root   2222091 Sep 15 16:23 ranger-2.4.0-hbase-plugin.tar.gz
-rw-r--r-- 1 root root   2426862 Sep 15 16:23 ranger-2.4.0-hdfs-plugin.tar.gz
-rw-r--r-- 1 root root   2225030 Sep 15 16:23 ranger-2.4.0-hive-plugin.tar.gz
-rw-r--r-- 1 root root   4163870 Sep 15 16:23 ranger-2.4.0-kafka-plugin.tar.gz
-rw-r--r-- 1 root root   4574569 Sep 15 16:23 ranger-2.4.0-kms.tar.gz
-rw-r--r-- 1 root root   2165369 Sep 15 16:23 ranger-2.4.0-knox-plugin.tar.gz
-rw-r--r-- 1 root root   2129157 Sep 15 16:23 ranger-2.4.0-kylin-plugin.tar.gz
-rw-r--r-- 1 root root     34196 Sep 15 16:23 ranger-2.4.0-migration-util.tar.gz
-rw-r--r-- 1 root root   2399366 Sep 15 16:23 ranger-2.4.0-ozone-plugin.tar.gz
-rw-r--r-- 1 root root   2976268 Sep 15 16:23 ranger-2.4.0-presto-plugin.tar.gz
-rw-r--r-- 1 root root    988484 Sep 15 16:23 ranger-2.4.0-ranger-tools.tar.gz
-rw-r--r-- 1 root root     36907 Sep 15 16:23 ranger-2.4.0-solr_audit_conf.tar.gz
-rw-r--r-- 1 root root   2188133 Sep 15 16:23 ranger-2.4.0-solr-plugin.tar.gz
-rw-r--r-- 1 root root   2379177 Sep 15 16:23 ranger-2.4.0-sqoop-plugin.tar.gz
-rw-r--r-- 1 root root   6376424 Sep 15 16:23 ranger-2.4.0-src.tar.gz
-rw-r--r-- 1 root root   2919373 Sep 15 16:23 ranger-2.4.0-storm-plugin.tar.gz
-rw-r--r-- 1 root root    927274 Sep 15 16:23 ranger-2.4.0-tagsync.tar.gz
-rw-r--r-- 1 root root   1275674 Sep 15 16:23 ranger-2.4.0-usersync.tar.gz
-rw-r--r-- 1 root root   2655161 Sep 15 16:23 ranger-2.4.0-yarn-plugin.tar.gz
-rw-r--r-- 1 root root     12985 Sep 15 16:23 ranger-examples-2.4.0-sampleapp-plugin.tar.gz
-rw-r--r-- 1 root root      9641 Sep 15 16:23 ranger-examples-2.4.0-sampleapp.tar.gz
-rw-r--r-- 1 root root  14780220 Sep 15 16:23 ranger-examples-2.4.0-sample-client.tar.gz

1.2 准备环境依赖

该示例中，将假设用户已经部署完成如下服务。

服务名称版本号
MySQL5.7（或 8+，自备驱动）
Hadoop3.2.4
HBase2.5.9
Hive3.1.3

确保当前节点中已经安装了 python3

yum -y install python3

二、部署 Ranger Admin

2.1 解压 Ranger Admin

cd /srv/datalight/RANGER

tar -zxf ranger-2.4.0-admin.tar.gz -C /srv/datalight/RANGER

2.2 初始化必要目录

mkdir -p /data/datalight/pids/RANGER
mkdir -p /data/datalight/logs/RANGER

chown datalight:datalight -R /data/datalight/pids/RANGER
chown datalight:datalight -R /data/datalight/logs/RANGER

2.3 修改配置文件

2.3.3 install.properties

  • 路径

    /srv/datalight/RANGER/ranger-2.4.0-admin/install.properties

  • 内容

    # Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

#
# This file provides a list of the deployment variables for the Policy Manager Web Application
#

#------------------------- DB CONFIG - BEGIN ----------------------------------
# Uncomment the below if the DBA steps need to be run separately
#setup_mode=SeparateDBA

PYTHON_COMMAND_INVOKER=python3

#DB_FLAVOR=MYSQL|ORACLE|POSTGRES|MSSQL|SQLA
DB_FLAVOR=MYSQL
#

#
# Location of DB client library (please check the location of the jar file)
#
#SQL_CONNECTOR_JAR=/usr/share/java/ojdbc6.jar
#SQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-java.jar
#SQL_CONNECTOR_JAR=/usr/share/java/postgresql.jar
#SQL_CONNECTOR_JAR=/usr/share/java/sqljdbc4.jar
#SQL_CONNECTOR_JAR=/opt/sqlanywhere17/java/sajdbc4.jar
SQL_CONNECTOR_JAR=/srv/datalight/RANGER/ranger-2.4.0-admin/db/mysql/mysql-connector-java-5.1.47.jar


#
# DB password for the DB admin user-id
# **************************************************************************
# ** If the password is left empty or not-defined here,
# ** it will try with blank password during installation process
# **************************************************************************
#
#db_root_user=root|SYS|postgres|sa|dba
#db_host=host:port              # for DB_FLAVOR=MYSQL|POSTGRES|SQLA|MSSQL       #for example: db_host=localhost:3306
#db_host=host:port:SID          # for DB_FLAVOR=ORACLE                          #for SID example: db_host=localhost:1521:ORCL
#db_host=host:port/ServiceName  # for DB_FLAVOR=ORACLE                          #for Service example: db_host=localhost:1521/XE
db_root_user=root
db_root_password=1qaz!QAZ
db_host=node01
#SSL config
db_ssl_enabled=false
db_ssl_required=false
db_ssl_verifyServerCertificate=false
#db_ssl_auth_type=1-way|2-way, where 1-way represents standard one way ssl authentication and 2-way represents mutual ssl authentication
db_ssl_auth_type=2-way
javax_net_ssl_keyStore=
javax_net_ssl_keyStorePassword=
javax_net_ssl_trustStore=
javax_net_ssl_trustStorePassword=
javax_net_ssl_trustStore_type=jks
javax_net_ssl_keyStore_type=jks

# For postgresql db
db_ssl_certificate_file=

#
# DB UserId used for the Ranger schema
#
db_name=db_ranger
db_user=root
db_password=1qaz!QAZ

#For over-riding the jdbc url.
is_override_db_connection_string=true
db_override_connection_string=jdbc:mysql://node01:3306/db_ranger?useSSL=false&createDatabaseIfNotExist=true


# change password. Password for below mentioned users can be changed only once using this property.
#PLEASE NOTE :: Password should be minimum 8 characters with min one alphabet and one numeric.
rangerAdmin_password=Datalight123!
rangerTagsync_password=Datalight123!
rangerUsersync_password=Datalight123!
keyadmin_password=Datalight123!


#Source for Audit Store. Currently solr, elasticsearch and cloudwatch logs are supported.
# * audit_store is solr
audit_store=solr

# * audit_solr_url Elasticsearch Host(s). E.g. 127.0.0.1
audit_elasticsearch_urls=
audit_elasticsearch_port=
audit_elasticsearch_protocol=
audit_elasticsearch_user=
audit_elasticsearch_password=
audit_elasticsearch_index=
audit_elasticsearch_bootstrap_enabled=true


# * audit_solr_url URL to Solr. E.g. http://<solr_host>:6083/solr/ranger_audits
audit_solr_urls=http://node01:6083/solr/ranger_audits
audit_solr_user=
audit_solr_password=
audit_solr_zookeepers=node01:2181,node02:2181,node03:2181

audit_solr_collection_name=ranger_audits
#solr Properties for cloud mode
audit_solr_config_name=ranger_audits
audit_solr_configset_location=
audit_solr_no_shards=1
audit_solr_no_replica=1
audit_solr_max_shards_per_node=1
# audit_solr_acl_user_list_sasl=solr,infra-solr
audit_solr_bootstrap_enabled=true

# * audit to amazon cloudwatch properties
audit_cloudwatch_region=
audit_cloudwatch_log_group=
audit_cloudwatch_log_stream_prefix=

#------------------------- DB CONFIG - END ----------------------------------

#
# ------- PolicyManager CONFIG ----------------
#

policymgr_external_url=http://node01:6080
policymgr_http_enabled=true
policymgr_https_keystore_file=
policymgr_https_keystore_keyalias=rangeradmin
policymgr_https_keystore_password=

#Add Supported Components list below separated by semi-colon, default value is empty string to support all components
#Example :  policymgr_supportedcomponents=hive,hbase,hdfs
policymgr_supportedcomponents=

#
# ------- PolicyManager CONFIG - END ---------------
#


#
# ------- UNIX User CONFIG ----------------
#
unix_user=datalight
unix_user_pwd=12345678
unix_group=datalight

#
# ------- UNIX User CONFIG  - END ----------------
#
#

#
# UNIX authentication service for Policy Manager
#
# PolicyManager can authenticate using UNIX username/password
# The UNIX server specified here as authServiceHostName needs to be installed with ranger-unix-ugsync package.
# Once the service is installed on authServiceHostName, the UNIX username/password from the host <authServiceHostName> can be used to login into policy manager
#
# ** The installation of ranger-unix-ugsync package can be installed after the policymanager installation is finished.
#
#LDAP|ACTIVE_DIRECTORY|UNIX|NONE
authentication_method=NONE
remoteLoginEnabled=true
authServiceHostName=localhost
authServicePort=5151
ranger_unixauth_keystore=keystore.jks
ranger_unixauth_keystore_password=password
ranger_unixauth_truststore=cacerts
ranger_unixauth_truststore_password=changeit

####LDAP settings - Required only if have selected LDAP authentication ####
#
# Sample Settings
#
#xa_ldap_url=ldap://127.0.0.1:389
#xa_ldap_userDNpattern=uid={0},ou=users,dc=xasecure,dc=net
#xa_ldap_groupSearchBase=ou=groups,dc=xasecure,dc=net
#xa_ldap_groupSearchFilter=(member=uid={0},ou=users,dc=xasecure,dc=net)
#xa_ldap_groupRoleAttribute=cn
#xa_ldap_base_dn=dc=xasecure,dc=net
#xa_ldap_bind_dn=cn=admin,ou=users,dc=xasecure,dc=net
#xa_ldap_bind_password=
#xa_ldap_referral=follow|ignore
#xa_ldap_userSearchFilter=(uid={0})

xa_ldap_url=
xa_ldap_userDNpattern=
xa_ldap_groupSearchBase=
xa_ldap_groupSearchFilter=
xa_ldap_groupRoleAttribute=
xa_ldap_base_dn=
xa_ldap_bind_dn=
xa_ldap_bind_password=
xa_ldap_referral=
xa_ldap_userSearchFilter=
####ACTIVE_DIRECTORY settings - Required only if have selected AD authentication ####
#
# Sample Settings
#
#xa_ldap_ad_domain=xasecure.net
#xa_ldap_ad_url=ldap://127.0.0.1:389
#xa_ldap_ad_base_dn=dc=xasecure,dc=net
#xa_ldap_ad_bind_dn=cn=administrator,ou=users,dc=xasecure,dc=net
#xa_ldap_ad_bind_password=
#xa_ldap_ad_referral=follow|ignore
#xa_ldap_ad_userSearchFilter=(sAMAccountName={0})

xa_ldap_ad_domain=
xa_ldap_ad_url=
xa_ldap_ad_base_dn=
xa_ldap_ad_bind_dn=
xa_ldap_ad_bind_password=
xa_ldap_ad_referral=
xa_ldap_ad_userSearchFilter=

#------------ Kerberos Config -----------------
spnego_principal=
spnego_keytab=
token_valid=30
cookie_domain=
cookie_path=/
admin_principal=
admin_keytab=
lookup_principal=
lookup_keytab=
hadoop_conf=/etc/hadoop/conf
#
#-------- SSO CONFIG - Start ------------------
#
sso_enabled=false
sso_providerurl=https://127.0.0.1:8443/gateway/knoxsso/api/v1/websso
sso_publickey=

#
#-------- SSO CONFIG - END ------------------

# Custom log directory path
RANGER_ADMIN_LOG_DIR=/data/datalight/logs/RANGER
RANGER_ADMIN_LOGBACK_CONF_FILE=

# PID file path
RANGER_PID_DIR_PATH=/data/datalight/pids/RANGER

# #################  DO NOT MODIFY ANY VARIABLES BELOW #########################
#
# --- These deployment variables are not to be modified unless you understand the full impact of the changes
#
################################################################################
XAPOLICYMGR_DIR=$PWD
app_home=$PWD/ews/webapp
TMPFILE=$PWD/.fi_tmp
LOGFILE=/data/datalight/logs/RANGER/logfile
LOGFILES="$LOGFILE"

JAVA_BIN='java'
JAVA_VERSION_REQUIRED='1.8'
JAVA_ORACLE='Java(TM) SE Runtime Environment'

ranger_admin_max_heap_size=1g
#retry DB and Java patches after the given time in seconds.
PATCH_RETRY_INTERVAL=120
STALE_PATCH_ENTRY_HOLD_TIME=10

#mysql_create_user_file=${PWD}/db/mysql/create_dev_user.sql
mysql_core_file=db/mysql/optimized/current/ranger_core_db_mysql.sql
mysql_audit_file=db/mysql/xa_audit_db.sql
#mysql_asset_file=${PWD}/db/mysql/reset_asset.sql

#oracle_create_user_file=${PWD}/db/oracle/create_dev_user_oracle.sql
oracle_core_file=db/oracle/optimized/current/ranger_core_db_oracle.sql
oracle_audit_file=db/oracle/xa_audit_db_oracle.sql
#oracle_asset_file=${PWD}/db/oracle/reset_asset_oracle.sql
#
postgres_core_file=db/postgres/optimized/current/ranger_core_db_postgres.sql
postgres_audit_file=db/postgres/xa_audit_db_postgres.sql
#
sqlserver_core_file=db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql
sqlserver_audit_file=db/sqlserver/xa_audit_db_sqlserver.sql
#
sqlanywhere_core_file=db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql
sqlanywhere_audit_file=db/sqlanywhere/xa_audit_db_sqlanywhere.sql
cred_keystore_filename=$app_home/WEB-INF/classes/conf/.jceks/rangeradmin.jceks
2.3.4 ranger-admin-services.sh

  • 路径

    /srv/datalight/RANGER/ranger-2.4.0-admin/ews/ranger-admin-services.sh

  • 内容

    # gc 日志路径修改为如下内容
-Xloggc:/data/datalight/logs/RANGER/gc-worker.log

2.4 修改数据库权限

GRANT ALL PRIVILEGES ON *.* TO 'root'@'node01' WITH GRANT OPTION;
FLUSH PRIVILEGES;

SHOW GRANTS FOR 'root'@'node01';

2.5 执行安装

cd /srv/datalight/RANGER/ranger-2.4.0-admin

./setup.sh

2.6 生成 Solr 审计配置

2.6.1 修改 contrib/solr_for_audit_setup

  • 路径

    /srv/datalight/RANGER/ranger-2.4.0-admin/contrib/solr_for_audit_setup/install.properties

  • 内容

    # Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

#Note: 
#1. This file is sourced from setup.sh, so make sure there are no spaces after the "="
#2. For variable with file path, please provide full path

#!/bin/bash

#JAVA_HOME to be used by Solr. Solr only support JDK 1.7 and above. If JAVA_HOME is not set
#in the env, then please set it here
#JAVA_HOME=

#The operating system (linux) user used by Solr process. You need to run Solr as the below user and group
SOLR_USER=datalight
SOLR_GROUP=datalight
#How long to keep the audit logs. Please note, audit records grows very rapidly. Make sure to 
#allocate enough memory and disk space to the server running Solr.
MAX_AUDIT_RETENTION_DAYS=180

#If you want this script to install Solr, set the value to true. If it is already installed, then set this to false
#If it is true, then it will download and install it.
#NOTE: If you want the script to install Solr, then this script needs to be executed as root.
SOLR_INSTALL=false

### BEGIN: if SOLR_INSTALL==true ###
#Location to download Solr. If SOLR_INSTALL is true, then SOLR_DOWNLOAD_URL is mandatory

#For open source version, pick a mirror from below. Recommended versions are Apache Solr 5.2.1 or above
#http://lucene.apache.org/solr/mirrors-solr-latest-redir.html

#Note: If possible, use the link from one of the mirror site
#SOLR_DOWNLOAD_URL=http://archive.apache.org/dist/lucene/solr/5.2.1/solr-5.2.1.tgz
SOLR_DOWNLOAD_URL=

### END: if SOLR_INSTALL==true ###

#The folder where Solr is installed. If SOLR_INSTALL=false, then Solr need to be preinstalled, else the setup will 
#install at the below location
#Note: If you are using RPM from LucidWorks in HDP, then Solr is by default installed in the following location:
#SOLR_INSTALL_FOLDER=/opt/lucidworks-hdpsearch/solr
SOLR_INSTALL_FOLDER=/opt/solr

#The location for the Solr configuration for Ranger. This script copies required configuration and
#startup scripts to the $SOLR_RANGER_HOME folder.
#NOTE: In SolrCloud mode, the data folders are under this folder. So make sure this is on seperate drive 
#      with enough disk space. Have 1TB free disk space on this volume. Also regularly monitor available disk space 
#      for this volume
#SOLR_RANGER_HOME=/opt/solr/ranger_audit_server
SOLR_RANGER_HOME=/srv/datalight/SOLR/server/solr/configsets/ranger_audits

#Port for Solr instance to be used by Ranger.
SOLR_RANGER_PORT=8983

#Standalone or SolrCloud. Valid values are "standalone" or "solrcloud"
SOLR_DEPLOYMENT=solrcloud

#### BEGIN: if SOLR_DEPLOYMENT=standalone ##########################
#Location for the data files. Make sure it has enough disk space. Since audits records can grow dramatically, 
#please have 1TB free disk space for the data folder. Also regularly monitor available disk space for this volume
SOLR_RANGER_DATA_FOLDER=/data/datalight/data/SOLR

#### END: if SOLR_DEPLOYMENT=standalone ##########################


#### BEGIN: if SOLR_DEPLOYMENT=solrcloud ##########################
#Comma seperated list of of zookeeper host and path. Give fully qualified domain name for the host
#SOLR_ZK=localhost:2181/ranger_audits
SOLR_ZK=node01:2181,node02:2181,node03:2181
#Base URL of the Solr. Used for creating collections
SOLR_HOST_URL=http://`hostname -f`:${SOLR_RANGER_PORT}
#Number of shards
SOLR_SHARDS=3
#Number of replication
SOLR_REPLICATION=2
#### END: if SOLR_DEPLOYMENT=solrcloud ##########################

#Location for the log file. Please note that "solr" or the process owner should have write permission 
#to log folder 
#SOLR_LOG_FOLDER=logs
SOLR_LOG_FOLDER=/data/datalight/logs/SOLR

SOLR_RANGER_COLLECTION=ranger_audits

#Memory for Solr. Both min and max memory to the java process are set to this value.
#Note: In production, please assign enough memory. It is recommended to have at least 2GB RAM.
#      Higher the RAM, the better. Solr core can take upto 32GB. For dev test you can use 512m
#SOLR_MAX_MEM=2g
#SOLR_MAX_MEM=512m
SOLR_MAX_MEM=512m
2.6.2 生成 Solr 配置

执行如下命令：

/srv/datalight/RANGER/ranger-2.4.0-admin/contrib/solr_for_audit_setup/setup.sh

之后得到 install_notes.txt，按照此步骤操作即可。

Solr installation notes for Ranger Audits.

Note: Don't edit this file. It will be over written if you run ./setup.sh again.

You have installed Solr in SolrCloud mode. You will have to do additional steps to create the collections for Ranger Audit. See below for instructions:

*** IMPORTANT ***
For configuring SolrCloud, you need to do the following:
1. Copy the same install.properties on all solr nodes and sing ./setup.sh script install and configure Solr for Ranger Audits on all other nodes also (don't start it yet)
2. Execute /srv/datalight/SOLR/server/solr/configsets/ranger_audits/scripts/add_ranger_audits_conf_to_zk.sh (only once from any node)
3. Start Solr on all nodes: /srv/datalight/SOLR/server/solr/configsets/ranger_audits/scripts/start_solr.sh
4. Create Ranger Audit collection: /srv/datalight/SOLR/server/solr/configsets/ranger_audits/scripts/create_ranger_audits_collection.sh (only once from any node)

Start and Stoping Solr:
Login as user datalight or root and the run the below commands to start or stop Solr:

To start Solr run: /srv/datalight/SOLR/server/solr/configsets/ranger_audits/scripts/start_solr.sh
To stop Solr run: /srv/datalight/SOLR/server/solr/configsets/ranger_audits/scripts/stop_solr.sh

After starting Solr for RangerAudit, Solr will listen at 6083. E.g http://node01:6083

Configure Ranger to use the following URL http://node01:6083/solr/ranger_audits

Solr HOME for Ranger Audit is /srv/datalight/SOLR/server/solr/configsets/ranger_audits

SOLR_REPLICATION: 2
SOLR_SHARDS: 3
DATA FOLDERS: /srv/datalight/SOLR/server/solr/configsets/ranger_audits/ranger_audits_shard*

Make sure you have enough disk space for index. In production, it is recommended to have at least 1TB free.
Filesystem               Size  Used Avail Use% Mounted on
/dev/mapper/centos-root  180G   97G   84G  54% /
2.6.3 创建配置挂载
/srv/datalight/SOLR/server/solr/configsets/ranger_audits/scripts/add_ranger_audits_conf_to_zk.sh

#!/bin/bash
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

#This scripts add the ranger_audit configuration with schema to zookeeper

function usage {
    echo "Error: The following properties need to be set in the script SOLR_ZK, SOLR_INSTALL_DIR and SOLR_RANGER_HOME"
    exit 1
}

JAVA_HOME=[E] 'JAVA_HOME' not found in /srv/datalight/RANGER/ranger-2.4.0-admin/contrib/solr_for_audit_setup/install.properties file while getting....!!
SOLR_USER=datalight
SOLR_ZK=node01:2181,node02:2181,node03:2181
SOLR_INSTALL_DIR=/srv/datalight/SOLR
SOLR_RANGER_HOME=/srv/datalight/SOLR/server/solr/configsets/ranger_audits

if [ "`whoami`" != "$SOLR_USER" ]; then
    if [ -w /etc/passwd ]; then
	echo "Running this script as $SOLR_USER..."
	su $SOLR_USER $0
    else
	echo "ERROR: You need to run this script $0 as user $SOLR_USER. You are currently running it as `whoami`"
    fi
    
    exit 1
fi

if [ "$SOLR_ZK" = "" ]; then
    usage
fi

if [ "$SOLR_INSTALL_DIR" = "" ]; then
    usage
fi

if [ "$SOLR_RANGER_HOME" = "" ]; then
    usage
fi

SOLR_RANGER_CONFIG_NAME=ranger_audits
SOLR_RANGER_CONFIG_LOCAL_PATH=${SOLR_RANGER_HOME}/conf
ZK_CLI=$SOLR_INSTALL_DIR/server/scripts/cloud-scripts/zkcli.sh


if [ ! -x $ZK_CLI ]; then
    echo "Error: $ZK_CLI is not found or you don't have permission to execute it."
    exit 1
fi
set -x
$ZK_CLI -cmd upconfig -zkhost $SOLR_ZK -confname $SOLR_RANGER_CONFIG_NAME -confdir $SOLR_RANGER_CONFIG_LOCAL_PATH
2.6.4 创建集合
/srv/datalight/SOLR/server/solr/configsets/ranger_audits/scripts/create_ranger_audits_collection.sh

#!/bin/bash
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

SOLR_HOST_URL=http://node01:8983
SOLR_ZK=node01:2181,node02:2181,node03:2181
SOLR_INSTALL_DIR=/srv/datalight/SOLR
SHARDS=3
REPLICATION=2

CONF_NAME=ranger_audits
COLLECTION_NAME=ranger_audits

which curl 2>&1 > /dev/null
if [ $? -ne 0 ]; then
    echo "curl is not found. Please install it for creating the collection"
    exit 1
fi

set -x
curl --negotiate -u : "${SOLR_HOST_URL}/solr/admin/collections?action=CREATE&name=${COLLECTION_NAME}&numShards=${SHARDS}&replicationFactor=${REPLICATION}&collection.configName=$CONF_NAME&maxShardsPerNode=100"

2.7 运行 Ranger Admin

su -c "/srv/datalight/RANGER/ranger-admin/ews/ranger-admin-services.sh start" "datalight"

# 访问页面
# 用户名：admin
# 密码：Datalight123!
http://192.168.137.10:6080/

2.8 停止 Ranger Admin

su -c "/srv/datalight/RANGER/ranger-2.4.0-admin/ews/ranger-admin-services.sh stop" "datalight"

三、部署 Ranger UserSync

Ranger UserSync 是 Apache Ranger 项目的一部分，Apache Ranger 是一个为 Hadoop 平台提供安全框架的工具，旨在帮助管理员管理、监控和审计对数据资源的访问。Ranger 提供了集中的服务来定义和实施企业安全策略，确保敏感数据的安全。

Ranger UserSync 组件的主要功能是从外部用户存储（如 LDAP 或 Active Directory）同步用户和组信息到 Ranger 管理系统中。这样做的好处是，可以确保 Ranger 中的用户和组信息与企业现有的身份管理系统保持一致，从而简化用户权限管理和维护工作。

3.1 解压 Ranger UserSync

cd /srv/datalight/RANGER

tar -zxf ranger-2.4.0-usersync.tar.gz -C /srv/datalight/RANGER

3.2 初始化必要目录

mkdir -p /data/datalight/pids/RANGER
mkdir -p /data/datalight/logs/RANGER

chown datalight:datalight -R /data/datalight/pids/RANGER
chown datalight:datalight -R /data/datalight/logs/RANGER

3.3 修改配置文件

3.3.1 install.properties

  • 路径

    /srv/datalight/RANGER/ranger-2.4.0-usersync/install.properties

  • 内容

    # Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# The base path for the usersync process
ranger_base_dir = /srv/datalight/RANGER/ranger-2.4.0-admin

#
# The following URL should be the base URL for connecting to the policy manager web application
# For example:
#
#  POLICY_MGR_URL = http://policymanager.xasecure.net:6080
#
POLICY_MGR_URL = http://node01:6080

# sync source,  only unix and ldap are supported at present
# defaults to unix
SYNC_SOURCE = unix

#
# Minimum Unix User-id to start SYNC.
# This should avoid creating UNIX system-level users in the Policy Manager
#
MIN_UNIX_USER_ID_TO_SYNC = 500

# Minimum Unix Group-id to start SYNC.
# This should avoid creating UNIX system-level users in the Policy Manager
#
MIN_UNIX_GROUP_ID_TO_SYNC = 500

# sync interval in minutes
# user, groups would be synced again at the end of each sync interval
# defaults to 5   if SYNC_SOURCE is unix
# defaults to 360 if SYNC_SOURCE is ldap
SYNC_INTERVAL = 1

#User and group for the usersync process
unix_user=datalight
unix_group=datalight

#change password of rangerusersync user. Please note that this password should be as per rangerusersync user in ranger
rangerUsersync_password=Datalight123!

#Set to run in kerberos environment
usersync_principal=
usersync_keytab=
hadoop_conf={{hadoop_conf}}
#
# The file where all credential is kept in cryptic format
#
CRED_KEYSTORE_FILENAME=/srv/datalight/RANGER/ranger-2.4.0-usersync/rangerusersync.jceks

# SSL Authentication
AUTH_SSL_ENABLED=false
#AUTH_SSL_KEYSTORE_FILE=/etc/ranger/usersync/conf/cert/unixauthservice.jks
#AUTH_SSL_KEYSTORE_PASSWORD=UnIx529p
AUTH_SSL_TRUSTSTORE_FILE=
AUTH_SSL_TRUSTSTORE_PASSWORD=

# ---------------------------------------------------------------
# The following properties are relevant only if SYNC_SOURCE = ldap
# ---------------------------------------------------------------

# The below properties ROLE_ASSIGNMENT_LIST_DELIMITER, USERS_GROUPS_ASSIGNMENT_LIST_DELIMITER, USERNAME_GROUPNAME_ASSIGNMENT_LIST_DELIMITER,
#and GROUP_BASED_ROLE_ASSIGNMENT_RULES can be used to assign role to LDAP synced users and groups
#NOTE all the delimiters should have different values and the delimiters should not contain characters that are allowed in userName or GroupName

# default value ROLE_ASSIGNMENT_LIST_DELIMITER = &
ROLE_ASSIGNMENT_LIST_DELIMITER = &

#default value USERS_GROUPS_ASSIGNMENT_LIST_DELIMITER = :
USERS_GROUPS_ASSIGNMENT_LIST_DELIMITER = :

#default value USERNAME_GROUPNAME_ASSIGNMENT_LIST_DELIMITER = ,
USERNAME_GROUPNAME_ASSIGNMENT_LIST_DELIMITER = ,

# with above mentioned delimiters a sample value would be ROLE_SYS_ADMIN:u:userName1,userName2&ROLE_SYS_ADMIN:g:groupName1,groupName2&ROLE_KEY_ADMIN:u:userName&ROLE_KEY_ADMIN:g:groupName&ROLE_USER:u:userName3,userName4&ROLE_USER:g:groupName3
#&ROLE_ADMIN_AUDITOR:u:userName&ROLE_KEY_ADMIN_AUDITOR:u:userName&ROLE_KEY_ADMIN_AUDITOR:g:groupName&ROLE_ADMIN_AUDITOR:g:groupName
GROUP_BASED_ROLE_ASSIGNMENT_RULES =

# URL of source ldap
# a sample value would be:  ldap://ldap.example.com:389
# Must specify a value if SYNC_SOURCE is ldap
SYNC_LDAP_URL =

# ldap bind dn used to connect to ldap and query for users and groups
# a sample value would be cn=admin,ou=users,dc=hadoop,dc=apache,dc=org
# Must specify a value if SYNC_SOURCE is ldap
SYNC_LDAP_BIND_DN =

# ldap bind password for the bind dn specified above
# please ensure read access to this file  is limited to root, to protect the password
# Must specify a value if SYNC_SOURCE is ldap
# unless anonymous search is allowed by the directory on users and group
SYNC_LDAP_BIND_PASSWORD =

# ldap delta sync flag used to periodically sync users and groups based on the updates in the server
# please customize the value to suit your deployment
# default value is set to true when is SYNC_SOURCE is ldap
SYNC_LDAP_DELTASYNC =

# search base for users and groups
# sample value would be dc=hadoop,dc=apache,dc=org
SYNC_LDAP_SEARCH_BASE =

# search base for users
# sample value would be ou=users,dc=hadoop,dc=apache,dc=org
# overrides value specified in SYNC_LDAP_SEARCH_BASE
SYNC_LDAP_USER_SEARCH_BASE =

# search scope for the users, only base, one and sub are supported values
# please customize the value to suit your deployment
# default value: sub
SYNC_LDAP_USER_SEARCH_SCOPE = sub

# objectclass to identify user entries
# please customize the value to suit your deployment
# default value: person
SYNC_LDAP_USER_OBJECT_CLASS = person

# optional additional filter constraining the users selected for syncing
# a sample value would be (dept=eng)
# please customize the value to suit your deployment
# default value is empty
SYNC_LDAP_USER_SEARCH_FILTER =

# attribute from user entry that would be treated as user name
# please customize the value to suit your deployment
# default value: cn
SYNC_LDAP_USER_NAME_ATTRIBUTE = cn

# attribute from user entry whose values would be treated as
# group values to be pushed into Policy Manager database
# You could provide multiple attribute names separated by comma
# default value: memberof, ismemberof
SYNC_LDAP_USER_GROUP_NAME_ATTRIBUTE = memberof,ismemberof
#
# UserSync - Case Conversion Flags
# possible values:  none, lower, upper
SYNC_LDAP_USERNAME_CASE_CONVERSION=lower
SYNC_LDAP_GROUPNAME_CASE_CONVERSION=lower

#user sync log path
logdir=/data/datalight/logs/RANGER
#/var/log/ranger/usersync

# PID DIR PATH
USERSYNC_PID_DIR_PATH=/data/datalight/pids/RANGER

# do we want to do ldapsearch to find groups instead of relying on user entry attributes
# valid values: true, false
# any value other than true would be treated as false
# default value: false
SYNC_GROUP_SEARCH_ENABLED=

# do we want to do ldapsearch to find groups instead of relying on user entry attributes and
# sync memberships of those groups
# valid values: true, false
# any value other than true would be treated as false
# default value: false
SYNC_GROUP_USER_MAP_SYNC_ENABLED=

# search base for groups
# sample value would be ou=groups,dc=hadoop,dc=apache,dc=org
# overrides value specified in SYNC_LDAP_SEARCH_BASE,  SYNC_LDAP_USER_SEARCH_BASE
# if a value is not specified, takes the value of  SYNC_LDAP_SEARCH_BASE
# if  SYNC_LDAP_SEARCH_BASE is also not specified, takes the value of SYNC_LDAP_USER_SEARCH_BASE
SYNC_GROUP_SEARCH_BASE=

# search scope for the groups, only base, one and sub are supported values
# please customize the value to suit your deployment
# default value: sub
SYNC_GROUP_SEARCH_SCOPE=

# objectclass to identify group entries
# please customize the value to suit your deployment
# default value: groupofnames
SYNC_GROUP_OBJECT_CLASS=

# optional additional filter constraining the groups selected for syncing
# a sample value would be (dept=eng)
# please customize the value to suit your deployment
# default value is empty
SYNC_LDAP_GROUP_SEARCH_FILTER=

# attribute from group entry that would be treated as group name
# please customize the value to suit your deployment
# default value: cn
SYNC_GROUP_NAME_ATTRIBUTE=

# attribute from group entry that is list of members
# please customize the value to suit your deployment
# default value: member
SYNC_GROUP_MEMBER_ATTRIBUTE_NAME=

# do we want to use paged results control during ldapsearch for user entries
# valid values: true, false
# any value other than true would be treated as false
# default value: true
# if the value is false, typical AD would not return more than 1000 entries
SYNC_PAGED_RESULTS_ENABLED=

# page size for paged results control
# search results would be returned page by page with the specified number of entries per page
# default value: 500
SYNC_PAGED_RESULTS_SIZE=
#LDAP context referral could be ignore or follow
SYNC_LDAP_REFERRAL =ignore

# if you want to enable or disable jvm metrics for usersync process
# valid values: true, false
# any value other than true would be treated as false
# default value: false
# if the value is false, jvm metrics is not created
JVM_METRICS_ENABLED=

# filename of jvm metrics created for usersync process
# default value: ranger_usersync_metric.json
JVM_METRICS_FILENAME=

#file directory for jvm metrics
# default value : logdir
JVM_METRICS_FILEPATH=

#frequency for jvm metrics to be updated
# default value : 10000 milliseconds 
JVM_METRICS_FREQUENCY_TIME_IN_MILLIS=

3.4 执行初始化

./setup.sh

3.5 启用插件

sed -i '/<name>ranger.usersync.enabled<\/name>/{n;s/<value>false<\/value>/<value>true<\/value>/}' /srv/datalight/RANGER/ranger-2.4.0-usersync/conf/ranger-ugsync-site.xml

3.6 运行 Ranger UserSync

su -c "/srv/datalight/RANGER/ranger-2.4.0-usersync/ranger-usersync-services.sh start" "datalight"

3.7 停止 Ranger UserSync

su -c "/srv/datalight/RANGER/ranger-2.4.0-usersync/ranger-usersync-services.sh stop" "datalight"

关注我们

感谢您一直以来对 DataLight 的支持和信任。DataLight 将为您带来更加智能和高效的数据管理体验。期待您的使用和反馈！
——DataLight 团队

关注我们，获取更多最新资讯：

微信公众号：

QQ 交流群：

微信交流：

一起见证数据世界的无限可能！

开源协议

本项目采用 Apache 2.0 开源协议。有关详细内容，请查看 Apache 2.0 LICENSE。

Boundivore 班德沃
关注
专栏目录
Ranger 编译部署
zachz的博客
05-01 706
大数据系列-Ranger
Centos7 + Apache Ranger 2.4.0 部署
snipercai的博客
08-30 3486
Centos7 + Apache Ranger 2.4.0 部署：Apache Ranger提供一个集中式安全管理框架, 并解决授权和审计。它可以对Hadoop生态的组件如HDFS、Yarn、Hive、Hbase等进行细粒度的数据访问控制。通过操作Ranger控制台,管理员可以轻松的通过配置策略来控制用户访问权限。
Ambari安装Ranger详细教程
MH_Tan的博客
08-16 7933
准备工作 1.找到ambari-server所在的服务器（这里是hadoop-02） 2.进入/usr/java/share/java/目录下查看数据库驱动包是否存在，并确认是什么类型的数据库 这里确认ambari使用的是mysql数据库，驱动包路径：/usr/share/java/mysql-connector-java.jar 3. 根据前两步执行命令 ambari-se...
Apache Ranger 2.4.0 安装部署
guofei_ok的博客
06-30 722
ranger2.4.0安装部署
Apache Ranger：(一)安装部署
小健的博客
10-09 2325
Ranger提供一个集中式安全管理框架, 并解决授权和审计。它可以对生态的组件如、Yarn、等进行细粒度的数据访问控制。通过操作Ranger控制台,管理员可以轻松的通过配置策略来控制用户访问权限。说白了就是管理大多数框架的授权问题。
Linux下ranger安装配置教程
汽车电子软件领域知识分享
04-08 1630
参考 在Linux上，如何安装Ranger终端文件管理器
ranger 教程
潜行
09-05 3081
ranger 综述性文章 [Apache ranger 简介、原理、安装部署]（https://blog.csdn.net/sudaxhh/article/details/52135184） … 源码解析 [Ranger对HDFS访问控制的实现原理]（https://www.cnblogs.com/qiuyuesu/p/6774520.html） … ...
Hadoop集群权限框架 Ranger2.1.0兼容Hadoop3.2.4部署
红桃∩
04-29 605
Ranger2.1.0源码兼容的是Hadoop3.1.0 本文章基于Hadoop3.2.4，对Ranger2.1.0做变动，打包部署，以及遇到的问题解决方案。
Apache Ranger-2.2 详细安装步骤
我胡汉三又回来了的博客
05-07 2786
Apache Ranger是一个用来在Hadoop平台上进行监控，启用服务，以及全方位数据安全访问管理的安全框架
ranger-2.0.0-hbase-plugin.tar.gz
06-04
描述中提到，通常在互联网上找到的教程多是关于如何从源代码编译Ranger，而这个压缩包提供了一个已经编译好的版本，节省了用户自行编译的时间和可能遇到的问题。编译Ranger时，可能需要依赖各种库和工具，配置环境...
基于ranger的presto账号权限管理及事件监听方案
a80090023的专栏
08-11 2399
背景：我司为解决hive数据分析慢问题，部署了presto 347版本集群。同时即席分析平台采用开源产品superset 1.0版本。 为满足公司业务场景，需开发以下功能解决认证及审计等问题。 jdbc等连接方式（开发同学专用）需验证ranger账号密码并通过ranger进行库表权限验证。 superset登录（数据分析同学使用）需验证公司sso账号，且superset端所有同学共用一个presto数据源连接，需单独设计权限认证方式。 开发事件监听器，用于用户SQL查询情况审计等。 一、账号密码
Ambari部署Kafka
Java框架、并发编程、分布式、微服务、Redis、HarmonyOS、中间件等技术
07-23 1340
添加kafka service 选择kafka Next 部署kafka HA 选择 ambari-01 ambari-02 ambari-03 ambari-01:2181 next 这里是HBase主堆大小没达到最小设置值，可以不用改，Proceed Anyway仍要继续 Review安装的组件 DEPLOY SUCCESS Next 成功后Next Summary 注意提示 Important: After closing .
随机森林模型部署与优化实践
AGI通用人工智能之禅
04-09 250
非常感谢您提供了如此详细的任务要求和约束条件。我将尽我所能按要求完成这篇高质量的技术博客文章。作为一位世界级的人工智能专家和计算机领域大师,我会以专业的技术语言,结合深入的研究和丰富的实践经验,为您呈现一篇内容丰富、结构清晰、见解独到的《随机森林模型部署与优化实践》技术博客。我会严格遵守您提出的各项要求,确保文章内容的准确性、实用性和可读性。让我们一起开始撰写这篇精彩的技术博客吧!
免费开源的功能强大并且识别率非常准确的验证码识别开源项目
代码讲故事
09-27 1112
免费开源的功能强大并且识别率非常准确的验证码识别开源项目。
清华大学开源视频转文本模型——CogVLM2-Llama3-Caption
weixin_41446370的博客
09-24 418
通常情况下，大多数视频数据并不附带相应的描述性文本，因此有必要将视频数据转换为文本描述，为文本到视频模型提供必要的训练数据。CogVLM2-Caption 是一个视频字幕模型，用于为 CogVideoX 模型生成训练数据。
【开源免费】基于SpringBoot+Vue.JS服装销售平台（JAVA毕业设计）
customer08的博客
09-27 1250
本站是一个B/S模式系统，采用Spring Boot框架作为开发技术，MYSQL数据库设计开发，充分保证系统的稳定性。系统具有界面清晰、操作简单，功能齐全的特点，使得服装销售平台管理工作系统化、规范化。
robomimic基础教程（四）——开源数据集
最新发布
qq_28912651的博客
09-27 468
robomimic开源了大量数据集及仿真环境，数据集标准格式为HDF5
云栖实录 | 开源大数据全面升级：Native 核心引擎、Serverless 化、湖仓架构引领云上大数据发展
科技很有意思
09-27 847
本文根据2024云栖大会开源大数据专场实录整理而成，开源大数据全面升级：Native 核心引擎、Serverless 化、湖仓架构引领云上大数据发展。
Apache Ranger REST API 操作指南
"Apache Ranger Public REST APIs" Apache Ranger 是一个用于大数据安全的服务，它提供了策略管理、审计和访问控制功能，广泛应用于Hadoop生态系统中的不同组件，如HDFS、Hive、Kafka等。Ranger 提供了丰富的...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值