Apache Ranger 2.4编译打包

longxibo

已于 2024-09-06 10:03:10 修改

阅读量691

点赞数 3

文章标签：大数据数据仓库

于 2024-09-06 09:59:20 首次发布

本文链接：https://blog.csdn.net/longxibo/article/details/141950272

版权

Apache Ranger 2.4编译打包

Apache Ranger

Apache Ranger框架，用于启用、监控和管理跨Hadoop平台的综合数据安全性。

Ranger的愿景是在Apache Hadoop生态系统中提供全面的安全性。随着Apache YARN的出现，Hadoop平台现在可以支持真正的数据湖架构。企业可以在多租户环境中运行多个工作负载。Hadoop中的数据安全需要不断发展，以支持数据访问的多个用例，同时还需要为安全策略的中央管理和用户访问的监控提供一个框架。

Apache Ranger有以下目标:

集中的安全管理，在一个中央UI或使用REST api管理所有与安全相关的任务。
细粒度授权，使用Hadoop组件/工具执行特定的操作和/或操作，并通过中央管理工具进行管理
标准化所有Hadoop组件的授权方法。
增强了对不同授权方法的支持——基于角色的访问控制、基于属性的访问控制等。

在Hadoop的所有组件中集中审计用户访问和管理操作(与安全相关)。

Apache Ranger

这么强大工具，已经开源，然而都是源码，没有编译好的包，需要自己花时间编译。为了节省大家时间，下面列出编译过程及编译好的程序包

下载Apache Ranger 2.4源码

假设下载到centos7 的目录 /usr/local目录

cd /usr/local
git clone -b ranger-2.4 https://github.com/apache/ranger.git

配置Maven配置文件

我本地使用mvn 3.6.3 jdk1.8。 mvn及jdk安装，请查看其他人的安装文档，这里省去500个字。
maven jdk版本
maven setting.xml配置内容如下（它的位置在mavenhome\conf下）：

<?xml version="1.0" encoding="UTF-8"?>

<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements.  See the NOTICE file
distributed with this work for additional information
regarding copyright ownership.  The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License.  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied.  See the License for the
specific language governing permissions and limitations
under the License.
-->

<!--
 | This is the configuration file for Maven. It can be specified at two levels:
 |
 |  1. User Level. This settings.xml file provides configuration for a single user,
 |                 and is normally provided in ${user.home}/.m2/settings.xml.
 |
 |                 NOTE: This location can be overridden with the CLI option:
 |
 |                 -s /path/to/user/settings.xml
 |
 |  2. Global Level. This settings.xml file provides configuration for all Maven
 |                 users on a machine (assuming they're all using the same Maven
 |                 installation). It's normally provided in
 |                 ${maven.conf}/settings.xml.
 |
 |                 NOTE: This location can be overridden with the CLI option:
 |
 |                 -gs /path/to/global/settings.xml
 |
 | The sections in this sample file are intended to give you a running start at
 | getting the most out of your Maven installation. Where appropriate, the default
 | values (values used when the setting is not specified) are provided.
 |
 |-->
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
  <!-- localRepository
   | The path to the local repository maven will use to store artifacts.
   |
   | Default: ${user.home}/.m2/repository
  <localRepository>/path/to/local/repo</localRepository>
  -->

  <!-- interactiveMode
   | This will determine whether maven prompts you when it needs input. If set to false,
   | maven will use a sensible default value, perhaps based on some other setting, for
   | the parameter in question.
   |
   | Default: true
  <interactiveMode>true</interactiveMode>
  -->

  <!-- offline
   | Determines whether maven should attempt to connect to the network when executing a build.
   | This will have an effect on artifact downloads, artifact deployment, and others.
   |
   | Default: false
  <offline>false</offline>
  -->

  <!-- pluginGroups
   | This is a list of additional group identifiers that will be searched when resolving plugins by their prefix, i.e.
   | when invoking a command line like "mvn prefix:goal". Maven will automatically add the group identifiers
   | "org.apache.maven.plugins" and "org.codehaus.mojo" if these are not already contained in the list.
   |-->
  <pluginGroups>
    <!-- pluginGroup
     | Specifies a further group identifier to use for plugin lookup.
    <pluginGroup>com.your.plugins</pluginGroup>
    -->
  </pluginGroups>

  <!-- proxies
   | This is a list of proxies which can be used on this machine to connect to the network.
   | Unless otherwise specified (by system property or command-line switch), the first proxy
   | specification in this list marked as active will be used.
   |-->
  <proxies>
    <!-- proxy
     | Specification for one proxy, to be used in connecting to the network.
     |
    <proxy>
      <id>optional</id>
      <active>true</active>
      <protocol>http</protocol>
      <username>proxyuser</username>
      <password>proxypass</password>
      <host>proxy.host.net</host>
      <port>80</port>
      <nonProxyHosts>local.net|some.host.com</nonProxyHosts>
    </proxy>
    -->
  </proxies>

  <!-- servers
   | This is a list of authentication profiles, keyed by the server-id used within the system.
   | Authentication profiles can be used whenever maven must make a connection to a remote server.
   |-->
  <servers>
    <!-- server
     | Specifies the authentication information to use when connecting to a particular server, identified by
     | a unique name within the system (referred to by the 'id' attribute below).
     |
     | NOTE: You should either specify username/password OR privateKey/passphrase, since these pairings are
     |       used together.
     |
    <server>
      <id>deploymentRepo</id>
      <username>repouser</username>
      <password>repopwd</password>
    </server>
    -->

    <!-- Another sample, using keys to authenticate.
    <server>
      <id>siteServer</id>
      <privateKey>/path/to/private/key</privateKey>
      <passphrase>optional; leave empty if not used.</passphrase>
    </server>
    -->
  </servers>

  <!-- mirrors
   | This is a list of mirrors to be used in downloading artifacts from remote repositories.
   |
   | It works like this: a POM may declare a repository to use in resolving certain artifacts.
   | However, this repository may have problems with heavy traffic at times, so people have mirrored
   | it to several places.
   |
   | That repository definition will have a unique id, so we can create a mirror reference for that
   | repository, to be used as an alternate download site. The mirror site will be the preferred
   | server for that repository.
   |-->
  <mirrors>
    <!-- mirror
     | Specifies a repository mirror site to use instead of a given repository. The repository that
     | this mirror serves has an ID that matches the mirrorOf element of this mirror. IDs are used
     | for inheritance and direct lookup purposes, and must be unique across the set of mirrors.
     |
    <mirror>
      <id>mirrorId</id>
      <mirrorOf>repositoryId</mirrorOf>
      <name>Human Readable Name for this Mirror.</name>
      <url>http://my.repository.com/repo/path</url>
    </mirror>
     -->
     <mirror>
       <id>alimaven</id>
       <name>aliyun maven</name>
       <url>http://maven.aliyun.com/repository/public</url>
       <mirrorOf>*</mirrorOf>
     </mirror>
     <mirror>
            <id>npm</id>
            <name>NPM Mirror</name>
            <url>https://npm.taobao.org</url>
            <mirrorOf>*</mirrorOf>
     </mirror>
    <!-- <mirror>
           <id>ibiblio.org</id>
           <name>ibiblio Mirror of http://repo1.maven.org/maven2/</name>
           <url>http://mirrors.ibiblio.org/pub/mirrors/maven2</url>
           <mirrorOf>central</mirrorOf>
     </mirror> 
     <mirror>
         <id>jboss-public-repository-group</id>
         <mirrorOf>central</mirrorOf>
         <name>JBoss Public Repository Group</name>
         <url>http://repository.jboss.org/nexus/content/groups/public</url>
     </mirror> -->
  </mirrors>

  <!-- profiles
   | This is a list of profiles which can be activated in a variety of ways, and which can modify
   | the build process. Profiles provided in the settings.xml are intended to provide local machine-
   | specific paths and repository locations which allow the build to work in the local environment.
   |
   | For example, if you have an integration testing plugin - like cactus - that needs to know where
   | your Tomcat instance is installed, you can provide a variable here such that the variable is
   | dereferenced during the build process to configure the cactus plugin.
   |
   | As noted above, profiles can be activated in a variety of ways. One way - the activeProfiles
   | section of this document (settings.xml) - will be discussed later. Another way essentially
   | relies on the detection of a system property, either matching a particular value for the property,
   | or merely testing its existence. Profiles can also be activated by JDK version prefix, where a
   | value of '1.4' might activate a profile when the build is executed on a JDK version of '1.4.2_07'.
   | Finally, the list of active profiles can be specified directly from the command line.
   |
   | NOTE: For profiles defined in the settings.xml, you are restricted to specifying only artifact
   |       repositories, plugin repositories, and free-form properties to be used as configuration
   |       variables for plugins in the POM.
   |
   |-->
  <profiles>
    <!-- profile
     | Specifies a set of introductions to the build process, to be activated using one or more of the
     | mechanisms described above. For inheritance purposes, and to activate profiles via <activatedProfiles/>
     | or the command line, profiles have to have an ID that is unique.
     |
     | An encouraged best practice for profile identification is to use a consistent naming convention
     | for profiles, such as 'env-dev', 'env-test', 'env-production', 'user-jdcasey', 'user-brett', etc.
     | This will make it more intuitive to understand what the set of introduced profiles is attempting
     | to accomplish, particularly when you only have a list of profile id's for debug.
     |
     | This profile example uses the JDK version to trigger activation, and provides a JDK-specific repo.
    <profile>
      <id>jdk-1.4</id>

      <activation>
        <jdk>1.4</jdk>
      </activation>

      <repositories>
        <repository>
          <id>jdk14</id>
          <name>Repository for JDK 1.4 builds</name>
          <url>http://www.myhost.com/maven/jdk14</url>
          <layout>default</layout>
          <snapshotPolicy>always</snapshotPolicy>
        </repository>
      </repositories>
    </profile>
    -->

    <!--
     | Here is another profile, activated by the system property 'target-env' with a value of 'dev',
     | which provides a specific path to the Tomcat instance. To use this, your plugin configuration
     | might hypothetically look like:
     |
     | ...
     | <plugin>
     |   <groupId>org.myco.myplugins</groupId>
     |   <artifactId>myplugin</artifactId>
     |
     |   <configuration>
     |     <tomcatLocation>${tomcatPath}</tomcatLocation>
     |   </configuration>
     | </plugin>
     | ...
     |
     | NOTE: If you just wanted to inject this configuration whenever someone set 'target-env' to
     |       anything, you could just leave off the <value/> inside the activation-property.
     |
    <profile>
      <id>env-dev</id>

      <activation>
        <property>
          <name>target-env</name>
          <value>dev</value>
        </property>
      </activation>

      <properties>
        <tomcatPath>/path/to/tomcat/instance</tomcatPath>
      </properties>
    </profile>
    -->
   <profile>
      <id>mvnrepository</id>
      <repositories>
        <repository>
          <id>mvnrepository</id>
          <name>MvnRepository</name>
          <url>https://repo1.maven.org/maven2</url>
          <releases>
            <enabled>true</enabled>
          </releases>
          <snapshots>
            <enabled>false</enabled>
          </snapshots>
        </repository>
      </repositories>
    </profile>
  </profiles>
  <activeProfiles>
    <activeProfile>mvnrepository</activeProfile>
  </activeProfiles>
  
</settings>

修改地方是映像使用阿里云仓库及淘宝npm库，主库使用MvnRepository

开始编译

在centos7 敲入命令行

cd /usr/local/ranger/
mvn clean compile package -DskipTests -Drat.skip=true -Dmaven.wagon.http.ssl.insecure=true

经过吭叽吭叽一两钟头，终于编译成功

[INFO] Reactor Summary for ranger 2.4.1-SNAPSHOT:
[INFO] 
[INFO] ranger ............................................. SUCCESS [  6.658 s]
[INFO] Jdbc SQL Connector ................................. SUCCESS [  2.699 s]
[INFO] Credential Support ................................. SUCCESS [  4.961 s]
[INFO] Audit Component .................................... SUCCESS [ 11.050 s]
[INFO] ranger-plugin-classloader .......................... SUCCESS [  0.750 s]
[INFO] Common library for Plugins ......................... SUCCESS [  8.316 s]
[INFO] ranger-intg ........................................ SUCCESS [  5.076 s]
[INFO] Installer Support Component ........................ SUCCESS [  0.821 s]
[INFO] Credential Builder ................................. SUCCESS [  3.138 s]
[INFO] Embedded Web Server Invoker ........................ SUCCESS [  5.996 s]
[INFO] Key Management Service ............................. SUCCESS [ 33.171 s]
[INFO] HBase Security Plugin Shim ......................... SUCCESS [ 11.669 s]
[INFO] HBase Security Plugin .............................. SUCCESS [ 12.572 s]
[INFO] Hdfs Security Plugin ............................... SUCCESS [  7.488 s]
[INFO] Hive Security Plugin ............................... SUCCESS [ 18.656 s]
[INFO] Knox Security Plugin Shim .......................... SUCCESS [  3.813 s]
[INFO] Knox Security Plugin ............................... SUCCESS [ 12.009 s]
[INFO] Storm Security Plugin .............................. SUCCESS [  5.834 s]
[INFO] YARN Security Plugin ............................... SUCCESS [  3.251 s]
[INFO] Ozone Security Plugin .............................. SUCCESS [  4.956 s]
[INFO] Ranger Util ........................................ SUCCESS [  6.301 s]
[INFO] Unix Authentication Client ......................... SUCCESS [  0.930 s]
[INFO] User Group Synchronizer Util ....................... SUCCESS [  0.381 s]
[INFO] Security Admin Web Application ..................... SUCCESS [02:50 min]
[INFO] KAFKA Security Plugin .............................. SUCCESS [ 12.968 s]
[INFO] SOLR Security Plugin ............................... SUCCESS [ 12.985 s]
[INFO] NiFi Security Plugin ............................... SUCCESS [  8.801 s]
[INFO] NiFi Registry Security Plugin ...................... SUCCESS [  7.111 s]
[INFO] Kudu Security Plugin ............................... SUCCESS [  6.246 s]
[INFO] Unix User Group Synchronizer ....................... SUCCESS [  9.066 s]
[INFO] Ldap Config Check Tool ............................. SUCCESS [  1.238 s]
[INFO] Unix Authentication Service ........................ SUCCESS [  7.356 s]
[INFO] Unix Native Authenticator .......................... SUCCESS [  4.229 s]
[INFO] KMS Security Plugin ................................ SUCCESS [ 12.488 s]
[INFO] Tag Synchronizer ................................... SUCCESS [  8.150 s]
[INFO] Hdfs Security Plugin Shim .......................... SUCCESS [  7.487 s]
[INFO] Hive Security Plugin Shim .......................... SUCCESS [ 29.801 s]
[INFO] YARN Security Plugin Shim .......................... SUCCESS [  6.206 s]
[INFO] OZONE Security Plugin Shim ......................... SUCCESS [  6.757 s]
[INFO] Storm Security Plugin shim ......................... SUCCESS [  8.732 s]
[INFO] KAFKA Security Plugin Shim ......................... SUCCESS [  6.501 s]
[INFO] SOLR Security Plugin Shim .......................... SUCCESS [  9.809 s]
[INFO] Atlas Security Plugin Shim ......................... SUCCESS [  7.767 s]
[INFO] KMS Security Plugin Shim ........................... SUCCESS [ 11.319 s]
[INFO] ranger-examples .................................... SUCCESS [  0.269 s]
[INFO] Ranger Examples - Conditions and ContextEnrichers .. SUCCESS [  6.454 s]
[INFO] Ranger Examples - SampleApp ........................ SUCCESS [  0.346 s]
[INFO] Ranger Examples - Ranger Plugin for SampleApp ...... SUCCESS [  6.723 s]
[INFO] sample-client ...................................... SUCCESS [  6.788 s]
[INFO] Apache Ranger Examples Distribution ................ SUCCESS [ 10.089 s]
[INFO] Ranger Tools ....................................... SUCCESS [  5.639 s]
[INFO] Atlas Security Plugin .............................. SUCCESS [  7.791 s]
[INFO] SchemaRegistry Security Plugin ..................... SUCCESS [ 17.089 s]
[INFO] Sqoop Security Plugin .............................. SUCCESS [  7.315 s]
[INFO] Sqoop Security Plugin Shim ......................... SUCCESS [  5.968 s]
[INFO] Kylin Security Plugin .............................. SUCCESS [ 18.689 s]
[INFO] Kylin Security Plugin Shim ......................... SUCCESS [ 13.926 s]
[INFO] Presto Security Plugin ............................. SUCCESS [ 10.961 s]
[INFO] Presto Security Plugin Shim ........................ SUCCESS [  9.074 s]
[INFO] Elasticsearch Security Plugin Shim ................. SUCCESS [  1.296 s]
[INFO] Elasticsearch Security Plugin ...................... SUCCESS [  8.167 s]
[INFO] Apache Ranger Distribution ......................... SUCCESS [02:52 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  13:56 min
[INFO] Finished at: 2024-09-06T09:52:25+08:00
[INFO] ------------------------------------------------------------------------

[root@bogon ranger]# ll target
总用量 1416928
drwxr-xr-x. 2 root root      4096 9月   5 20:09 antrun
drwxr-xr-x. 3 root root      4096 9月   5 20:09 maven-shared-archive-resources
-rw-r--r--. 1 root root 579530756 9月   6 09:51 ranger-2.4.1-SNAPSHOT-admin.tar.gz
-rw-r--r--. 1 root root  43734162 9月   6 09:52 ranger-2.4.1-SNAPSHOT-atlas-plugin.tar.gz
-rw-r--r--. 1 root root  34175000 9月   6 09:52 ranger-2.4.1-SNAPSHOT-elasticsearch-plugin.tar.gz
-rw-r--r--. 1 root root  39125057 9月   6 09:49 ranger-2.4.1-SNAPSHOT-hbase-plugin.tar.gz
-rw-r--r--. 1 root root  37689521 9月   6 09:49 ranger-2.4.1-SNAPSHOT-hdfs-plugin.tar.gz
-rw-r--r--. 1 root root  37482544 9月   6 09:49 ranger-2.4.1-SNAPSHOT-hive-plugin.tar.gz
-rw-r--r--. 1 root root  56853557 9月   6 09:50 ranger-2.4.1-SNAPSHOT-kafka-plugin.tar.gz
-rw-r--r--. 1 root root 195392451 9月   6 09:51 ranger-2.4.1-SNAPSHOT-kms.tar.gz
-rw-r--r--. 1 root root  51457769 9月   6 09:50 ranger-2.4.1-SNAPSHOT-knox-plugin.tar.gz
-rw-r--r--. 1 root root  36632033 9月   6 09:52 ranger-2.4.1-SNAPSHOT-kylin-plugin.tar.gz
-rw-r--r--. 1 root root     34257 9月   6 09:51 ranger-2.4.1-SNAPSHOT-migration-util.tar.gz
-rw-r--r--. 1 root root  43401120 9月   6 09:50 ranger-2.4.1-SNAPSHOT-ozone-plugin.tar.gz
-rw-r--r--. 1 root root  57427901 9月   6 09:52 ranger-2.4.1-SNAPSHOT-presto-plugin.tar.gz
-rw-r--r--. 1 root root  16565864 9月   6 09:51 ranger-2.4.1-SNAPSHOT-ranger-tools.tar.gz
-rw-r--r--. 1 root root    905429 9月   6 09:52 ranger-2.4.1-SNAPSHOT-schema-registry-plugin.jar
-rw-r--r--. 1 root root     37663 9月   6 09:51 ranger-2.4.1-SNAPSHOT-solr_audit_conf.tar.gz
-rw-r--r--. 1 root root     40706 9月   6 09:51 ranger-2.4.1-SNAPSHOT-solr_audit_conf.zip
-rw-r--r--. 1 root root  38262105 9月   6 09:50 ranger-2.4.1-SNAPSHOT-solr-plugin.tar.gz
-rw-r--r--. 1 root root  36872196 9月   6 09:52 ranger-2.4.1-SNAPSHOT-sqoop-plugin.tar.gz
-rw-r--r--. 1 root root   6495866 9月   6 09:51 ranger-2.4.1-SNAPSHOT-src.tar.gz
-rw-r--r--. 1 root root  51762378 9月   6 09:50 ranger-2.4.1-SNAPSHOT-storm-plugin.tar.gz
-rw-r--r--. 1 root root  31048308 9月   6 09:51 ranger-2.4.1-SNAPSHOT-tagsync.tar.gz
-rw-r--r--. 1 root root  20129621 9月   6 09:51 ranger-2.4.1-SNAPSHOT-usersync.tar.gz
-rw-r--r--. 1 root root  35802459 9月   6 09:50 ranger-2.4.1-SNAPSHOT-yarn-plugin.tar.gz
-rw-r--r--. 1 root root        14 9月   6 09:38 version