【CDH 6.3.1 集成Atlas】

CDH 6.3.1 集成Atlas

环境准备

windows10、IDEA
JDK1.80_201
Zookeeper
kafka
Hbase
solr
hive

Atlas下载

现在linux环境通过git下载:
下面展示一些 内联代码片。
git clone -b release-2.1.0-rc3 https://github.com/apache/atlas.git
然后拷贝回到windows环境下，因为后期要用编译ATLAS源码。

Atlas源码编译

##IDEA导入Maven项目
1、选择OPen

2、选择atlas解压的位置

3、完成后进入编辑页面
4、点开 File -> NewProjects Setup -> Structure… 设置全局 JDK
5、设置 Maven 点击 File -> Settings

##修改pom文件
在repositories中新增以下部分：

   <repository>
        <id>aliyun</id>
        <name>aliyun</name>
        <url>https://maven.aliyun.com/repository/public</url>
    </repository>
    <repository>
        <id>cloudera</id>
        <url>https://repository.cloudera.com/artifactory/cloudera-repos</url>
        <releases>
            <enabled>true</enabled>
        </releases>
        <snapshots>
            <enabled>false</enabled>
        </snapshots>
    </repository>

修改CHD对应的版本
<lucene-solr.version>7.4.0-cdh6.3.1</lucene-solr.version>
<hadoop.version>3.0.0-cdh6.3.1</hadoop.version>
<hbase.version>2.1.0-cdh6.3.1</hbase.version>
<solr.version>7.4.0-cdh6.3.1</solr.version>
<hive.version>2.1.1-cdh6.3.1</hive.version>
<kafka.version>2.2.1-cdh6.3.1</kafka.version>
<kafka.scala.binary.version>2.11</kafka.scala.binary.version>
<zookeeper.version>3.4.5-cdh6.3.1</zookeeper.version>
<sqoop.version>1.4.7-cdh6.3.1</sqoop.version>
兼容HIVE2.1.1版本，修改ATLAS源代码
①.org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java 577行

String catalogName = hiveDB.getCatalogName() != null ? hiveDB.getCatalogName().toLowerCase() : null;
改为：
String catalogName = null;
②.org/apache/atlas/hive/hook/AtlasHiveHookContext.java 81行
this.metastoreHandler = (listenerEvent != null) ? metastoreEvent.getIHMSHandler() : null;
改为：
this.metastoreHandler = null;
修改maven版本号、JDK版本号

##修改兼容版本
##编译
命令:
mvn clean -DskipTests package -Pdist -X
##编译问题分析：
[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.6:install-node-and-npm (install node and npm) on project XXX:
dashboardv2和dashboardv3的pom文件修改

v9.11.1
http://nodejs.org/dist/
http://registry.npmjs.org/npm/-/

##编译成功

Atlas安装

1、解压
tar -zxvf /opt/apache-atlas-2.1.0-server.tar.gz -C /opt/
2、修改配置文件atlas-env.sh
export HBASE_CONF_DIR=/etc/hbase/conf
export ATLAS_SERVER_HEAP=“-Xms15360m -Xmx15360m -XX:MaxNewSize=5120m -XX:MetaspaceSize=100M -XX:MaxMetaspaceSize=512m”
export ATLAS_SERVER_OPTS=“-server -XX:SoftRefLRUPolicyMSPerMB=0 -XX:+CMSClassUnloadingEnabled -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:+PrintTenuringDistribution -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=dumps/atlas_server.hprof -Xloggc:logs/gc-worker.log -verbose:gc -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1m -XX:+PrintGCDetails -XX:+PrintHeapAtGC -XX:+PrintGCTimeStamps”
export MANAGE_LOCAL_HBASE=false
export MANAGE_LOCAL_SOLR=false
export MANAGE_EMBEDDED_CASSANDRA=false
export MANAGE_LOCAL_ELASTICSEARCH=false
3、修改配置文件atlas-application.properties
清空配置文件，直接添加下方文件：

#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

#########  Graph Database Configs  #########

# Graph Database

#Configures the graph database to use.  Defaults to JanusGraph
#atlas.graphdb.backend=org.apache.atlas.repository.graphdb.janus.AtlasJanusGraphDatabase

# Graph Storage
# Set atlas.graph.storage.backend to the correct value for your desired storage
# backend. Possible values:
#
# hbase
# cassandra
# embeddedcassandra - Should only be set by building Atlas with  -Pdist,embedded-cassandra-solr
# berkeleyje
#
# See the configur