Apache atlas 在CDH集群下的搭建问题及措施

     从apache atlas 下载指定版本的源码，通过查看源码，特别是hive/hbase bridge模块，发现其相关依赖为原生的hbase client客户端，直接编译源码，则无法在hbase同步元数据，并且上述模块存在缺少jar的情况。综合同步CDH5.7.1 hive和hbase经验，总结以下修改步骤：

    (1) import-hive.sh 和import-hbase.sh 

    路径 ：addons\hbase-bridge\src\bin ，addons\hive-bridge\src\bin

"${JAVA_BIN}" ${JAVA_PROPERTIES} -Datlas.conf=/etc/hvie/conf  -cp "${CP}" org.apache.atlas.hive.bridge.HiveMetaStoreBridge $IMPORT_ARGS

   添加atlas.conf属性，执行atlas-application.properties所在路径，执行import-hive.sh 和import-hbase.sh时会出现找不到application location的问题。

org.apache.atlas.AtlasException: Failed to load application properties
 at org.apache.atlas.ApplicationProperties.get(ApplicationProperties.java:109)
 at org.apache.atlas.ApplicationProperties.get(ApplicationProperties.java:71)
 at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:119)
Caused by: org.apache.commons.configuration.ConfigurationException: Cannot locate configuration source null
 at org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:259)
 at org.apache.commons.configuration.AbstractFileConfiguration.load(AbstractFileConfiguration.java:238)
 at org.apache.commons.configuration.AbstractFileConfiguration.<init>(AbstractFileConfiguration.java:197)
 at org.apache.commons.configuration.PropertiesConfiguration.<init>(PropertiesConfiguration.java:284)
 at org.apache.atlas.ApplicationProperties.<init>(ApplicationProperties.java:54)
 at org.apache.atlas.ApplicationProperties.get(ApplicationProperties.java:100)

 （2）增加相关依赖

    在  hbase-bridge和hive-bridge的pom.xml中增加以下依赖。

                                        <dependency>
                                            <groupId>com.fasterxml.jackson.jaxrs</groupId>
                                            <artifactId>jackson-jaxrs-json-provider</artifactId>
                                            <version>${jackson.version}</version>
                                        </dependency>
                                        <dependency>
                                            <groupId>com.fasterxml.jackson.module</groupId>
                                            <artifactId>jackson-module-jaxb-annotations</artifactId>
                                            <version>${jackson.version}</version>
                                        </dependency>
                                        <dependency>
                                            <groupId>com.fasterxml.jackson.jaxrs</groupId>
                                            <artifactId>jackson-jaxrs-base</artifactId>
                                            <version>${jackson.version}</version>
                                        </dependency>

  如果不添加上述相关依赖，则会出现找不到相关方法的错误，如

Exception in thread "main" java.lang.NoClassDefFoundError: com/fasterxml/jackson/jaxrs/json/JacksonJaxbJsonProvider
 at org.apache.atlas.AtlasBaseClient.getClient(AtlasBaseClient.java:253)
 at org.apache.atlas.AtlasBaseClient.initializeState(AtlasBaseClient.java:425)
 at org.apache.atlas.AtlasBaseClient.initializeState(AtlasBaseClient.java:420)
 at org.apache.atlas.AtlasBaseClient.<init>(AtlasBaseClient.java:115)
 at org.apache.atlas.AtlasClientV2.<init>(AtlasClientV2.java:77)
 at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:131)
Caused by: java.lang.ClassNotFoundException: com.fasterxml.jackson.jaxrs.json.JacksonJaxbJsonProvider
 at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
 at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
 ... 6 more

 （3） 修改hbase bridge 调用接口

    hbase bridge 采用的hbase-client原生jar,而在CDH中采用的jar包为hbase-client-1.2.0-cdh5.7.1.jar，由于CDK对其中的方法进行了重写，需进行修改。

   修改文件：HBaseAtlasHook.java HBaseBridge.java
  将 getKeepDeletedCells 变为getKeepDeletedCellsAsEnum() 。

 并删除 hbase bridge模块pom.xml中hbase-common的依赖，添加hbase-client-1.2.0-cdh5.7.1.jar  相关依赖，hbase-client-1.2.0-cdh5.7.1.jar 引用的是hbase-common-1.2.0-cdh5.7.1.jar,而非原生 hbase-common-1.2.0.jar, 未删除会出现以下错误。

java.io.IOException: java.lang.reflect.InvocationTargetException
 at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:240)
 at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218)
 at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:119)
 at org.apache.hadoop.hbase.client.HBaseAdmin.checkHBaseAvailable(HBaseAdmin.java:3006)
 at org.apache.atlas.hbase.bridge.HBaseBridge.<init>(HBaseBridge.java:202)
 at org.apache.atlas.hbase.bridge.HBaseBridge.main(HBaseBridge.java:148)
Caused by: java.lang.reflect.InvocationTargetException
 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
 at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
 at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
 at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
 at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)
 ... 5 more
Caused by: java.lang.NoSuchFieldError: HBASE_CLIENT_PREFETCH
 at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:713)
 at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:652)
 ... 10 more

经过上述修改，即可重新编译atlas，进行部署和同步。