apache atlas 1.2.0部署安装 自用

1.下载

下载地址：
https://www.apache.org/dyn/closer.cgi/atlas/1.2.0/apache-atlas-1.2.0-sources.tar.gz

本文安装的是内嵌的hbase和solr

2.编译打包

由于官网下载的只有源码包，需要单独编译打包，因此需要用到maven进行打包，本文使用的maven版本是3.8.1.
首先解压atlas，之后使用maven进行编译打包

//解压
tar -zxvf apache-atlas-1.2.0-sources.tar.gz -C 指定目录
//进入apache-atlas-1.2.0-sources目录
cd apache-atlas-1.2.0-sources
//maven编译打包,内嵌hbase和solr
mvn clean -DskipTests package -Pdist,embedded-hbase-solr

3 解压安装

// 进入文件夹,解压(文件再distro文件下)
cd distro/target/
//解压
tar -zxvf apache-atlas-1.2.0-server.tar.gz
cd apache-atlas-1.2.0

编辑java环境变量

vim /conf/atlas-env.sh
//编辑下面这一行
export JAVA_HOME=/你的目录/jdk1.8.0_261

4 启动

bin/atlas_start.py

5 导入hive的元数据

1.修改文件ATLAS-APPLICATION.PROPERTIES

cd /opt/module/apache-atlas-1.2.0/conf
vim atlas-application.properties
//增加如下的配置
atlas.hook.hive.synchronous=false
atlas.hook.hive.numRetries=3
atlas.hook.hive.queueSize=10000
atlas.cluster.name=primary

2.将atlas-application.properties配置文件加入到atlas-plugin-classloader-1.2.0.jar中，将atlas-application.properties配置文件拷贝到hive/conf

cd hook/hive
zip -u atlas-plugin-classloader-1.2.0.jar atlas-application.properties
cp /opt/atlas/distro/target/apache-atlas-1.2.0-bin/apache-atlas-2.1.0/conf/atlas-application.properties /opt/hive/conf/

3.在/HIVE/CONF/HIVE-SITE.XML文件中设置ATLAS HOOK

vim /opt/module/hive/conf/hive-site.xml
//增加如下配置
<property>
   <name>hive.exec.post.hooks</name>
   <value>org.apache.atlas.hive.hook.HiveHook</value>
</property>

4.追加HIVE插件相关JAR包

vim hive-env.sh
export HIVE_AUX_JARS_PATH=/opt/atlas/distro/target/apache-atlas-1.2.0-bin/apache-atlas-1.2.0/hook/hive

5.执行脚本导入hive元数据

bin/import-hive.sh

如果成功会看见以下的数据

5 常见错误

1.[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 04:59 min
[INFO] Finished at: 2021-07-19T09:39:33+08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project hive-bridge-shim: Could not resolve dependencies for project org.apache.atlas:hive-bridge-shim:jar:1.2.0: Could not find artifact org.pentaho:pentaho-aggdesigner-algorithm:jar:5.1.5-jhyde in nexus-aliyun (http://maven.aliyun.com/nexus/content/groups/public) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :hive-bridge-shim

解决方法：
pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar这个jar包在apache的maven库中没有，这个是spring库中的，下载这个jar，手动安装到本地的maven库中。
下载地址：
https://public.nexus.pentaho.org/repository/proxy-public-3rd-party-release/org/pentaho/pentaho-aggdesigner-algorithm/5.1.5-jhyde/pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar
安装到本地maven库中：

mvn install:install-file -Dfile=./pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar -DgroupId=org.pentaho -DartifactId=pentaho-aggdesigner-algorithm -Dversion=5.1.5-jhyde -Dpackaging=jar

编译打包过程中如果遇到类似其他jar包无法下载，都可自行下载打包。

2.Exception in thread “main” java.lang.NoClassDefFoundError: com/fasterxml/jackson/module/jaxb/JaxbAnnotationIntrospector
解决方案：找不到对应的jar包
可以参考以下链接：
https://blog.csdn.net/wzj_wp/article/details/108465056

3.未完待续