1.下载
下载地址:
https://www.apache.org/dyn/closer.cgi/atlas/1.2.0/apache-atlas-1.2.0-sources.tar.gz
本文安装的是内嵌的hbase和solr
2.编译打包
由于官网下载的只有源码包,需要单独编译打包,因此需要用到maven进行打包,本文使用的maven版本是3.8.1.
首先解压atlas,之后使用maven进行编译打包
//解压
tar -zxvf apache-atlas-1.2.0-sources.tar.gz -C 指定目录
//进入apache-atlas-1.2.0-sources目录
cd apache-atlas-1.2.0-sources
//maven编译打包,内嵌hbase和solr
mvn clean -DskipTests package -Pdist,embedded-hbase-solr
3 解压安装
// 进入文件夹,解压(文件再distro文件下)
cd distro/target/
//解压
tar -zxvf apache-atlas-1.2.0-server.tar.gz
cd apache-atlas-1.2.0
编辑java环境变量
vim /conf/atlas-env.sh
//编辑下面这一行
export JAVA_HOME=/你的目录/jdk1.8.0_261
4 启动
bin/atlas_start.py
5 导入hive的元数据
1.修改文件ATLAS-APPLICATION.PROPERTIES
cd /opt/module/apache-atlas-1.2.0/conf
vim atlas-application.properties
//增加如下的配置
atlas.hook.hive.synchronous=false
atlas.hook.hive.numRetries=3
atlas.hook.hive.queueSize=10000
atlas.cluster.name=primary
2.将atlas-application.properties配置文件加入到atlas-plugin-classloader-1.2.0.jar中,将atlas-application.properties配置文件拷贝到hive/conf
cd hook/hive
zip -u atlas-plugin-classloader-1.2.0.jar atlas-application.properties
cp /opt/atlas/distro/target/apache-atlas-1.2.0-bin/apache-atlas-2.1.0/conf/atlas-application.properties /opt/hive/conf/
3.在/HIVE/CONF/HIVE-SITE.XML文件中设置ATLAS HOOK
vim /opt/module/hive/conf/hive-site.xml
//增加如下配置
<property>
<name>hive.exec.post.hooks</name>
<value>org.apache.atlas.hive.hook.HiveHook</value>
</property>
4.追加HIVE插件相关JAR包
vim hive-env.sh
export HIVE_AUX_JARS_PATH=/opt/atlas/distro/target/apache-atlas-1.2.0-bin/apache-atlas-1.2.0/hook/hive
5.执行脚本导入hive元数据
bin/import-hive.sh
如果成功会看见以下的数据
5 常见错误
1.[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 04:59 min
[INFO] Finished at: 2021-07-19T09:39:33+08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project hive-bridge-shim: Could not resolve dependencies for project org.apache.atlas:hive-bridge-shim:jar:1.2.0: Could not find artifact org.pentaho:pentaho-aggdesigner-algorithm:jar:5.1.5-jhyde in nexus-aliyun (http://maven.aliyun.com/nexus/content/groups/public) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :hive-bridge-shim
解决方法:
pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar这个jar包在apache的maven库中没有,这个是spring库中的,下载这个jar,手动安装到本地的maven库中。
下载地址:
https://public.nexus.pentaho.org/repository/proxy-public-3rd-party-release/org/pentaho/pentaho-aggdesigner-algorithm/5.1.5-jhyde/pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar
安装到本地maven库中:
mvn install:install-file -Dfile=./pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar -DgroupId=org.pentaho -DartifactId=pentaho-aggdesigner-algorithm -Dversion=5.1.5-jhyde -Dpackaging=jar
编译打包过程中如果遇到类似其他jar包无法下载,都可自行下载打包。
2.Exception in thread “main” java.lang.NoClassDefFoundError: com/fasterxml/jackson/module/jaxb/JaxbAnnotationIntrospector
解决方案:找不到对应的jar包
可以参考以下链接:
https://blog.csdn.net/wzj_wp/article/details/108465056
3.未完待续