Atlas编译安装以及Hive元数据导入

最新推荐文章于 2023-12-17 16:28:48 发布

谷应泰

最新推荐文章于 2023-12-17 16:28:48 发布

阅读量2.9k

点赞数 1

分类专栏：大数据文章标签： hive solr hbase Atlas

本文链接：https://blog.csdn.net/m0_37135879/article/details/122769913

版权

大数据专栏收录该内容

24 篇文章 3 订阅

订阅专栏

一.解压

二.修改pom文件

 645         <npm-for-v2.version>3.10.8</npm-for-v2.version>
 652         <hadoop.version>2.9.2</hadoop.version>

三.创建文件夹,手动放入solr,hbase(不然maven下载时间较长)

/opt/lagou/servers/apache-atlas-sources-1.2.0/distro
[root@Linux122 distro]# mkdir solr
[root@Linux122 distro]# mkdir hbase

guchenfeideMacBook-Pro:Downloads guchenfei$ scp hbase-1.1.2.tar.gz root@Linux122:/opt/lagou/servers/apache-atlas-sources-1.2.0/distro/hbase
guchenfeideMacBook-Pro:Downloads guchenfei$ scp solr-5.5.1.tgz root@Linux122:/opt/lagou/servers/apache-atlas-sources-1.2.0/distro/solr

四.利用maven编译

atlas路径下
export MAVEN_OPTS="-Xms2g -Xmx2g"
mvn clean -DskipTests package -Pdist,embeded-hbase-solr

编译过程中遇到以下错误
报错1

[ERROR] Failed to execute goal on project atlas-graphdb-janus: Could not resolve dependencies for project org.apache.atlas:atlas-graphdb-janus:jar:1.2.0: Failure to find com.sleepycat:je:jar:7.4.5 in http://maven.aliyun.com/nexus/content/groups/public/ was cached in the local repository, resolution will not be reattempted until the update interval of nexus has elapsed or updates are forced -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :atlas-graphdb-janus

在atlas文件下修改配置
pom.xml

 <repository>
        <id>oracleReleases</id>
        <name>Oracle Released Java Packages</name>
        <url>http://download.oracle.com/maven</url>
        <layout>default</layout>
    </repository>

setting.xml

   <mirror>
      <id>oracleReleases</id>
      <mirrorOf>oracleReleases</mirrorOf>
      <name>Oracle Released Java Packages</name>
      <url>http://download.oracle.com/maven</url>
    </mirror>

报错2

[WARNING] The requested profile "embeded-hbase-solr" could not be activated because it does not exist.
[ERROR] Failed to execute goal on project hive-bridge-shim: Could not resolve dependencies for project org.apache.atlas:hive-bridge-shim:jar:1.2.0: Could not find artifact org.pentaho:pentaho-aggdesigner-algorithm:jar:5.1.5-jhyde in nexus (http://maven.aliyun.com/nexus/content/groups/public/) -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :hive-bridge-shim

原因：引用jar包找不到
解决方法：从如下网址下载jar，上传到maven本地库路径

路径:org/pentaho/pentaho-aggdesigner-algorithm/5.1.5-jhyde

https://public.nexus.pentaho.org/content/groups/omni/org/pentaho/pentaho-aggdesigner-algorithm/5.1.5-jhyde/
注意下载正确的jar包
报错3

[WARNING] The requested profile "embeded-hbase-solr" could not be activated because it does not exist.
[ERROR] Failed to execute goal on project sqoop-bridge-shim: Could not resolve dependencies for project org.apache.atlas:sqoop-bridge-shim:jar:1.2.0: Could not find artifact org.apache.sqoop:sqoop:jar:1.4.6.2.3.99.0-195 in nexus (http://maven.aliyun.com/nexus/content/groups/public/) -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :sqoop-bridge-shim

同样下载jar包放到以下路径

guchenfeideMacBook-Pro:Downloads guchenfei$ scp sqoop-1.4.6.2.3.99.0-195.jar root@Linux122:/root/.m2/repository/org/apache/sqoop/sqoop/1.4.6.2.3.99.0-195

编译成功

[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Atlas Server Build Tools 1.0 ................ SUCCESS [  6.071 s]
[INFO] apache-atlas 1.2.0 ................................. SUCCESS [ 23.063 s]
[INFO] Apache Atlas Test Utility Tools 1.2.0 .............. SUCCESS [ 18.065 s]
[INFO] Apache Atlas Integration 1.2.0 ..................... SUCCESS [ 13.976 s]
[INFO] Apache Atlas Common 1.2.0 .......................... SUCCESS [  4.861 s]
[INFO] Apache Atlas Client 1.2.0 .......................... SUCCESS [  0.704 s]
[INFO] atlas-client-common 1.2.0 .......................... SUCCESS [  2.351 s]
[INFO] atlas-client-v1 1.2.0 .............................. SUCCESS [  3.047 s]
[INFO] Apache Atlas Server API 1.2.0 ...................... SUCCESS [  3.021 s]
[INFO] Apache Atlas Notification 1.2.0 .................... SUCCESS [  8.389 s]
[INFO] atlas-client-v2 1.2.0 .............................. SUCCESS [  2.216 s]
[INFO] Apache Atlas Graph Database Projects 1.2.0 ......... SUCCESS [  0.511 s]
[INFO] Apache Atlas Graph Database API 1.2.0 .............. SUCCESS [  2.738 s]
[INFO] Graph Database Common Code 1.2.0 ................... SUCCESS [  2.501 s]
[INFO] Apache Atlas JanusGraph DB Impl 1.2.0 .............. SUCCESS [ 17.061 s]
[INFO] Apache Atlas Graph Database Implementation Dependencies 1.2.0 SUCCESS [  3.629 s]
[INFO] Shaded version of Apache hbase client 1.2.0 ........ SUCCESS [ 13.936 s]
[INFO] Shaded version of Apache hbase server 1.2.0 ........ SUCCESS [ 36.452 s]
[INFO] Apache Atlas Authorization 1.2.0 ................... SUCCESS [  3.678 s]
[INFO] Apache Atlas Repository 1.2.0 ...................... SUCCESS [ 23.099 s]
[INFO] Apache Atlas UI 1.2.0 .............................. SUCCESS [04:08 min]
[INFO] Apache Atlas Web Application 1.2.0 ................. SUCCESS [06:59 min]
[INFO] Apache Atlas Documentation 1.2.0 ................... SUCCESS [ 33.017 s]
[INFO] Apache Atlas FileSystem Model 1.2.0 ................ SUCCESS [ 18.675 s]
[INFO] Apache Atlas Plugin Classloader 1.2.0 .............. SUCCESS [  2.905 s]
[INFO] Apache Atlas Hive Bridge Shim 1.2.0 ................ SUCCESS [ 13.538 s]
[INFO] Apache Atlas Hive Bridge 1.2.0 ..................... SUCCESS [ 43.045 s]
[INFO] Apache Atlas Falcon Bridge Shim 1.2.0 .............. SUCCESS [  9.073 s]
[INFO] Apache Atlas Falcon Bridge 1.2.0 ................... SUCCESS [ 14.093 s]
[INFO] Apache Atlas Sqoop Bridge Shim 1.2.0 ............... SUCCESS [ 11.648 s]
[INFO] Apache Atlas Sqoop Bridge 1.2.0 .................... SUCCESS [ 16.375 s]
[INFO] Apache Atlas Storm Bridge Shim 1.2.0 ............... SUCCESS [ 18.567 s]
[INFO] Apache Atlas Storm Bridge 1.2.0 .................... SUCCESS [ 26.652 s]
[INFO] Apache Atlas Hbase Bridge Shim 1.2.0 ............... SUCCESS [  9.138 s]
[INFO] Apache Atlas Hbase Bridge 1.2.0 .................... SUCCESS [ 33.532 s]
[INFO] Apache Atlas Kafka Bridge 1.2.0 .................... SUCCESS [ 12.331 s]
[INFO] Apache Atlas Distribution 1.2.0 .................... SUCCESS [01:53 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  20:40 min
[INFO] Finished at: 2022-01-30T19:43:07+08:00
[INFO] ------------------------------------------------------------------------
[WARNING] The requested profile "embeded-hbase-solr" could not be activated because it does not exist.
[root@Linux122 apache-atlas-sources-1.2.0]# ls

五.安装

进入编译产物目录下

/opt/lagou/servers/apache-atlas-sources-1.2.0/distro/target

将apache-atlas-1.2.0-bin.tar.gz copy到安装目录

[root@Linux122 target]# cp apache-atlas-1.2.0-bin.tar.gz /opt/lagou/servers/

解压

[root@Linux122 servers]# tar -zxvf apache-atlas-1.2.0-bin.tar.gz
[root@Linux122 servers]# mv apache-atlas-1.2.0 atlas-1.2.0

六.配置环境变量

export ATLAS_HOME=/opt/lagou/servers/atlas-1.2.0
export PATH=$PATH:$ATLAS_HOME/bin


/opt/lagou/servers/atlas-1.2.0/conf
atlas-env.sh
添加hbase的配置
#HBase配置文件路径
export HBASE_CONF_DIR=/opt/lagou/servers/atlas-1.2.0/conf/hbase/hbase-site.xml
记得在对应路径下将模版文件改成配置文件

七.启动服务

执行bin下脚本
[root@Linux122 bin]# ./atlas_start.py
starting atlas on host localhost
starting atlas on port 21000
.....................................................................................................
Apache Atlas Server started!!!

查看后台进程

[root@Linux122 bin]# ps -ef | grep atlas
root       7576      1  2 21:58 pts/0    00:00:13 /opt/lagou/servers/jdk1.8.0_301/bin/java -Datlas.log.dir=/opt/lagou/servers/atlas-1.2.0/logs -Datlas.log.file=application.log -Datlas.home=/opt/lagou/servers/atlas-1.2.0 -Datlas.conf=/opt/lagou/servers/atlas-1.2.0/conf -Xmx1024m -Dlog4j.configuration=atlas-log4j.xml -Djava.net.preferIPv4Stack=true -server -classpath /opt/lagou/servers/atlas-1.2.0/conf:/opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/classes:/opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/*:/opt/lagou/servers/atlas-1.2.0/libext/*:/opt/lagou/servers/atlas-1.2.0/conf/hbase/hbase-site.xml org.apache.atlas.Atlas -app /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas
root       7954   3409  1 22:06 pts/0    00:00:00 grep --color=auto atlas

通过查看启动的进程发现缺少其他服务
于是重新进行了编译

mvn clean -DskipTests install
mvn clean -DskipTests package -Pdist,embedded-hbase-solr

剩余工作一样
重新启动服务

[root@Linux122 bin]# ./atlas_start.py
configured for local hbase.
hbase started.
configured for local solr.
solr started.
setting up solr collections...
starting atlas on host localhost
starting atlas on port 21000
..............................................................................................................................................................................................................................................................................................................
Apache Atlas Server started!!!

查看进程

[root@Linux122 bin]# ps -ef | grep atlas
root       7013      1  0 16:59 pts/0    00:00:00 bash /opt/lagou/servers/atlas-1.2.0/hbase/bin/hbase-daemon.sh --config /opt/lagou/servers/atlas-1.2.0/hbase/conf foreground_start master
root       7107      1  5 16:59 pts/0    00:00:23 /opt/lagou/servers/jdk1.8.0_301/bin/java -server -Xms512m -Xmx512m -XX:NewRatio=3 -XX:SurvivorRatio=4 -XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=8 -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:ConcGCThreads=4 -XX:ParallelGCThreads=4 -XX:+CMSScavengeBeforeRemark -XX:PretenureSizeThreshold=64m -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=50 -XX:CMSMaxAbortablePrecleanTime=6000 -XX:+CMSParallelRemarkEnabled -XX:+ParallelRefProcEnabled -verbose:gc -XX:+PrintHeapAtGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:/opt/lagou/servers/atlas-1.2.0/solr/server/logs/solr_gc.log -DzkClientTimeout=15000 -DzkHost=localhost:2181 -Djetty.port=9838 -DSTOP.PORT=8838 -DSTOP.KEY=solrrocks -Duser.timezone=UTC -Djetty.home=/opt/lagou/servers/atlas-1.2.0/solr/server -Dsolr.solr.home=/opt/lagou/servers/atlas-1.2.0/solr/server/solr -Dsolr.install.dir=/opt/lagou/servers/atlas-1.2.0/solr -Xss256k -XX:OnOutOfMemoryError=/opt/lagou/servers/atlas-1.2.0/solr/bin/oom_solr.sh 9838 /opt/lagou/servers/atlas-1.2.0/solr/server/logs -jar start.jar --module=http
root       7767      1 12 17:00 pts/0    00:00:47 /opt/lagou/servers/jdk1.8.0_301/bin/java -Datlas.log.dir=/opt/lagou/servers/atlas-1.2.0/logs -Datlas.log.file=application.log -Datlas.home=/opt/lagou/servers/atlas-1.2.0 -Datlas.conf=/opt/lagou/servers/atlas-1.2.0/conf -Xmx1024m -Dlog4j.configuration=atlas-log4j.xml -Djava.net.preferIPv4Stack=true -server -classpath /opt/lagou/servers/atlas-1.2.0/conf:/opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/classes:/opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/*:/opt/lagou/servers/atlas-1.2.0/libext/*:/opt/lagou/servers/atlas-1.2.0/hbase/conf org.apache.atlas.Atlas -app /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas
root       8796   3249  0 17:06 pts/0    00:00:00 grep --color=auto atlas

启动成功
访问网址

http://Linux122:21000/login.jsp

在这里插入图片描述
用缺省的账户密码:admin,admin登录

自定义用户名和密码

/opt/lagou/servers/atlas-1.2.0/conf
[root@Linux122 conf]# more users-credentials.properties
#username=group::sha256-password
admin=ADMIN::8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918
rangertagsync=RANGER_TAG_SYNC::e3f67240f5117d1753c940dae9eea772d36ed5fe9bd9c94a300e4041
3f1afb9d

其中

ADMIN::8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918这个是账号和密码
[root@Linux122 conf]# echo -n "admin" | sha256sum
8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918  -

修改密码可以将自己的自定义密码改成计算后的sha256sum值

八.Hive血缘关系导入

1.配置hive环境变量,将/opt/lagou/servers/atlas-1.2.0/conf/atlas-application.properties拷贝到/opt/lagou/servers/hive-2.3.7/conf让hive元数据变更时候告诉atlas

软连接和硬拷贝都可以

ln -s /opt/lagou/servers/atlas-1.2.0/conf/atlas-application.properties /opt/lagou/servers/hive-2.3.7/conf

2.将json解析相关的jar包拷贝

/opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib
[root@Linux122 lib]# ls jackson*
jackson-annotations-2.9.9.jar       jackson-jaxrs-base-2.9.9.jar
jackson-core-2.9.9.jar              jackson-jaxrs-json-provider-2.9.9.jar
jackson-core-asl-1.9.2.jar          jackson-mapper-asl-1.9.2.jar
jackson-databind-2.7.8.jar          jackson-module-jaxb-annotations-2.9.9.jar
jackson-dataformat-smile-2.5.4.jar

将以下jar包软连接到/opt/lagou/servers/atlas-1.2.0/hook/hive/atlas-hive-plugin-impl

[root@Linux122 lib]# ln -s /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/jackson-jaxrs-base-2.9.9.jar /opt/lagou/servers/atlas-1.2.0/hook/hive/atlas-hive-plugin-impl/jackson-jaxrs-base-2.9.9.jar
[root@Linux122 lib]# ln -s /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/jackson-jaxrs-json-provider-2.9.9.jar /opt/lagou/servers/atlas-1.2.0/hook/hive/atlas-hive-plugin-impl/jackson-jaxrs-json-provider-2.9.9.jar
[root@Linux122 lib]# ln -s /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/jackson-module-jaxb-annotations-2.9.9.jar /opt/lagou/servers/atlas-1.2.0/hook/hive/atlas-hive-plugin-impl/jackson-module-jaxb-annotations-2.9.9.jar
[root@Linux122 lib]# cd /opt/lagou/servers/atlas-1.2.0/hook/hive/atlas-hive-plugin-impl/
[root@Linux122 atlas-hive-plugin-impl]# ls
atlas-client-common-1.2.0.jar   jackson-core-2.9.9.jar
atlas-client-v1-1.2.0.jar       jackson-databind-2.9.9.jar
atlas-client-v2-1.2.0.jar       jackson-jaxrs-base-2.9.9.jar
atlas-common-1.2.0.jar          jackson-jaxrs-json-provider-2.9.9.jar
atlas-intg-1.2.0.jar            jackson-module-jaxb-annotations-2.9.9.jar
atlas-notification-1.2.0.jar    jersey-json-1.19.jar
commons-configuration-1.10.jar  jersey-multipart-1.19.jar
hbase-common-1.1.2.jar          jsr311-api-1.1.jar
hbase-server-1.1.2.jar          kafka_2.11-1.0.0.jar
hdfs-model-1.2.0.jar            kafka-clients-1.0.0.jar
hive-bridge-1.2.0.jar           scala-library-2.11.12.jar
jackson-annotations-2.9.9.jar
[root@Linux122 atlas-hive-plugin-impl]# ll
total 21996
-rw-r--r--. 1 root root   37040 Feb  1 16:36 atlas-client-common-1.2.0.jar
-rw-r--r--. 1 root root   41829 Feb  1 16:36 atlas-client-v1-1.2.0.jar
-rw-r--r--. 1 root root   19534 Feb  1 16:36 atlas-client-v2-1.2.0.jar
-rw-r--r--. 1 root root   70743 Feb  1 16:36 atlas-common-1.2.0.jar
-rw-r--r--. 1 root root  484994 Feb  1 16:36 atlas-intg-1.2.0.jar
-rw-r--r--. 1 root root   59523 Feb  1 16:36 atlas-notification-1.2.0.jar
-rw-r--r--. 1 root root  362679 Jan 30 14:52 commons-configuration-1.10.jar
-rw-r--r--. 1 root root  530078 Jan 30 15:17 hbase-common-1.1.2.jar
-rw-r--r--. 1 root root 3978343 Jan 30 15:18 hbase-server-1.1.2.jar
-rw-r--r--. 1 root root   10869 Feb  1 16:42 hdfs-model-1.2.0.jar
-rw-r--r--. 1 root root   80919 Feb  1 16:43 hive-bridge-1.2.0.jar
-rw-r--r--. 1 root root   66897 Jan 30 14:53 jackson-annotations-2.9.9.jar
-rw-r--r--. 1 root root  325632 Jan 30 14:53 jackson-core-2.9.9.jar
-rw-r--r--. 1 root root 1348108 Jan 30 14:53 jackson-databind-2.9.9.jar
lrwxrwxrwx. 1 root root      91 Feb  1 18:34 jackson-jaxrs-base-2.9.9.jar -> /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/jackson-jaxrs-base-2.9.9.jar
lrwxrwxrwx. 1 root root     100 Feb  1 18:35 jackson-jaxrs-json-provider-2.9.9.jar -> /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/jackson-jaxrs-json-provider-2.9.9.jar
lrwxrwxrwx. 1 root root     104 Feb  1 18:37 jackson-module-jaxb-annotations-2.9.9.jar -> /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/jackson-module-jaxb-annotations-2.9.9.jar
-rw-r--r--. 1 root root  165345 Jan 30 19:08 jersey-json-1.19.jar
-rw-r--r--. 1 root root   53275 Jan 30 14:53 jersey-multipart-1.19.jar
-rw-r--r--. 1 root root   45927 Jan 30 19:08 jsr311-api-1.1.jar
-rw-r--r--. 1 root root 7452292 Jan 30 14:53 kafka_2.11-1.0.0.jar
-rw-r--r--. 1 root root 1591338 Jan 30 14:53 kafka-clients-1.0.0.jar
-rw-r--r--. 1 root root 5749423 Jan 30 19:08 scala-library-2.11.12.jar
[root@Linux122 atlas-hive-plugin-impl]#

3.修改hive配置

/opt/lagou/servers/hive-2.3.7/conf/hive-site.xml
增加配置

<property>
  <name>hive.exec.post.hooks</name>
  <value>org.apache.atlas.hive.hook.HiveHook</value>
</property>

/opt/lagou/servers/hive-2.3.7/conf/hive-env.sh
新增配置,告诉jar包地址

export HIVE_AUX_JARS_PATH=/opt/lagou/servers/atlas-1.2.0/hook/hive

4.批量导入hive元数据

执行以下脚本

/opt/lagou/servers/atlas-1.2.0/bin/import-hive.sh

错误一

[root@Linux122 bin]# import-hive.sh
Using Hive configuration directory [/opt/lagou/servers/hive-2.3.7/conf]
Log file for import is /opt/lagou/servers/atlas-1.2.0/logs/import-hive.log
2022-02-01T18:57:51,339 INFO [main] org.apache.atlas.ApplicationProperties - Looking for atlas-application.properties in classpath
2022-02-01T18:57:51,350 INFO [main] org.apache.atlas.ApplicationProperties - Loading atlas-application.properties from file:/opt/lagou/servers/hive-2.3.7/conf/atlas-application.properties
2022-02-01T18:57:51,608 INFO [main] org.apache.atlas.ApplicationProperties - Property (set to default) atlas.graph.cache.db-cache = true
2022-02-01T18:57:51,608 INFO [main] org.apache.atlas.ApplicationProperties - Property (set to default) atlas.graph.cache.db-cache-clean-wait = 20
2022-02-01T18:57:51,608 INFO [main] org.apache.atlas.ApplicationProperties - Property (set to default) atlas.graph.cache.db-cache-size = 0.5
2022-02-01T18:57:51,608 INFO [main] org.apache.atlas.ApplicationProperties - Property (set to default) atlas.graph.cache.tx-cache-size = 15000
2022-02-01T18:57:51,608 INFO [main] org.apache.atlas.ApplicationProperties - Property (set to default) atlas.graph.cache.tx-dirty-size = 120
Enter username for atlas :- admin
Enter password for atlas :- 
Exception in thread "main" java.lang.NoClassDefFoundError: com/sun/jersey/core/util/FeaturesAndProperties
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
	at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:131)
Caused by: java.lang.ClassNotFoundException: com.sun.jersey.core.util.FeaturesAndProperties
	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	... 13 more
Failed to import Hive Meta Data!!!
[root@Linux122 bin]#

注意启动服务前将hdfs-yarn metastore 服务起来保证hive能正常使用

错误二

Exception in thread "main" java.lang.NoClassDefFoundError: com/sun/jersey/core/util/FeaturesAndProperties
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
	at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:131)
Caused by: java.lang.ClassNotFoundException: com.sun.jersey.core.util.FeaturesAndProperties
	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	... 13 more
Failed to import Hive Meta Data!!!

通过网上查找说缺少以下jar包

jersey-core-1.9.jar
jersey-server-1.9.jar
jersey-client-1.9.jar

我们在上边配置引入相关jar包时候发现有以上包

[root@Linux122 lib]# ls jersey*
jersey-client-1.19.jar  jersey-json-1.9.jar        jersey-server-1.19.jar   jersey-spring-1.19.4.jar
jersey-core-1.19.jar    jersey-multipart-1.19.jar  jersey-servlet-1.19.jar
[root@Linux122 lib]# pwd
/opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib
[root@Linux122 lib]# ln -s /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/jersey-core-1.19.jar /opt/lagou/servers/atlas-1.2.0/hook/hive/atlas-hive-plugin-impl/jersey-core-1.19.jar
[root@Linux122 lib]# ln -s /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/jersey-server-1.19.jar /opt/lagou/servers/atlas-1.2.0/hook/hive/atlas-hive-plugin-impl/jersey-server-1.19.jar
[root@Linux122 lib]# ln -s /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/jersey-client-1.19.jar /opt/lagou/servers/atlas-1.2.0/hook/hive/atlas-hive-plugin-impl/jersey-client-1.19.jar

错误三

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/security/authentication/client/ConnectionConfigurator
	at org.apache.atlas.AtlasBaseClient.getClient(AtlasBaseClient.java:287)
	at org.apache.atlas.AtlasBaseClient.initializeState(AtlasBaseClient.java:453)
	at org.apache.atlas.AtlasBaseClient.initializeState(AtlasBaseClient.java:448)
	at org.apache.atlas.AtlasBaseClient.<init>(AtlasBaseClient.java:132)
	at org.apache.atlas.AtlasClientV2.<init>(AtlasClientV2.java:82)
	at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:131)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.security.authentication.client.ConnectionConfigurator
	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	... 6 more
Failed to import Hive Meta Data!!!

继续寻找jar包

ln -s /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/hadoop-auth-2.9.2.jar /opt/lagou/servers/atlas-1.2.0/hook/hive/atlas-hive-plugin-impl/hadoop-auth-2.9.2.jar

错误四

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
	at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:138)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.conf.Configuration
	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	... 13 more
Failed to import Hive Meta Data!!!

看样子缺少hadoop依赖,我将他们都软连接了

[root@Linux122 lib]# ln -s /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/hadoop-annotations-2.9.2.jar /opt/lagou/servers/atlas-1.2.0/hook/hive/atlas-hive-plugin-impl/hadoop-annotations-2.9.2.jar
[root@Linux122 lib]# ln -s /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/hadoop-common-2.9.2.jar /opt/lagou/servers/atlas-1.2.0/hook/hive/atlas-hive-plugin-impl/hadoop-common-2.9.2.jar
[root@Linux122 lib]# ln -s /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/hadoop-hdfs-2.9.2.jar /opt/lagou/servers/atlas-1.2.0/hook/hive/atlas-hive-plugin-impl/hadoop-hdfs-2.9.2.jar
[root@Linux122 lib]# ln -s /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/hadoop-hdfs-client-2.9.2.jar /opt/lagou/servers/atlas-1.2.0/hook/hive/atlas-hive-plugin-impl/hadoop-hdfs-client-2.9.2.jar

错误五

Exception in thread "main" java.lang.NoClassDefFoundError: com/ctc/wstx/io/InputBootstrapper
	at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:138)
Caused by: java.lang.ClassNotFoundException: com.ctc.wstx.io.InputBootstrapper
	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	... 1 more
Failed to import Hive Meta Data!!!

缺少如下jar包

[root@Linux122 lib]# ln -s /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/woodstox-core-5.0.3.jar /opt/lagou/servers/atlas-1.2.0/hook/hive/atlas-hive-plugin-impl/woodstox-core-5.0.3.jar

错误六

Exception in thread "main" java.lang.NoClassDefFoundError: org/codehaus/stax2/XMLInputFactory2
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
	at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	at org.apache.hadoop.conf.Configuration.<clinit>(Configuration.java:312)
	at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:138)
Caused by: java.lang.ClassNotFoundException: org.codehaus.stax2.XMLInputFactory2
	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	... 14 more
Failed to import Hive Meta Data!!!
[root@Linux122 bin]#

继续

[root@Linux122 lib]# ln -s /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/stax2-api-3.1.4.jar /opt/lagou/servers/atlas-1.2.0/hook/hive/atlas-hive-plugin-impl/stax2-api-3.1.4.jar

错误六

2022-02-02T10:58:54,673 INFO [main] hive.metastore - Connected to metastore.
2022-02-02T10:58:54,996 INFO [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Importing Hive metadata
2022-02-02T10:58:55,022 INFO [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Found 11 databases
2022-02-02T10:58:55,339 ERROR [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Import failed
com.sun.jersey.api.client.ClientHandlerException: java.net.ConnectException: Connection refused (Connection refused)
	at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:155) ~[jersey-client-1.19.jar:1.19]
	at com.sun.jersey.api.client.filter.HTTPBasicAuthFilter.handle(HTTPBasicAuthFilter.java:105) ~[jersey-client-1.19.jar:1.19]
	at com.sun.jersey.api.client.Client.handle(Client.java:652) ~[jersey-client-1.19.jar:1.19]
	at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682) ~[jersey-client-1.19.jar:1.19]
	at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) ~[jersey-client-1.19.jar:1.19]
	at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:634) ~[jersey-client-1.19.jar:1.19]
	at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:382) ~[atlas-client-common-1.2.0.jar:1.2.0]
	at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:353) ~[atlas-client-common-1.2.0.jar:1.2.0]
	at org.apache.atlas.AtlasBaseClient.callAPI(AtlasBaseClient.java:246) ~[atlas-client-common-1.2.0.jar:1.2.0]
	at org.apache.atlas.AtlasClientV2.getEntityByAttribute(AtlasClientV2.java:285) ~[atlas-client-v2-1.2.0.jar:1.2.0]
	at org.apache.atlas.AtlasClientV2.getEntityByAttribute(AtlasClientV2.java:276) ~[atlas-client-v2-1.2.0.jar:1.2.0]
	at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.findEntity(HiveMetaStoreBridge.java:780) ~[hive-bridge-1.2.0.jar:1.2.0]
	at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.findDatabase(HiveMetaStoreBridge.java:745) ~[hive-bridge-1.2.0.jar:1.2.0]
	at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerDatabase(HiveMetaStoreBridge.java:399) ~[hive-bridge-1.2.0.jar:1.2.0]
	at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importDatabases(HiveMetaStoreBridge.java:281) ~[hive-bridge-1.2.0.jar:1.2.0]
	at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importHiveMetadata(HiveMetaStoreBridge.java:251) ~[hive-bridge-1.2.0.jar:1.2.0]
	at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:168) [hive-bridge-1.2.0.jar:1.2.0]
Caused by: java.net.ConnectException: Connection refused (Connection refused)
	at java.net.PlainSocketImpl.socketConnect(Native Method) ~[?:1.8.0_301]
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:476) ~[?:1.8.0_301]
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:218) ~[?:1.8.0_301]
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:200) ~[?:1.8.0_301]
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:394) ~[?:1.8.0_301]
	at java.net.Socket.connect(Socket.java:606) ~[?:1.8.0_301]
	at sun.net.NetworkClient.doConnect(NetworkClient.java:175) ~[?:1.8.0_301]
	at sun.net.www.http.HttpClient.openServer(HttpClient.java:463) ~[?:1.8.0_301]
	at sun.net.www.http.HttpClient.openServer(HttpClient.java:558) ~[?:1.8.0_301]
	at sun.net.www.http.HttpClient.<init>(HttpClient.java:242) ~[?:1.8.0_301]
	at sun.net.www.http.HttpClient.New(HttpClient.java:339) ~[?:1.8.0_301]
	at sun.net.www.http.HttpClient.New(HttpClient.java:357) ~[?:1.8.0_301]
	at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1226) ~[?:1.8.0_301]
	at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1162) ~[?:1.8.0_301]
	at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1056) ~[?:1.8.0_301]
	at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:990) ~[?:1.8.0_301]
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1570) ~[?:1.8.0_301]
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498) ~[?:1.8.0_301]
	at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480) ~[?:1.8.0_301]
	at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:253) ~[jersey-client-1.19.jar:1.19]
	at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:153) ~[jersey-client-1.19.jar:1.19]
	... 16 more
Failed to import Hive Meta Data!!!

通过查资料看官方文档
重新启动Atlas

atlas_stop.py

Running Apache Atlas with Local Apache HBase & Apache Solr
To run Apache Atlas with local Apache HBase & Apache Solr instances that are started/stopped along with Atlas start/stop, run following commands:

export MANAGE_LOCAL_HBASE=true
export MANAGE_LOCAL_SOLR=true
bin/atlas_start.py

检查服务启动状态

[root@Linux122 bin]# curl -u admin:admin http://localhost:21000/api/atlas/admin/version
{"Description":"Metadata Management and Data Governance Platform over Hadoop","Revision":"release","Version":"1.2.0","Name":"apache-atlas"}

错误七

2022-02-02T13:42:06,735 ERROR [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Import failed
com.sun.jersey.api.client.ClientHandlerException: java.net.SocketTimeoutException: Read timed out
	at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:155) ~[jersey-client-1.19.jar:1.19]
	at com.sun.jersey.api.client.filter.HTTPBasicAuthFilter.handle(HTTPBasicAuthFilter.java:105) ~[jersey-client-1.19.jar:1.19]
	at com.sun.jersey.api.client.Client.handle(Client.java:652) ~[jersey-client-1.19.jar:1.19]
	at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682) ~[jersey-client-1.19.jar:1.19]
	at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) ~[jersey-client-1.19.jar:1.19]
	at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:634) ~[jersey-client-1.19.jar:1.19]
	at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:382) ~[atlas-client-common-1.2.0.jar:1.2.0]
	at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:353) ~[atlas-client-common-1.2.0.jar:1.2.0]
	at org.apache.atlas.AtlasBaseClient.callAPI(AtlasBaseClient.java:246) ~[atlas-client-common-1.2.0.jar:1.2.0]
	at org.apache.atlas.AtlasClientV2.getEntityByAttribute(AtlasClientV2.java:285) ~[atlas-client-v2-1.2.0.jar:1.2.0]
	at org.apache.atlas.AtlasClientV2.getEntityByAttribute(AtlasClientV2.java:276) ~[atlas-client-v2-1.2.0.jar:1.2.0]
	at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.findEntity(HiveMetaStoreBridge.java:780) ~[hive-bridge-1.2.0.jar:1.2.0]
	at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.findDatabase(HiveMetaStoreBridge.java:745) ~[hive-bridge-1.2.0.jar:1.2.0]
	at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerDatabase(HiveMetaStoreBridge.java:399) ~[hive-bridge-1.2.0.jar:1.2.0]
	at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importDatabases(HiveMetaStoreBridge.java:281) ~[hive-bridge-1.2.0.jar:1.2.0]
	at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importHiveMetadata(HiveMetaStoreBridge.java:251) ~[hive-bridge-1.2.0.jar:1.2.0]
	at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:168) [hive-bridge-1.2.0.jar:1.2.0]
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method) ~[?:1.8.0_301]
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) ~[?:1.8.0_301]
	at java.net.SocketInputStream.read(SocketInputStream.java:171) ~[?:1.8.0_301]
	at java.net.SocketInputStream.read(SocketInputStream.java:141) ~[?:1.8.0_301]
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) ~[?:1.8.0_301]
	at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) ~[?:1.8.0_301]
	at java.io.BufferedInputStream.read(BufferedInputStream.java:345) ~[?:1.8.0_301]
	at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:735) ~[?:1.8.0_301]
	at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678) ~[?:1.8.0_301]
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1593) ~[?:1.8.0_301]
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498) ~[?:1.8.0_301]
	at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480) ~[?:1.8.0_301]
	at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:253) ~[jersey-client-1.19.jar:1.19]
	at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:153) ~[jersey-client-1.19.jar:1.19]
	... 16 more
Failed to import Hive Meta Data!!!

通过查资料发现是需要配置超时时间

/opt/lagou/servers/atlas-1.2.0/conf/atlas-application.properties
增加如下配置
atlas.hook.hive.keepAliveTime=1000000

重新导入元数据

[root@Linux122 bin]# import-hive.sh
Using Hive configuration directory [/opt/lagou/servers/hive-2.3.7/conf]
Log file for import is /opt/lagou/servers/atlas-1.2.0/logs/import-hive.log
2022-02-02T14:30:48,137 INFO [main] org.apache.atlas.ApplicationProperties - Looking for atlas-application.properties in classpath
2022-02-02T14:30:48,144 INFO [main] org.apache.atlas.ApplicationProperties - Loading atlas-application.properties from file:/opt/lagou/servers/hive-2.3.7/conf/atlas-application.properties
2022-02-02T14:30:48,317 INFO [main] org.apache.atlas.ApplicationProperties - Property (set to default) atlas.graph.cache.db-cache = true
2022-02-02T14:30:48,317 INFO [main] org.apache.atlas.ApplicationProperties - Property (set to default) atlas.graph.cache.db-cache-clean-wait = 20
2022-02-02T14:30:48,317 INFO [main] org.apache.atlas.ApplicationProperties - Property (set to default) atlas.graph.cache.db-cache-size = 0.5
2022-02-02T14:30:48,317 INFO [main] org.apache.atlas.ApplicationProperties - Property (set to default) atlas.graph.cache.tx-cache-size = 15000
2022-02-02T14:30:48,318 INFO [main] org.apache.atlas.ApplicationProperties - Property (set to default) atlas.graph.cache.tx-dirty-size = 120
Enter username for atlas :- admin
Enter password for atlas :- 
2022-02-02T14:30:55,028 INFO [main] org.apache.atlas.AtlasBaseClient - Client has only one service URL, will use that for all actions: http://localhost:21000
2022-02-02T14:30:55,218 INFO [main] org.apache.hadoop.hive.conf.HiveConf - Found configuration file file:/opt/lagou/servers/hive-2.3.7/conf/hive-site.xml
2022-02-02T14:30:57,861 WARN [main] org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2022-02-02T14:30:58,003 INFO [main] hive.metastore - Trying to connect to metastore with URI thrift://Linux121:9083
2022-02-02T14:30:58,108 INFO [main] hive.metastore - Opened a connection to metastore, current connections: 1
2022-02-02T14:30:58,240 INFO [main] hive.metastore - Connected to metastore.
2022-02-02T14:30:58,801 INFO [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Importing Hive metadata
2022-02-02T14:30:58,853 INFO [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Found 11 databases
2022-02-02T14:31:00,430 INFO [main] org.apache.atlas.AtlasBaseClient - method=GET path=api/atlas/v2/entity/uniqueAttribute/type/ contentType=application/json; charset=UTF-8 accept=application/json status=200
2022-02-02T14:31:01,109 INFO [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Database ads is already registered - id=897fe37d-7eaa-499f-8843-690003f22e9b. Updating it.
2022-02-02T14:31:01,449 INFO [main] org.apache.atlas.AtlasBaseClient - method=POST path=api/atlas/v2/entity/ contentType=application/json; charset=UTF-8 accept=application/json status=200
2022-02-02T14:31:01,465 INFO [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Updated hive_db entity: name=ads@primary, guid=897fe37d-7eaa-499f-8843-690003f22e9b
2022-02-02T14:31:02,280 INFO [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Found 12 tables to import in database ads
2022-02-02T14:31:03,605 INFO [main] org.apache.atlas.AtlasBaseClient - method=GET path=api/atlas/v2/entity/uniqueAttribute/type/ contentType=application/json; charset=UTF-8 accept=application/json status=200
2022-02-02T14:31:03,607 INFO [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Table ads.ads_active_member_frequenty_3of_recent6_cnt is already registered with id addb017c-5389-44c1-9b39-afacf92208d0. Updating entity.
2022-02-02T14:31:05,365 INFO [main] org.apache.atlas.AtlasBaseClient - method=POST path=api/atlas/v2/entity/ contentType=application/json; charset=UTF-8 accept=application/json status=200
2022-02-02T14:31:05,366 INFO [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Updated hive_table entity: name=ads.ads_active_member_frequenty_3of_recent6_cnt@primary, guid=addb017c-5389-44c1-9b39-afacf92208d0
2022-02-02T14:31:05,798 INFO [main] org.apache.atlas.AtlasBaseClient - method=GET path=api/atlas/v2/entity/uniqueAttribute/type/ contentType=application/json; charset=UTF-8 accept=application/json status=200
2022-02-02T14:31:05,801 INFO [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Table ads.ads_ad_show is already registered with id ccd6ab90-fed1-4530-b03d-0b76e56950f9. Updating entity.
2022-02-02T14:31:06,912 INFO [main] org.apache.atlas.AtlasBaseClient - method=POST path=api/atlas/v2/entity/ contentType=application/json; charset=UTF-8 accept=application/json status=200
2022-02-02T14:31:06,925 INFO [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Updated hive_table entity: name=ads.ads_ad_show@primary, guid=ccd6ab90-fed1-4530-b03d-0b76e56950f9
2022-02-02T14:31:07,303 INFO [main] org.apache.atlas.AtlasBaseClient - method=GET path=api/atlas/v2/entity/uniqueAttribute/type/ contentType=application/json; charset=UTF-8 accept=application/json status=200
2022-02-02T14:31:07,309 INFO [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Table ads.ads_ad_show_place is already registered with id 7ea4a597-2a7d-412d-8b85-704b41d3f183. Updating entity.
2022-02-02T14:31:07,861 INFO [main] org.apache.atlas.AtlasBaseClient - method=POST path=api/atlas/v2/entity/ contentType=application/json; charset=UTF-8 accept=application/json status=200
2022-02-02T14:31:07,862 INFO [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Updated hive_table entity: name=ads.ads_ad_show_place@primary, guid=7ea4a597-2a7d-412d-8b85-704b41d3f183
2022-02-02T14:31:08,293 INFO [main] org.apache.atlas.AtlasBaseClient - method=GET path=api/atlas/v2/entity/uniqueAttribute/type/ contentType=application/json; charset=UTF-8 accept=application/json status=200
2022-02-02T14:31:08,295 INFO [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Table ads.ads_ad_show_place_window is already registered with id 79bda62d-3132-4acc-8729-95e3b7426dda. Updating entity.
2022-02-02T14:31:09,438 INFO [main] org.apache.atlas.AtlasBaseClient - method=POST path=api/atlas/v2/entity/ contentType=application/json; charset=UTF-8 accept=application/json status=200
2022-02-02T14:31:09,443 INFO [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Updated hive_table entity: name=ads.ads_ad_show_place_window@primary, guid=79bda62d-3132-4acc-8729-95e3b7426dda
2022-02-02T14:31:09,833 INFO [main] org.apache.atlas.AtlasBaseClient - method=GET path=api/atlas/v2/entity/uniqueAttribute/type/ contentType=application/json; charset=UTF-8 accept=application/json status=200
2022-02-02T14:31:09,837 INFO [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Table ads.ads_ad_show_rate is already registered with id e67b2530-d5e4-4da7-88a5-19b60581cf4e. Updating entity.
2022-02-02T14:31:12,648 INFO [main] org.apache.atlas.AtlasBaseClient - method=POST path=api/atlas/v2/entity/ contentType=application/json; charset=UTF-8 accept=application/json status=200
2022-02-02T14:31:12,686 INFO [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Updated hive_table entity: name=ads.ads_ad_show_rate@primary, guid=e67b2530-d5e4-4da7-88a5-19b60581cf4e
2022-02-02T14:31:13,124 INFO [main] org.apache.atlas.AtlasBaseClient - method=GET path=api/atlas/v2/entity/uniqueAttribute/type/ contentType=application/json; charset=UTF-8 accept=application/json status=200
2022-02-02T14:31:13,126 INFO [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Table ads.ads_member_active_count is already registered with id 50271cdb-74b9-42f3-a537-340d2678b2d9. Updating entity.
2022-02-02T14:31:13,750 INFO [main] org.apache.atlas.AtlasBaseClient - method=POST path=api/atlas/v2/entity/ contentType=application/json; charset=UTF-8 accept=application/json status=200
2022-02-02T14:31:13,751 INFO [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Updated hive_table entity: name=ads.ads_member_active_count@primary, guid=50271cdb-74b9-42f3-a537-340d2678b2d9
2022-02-02T14:31:14,050 INFO [main] org.apache.atlas.AtlasBaseClient - method=GET path=api/atlas/v2/entity/uniqueAttribute/type/ contentType=application/json; charset=UTF-8 accept=application/json status=200
2022-02-02T14:31:14,052 INFO [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Table ads.ads_member_retention_count is already registered with id dd245b45-f234-46ab-94d2-01a318f83b06. Updating entity.
2022-02-02T14:31:14,500 INFO [main] org.apache.atlas.AtlasBaseClient - method=POST path=api/atlas/v2/entity/ contentType=application/json; charset=UTF-8 accept=application/json status=200
2022-02-02T14:31:14,501 INFO [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Updated hive_table entity: name=ads.ads_member_retention_count@primary, guid=dd245b45-f234-46ab-94d2-01a318f83b06
2022-02-02T14:31:14,663 INFO [main] org.apache.atlas.AtlasBaseClient - method=GET path=api/atlas/v2/entity/uniqueAttribute/type/ contentType=application/json; charset=UTF-8 accept=application/json status=200
2022-02-02T14:31:14,664 INFO [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Table ads.ads_member_retention_rate is already registered with id 706f5111-f1eb-4c43-b689-ec9824b0a30b. Updating entity.
2022-02-02T14:31:15,882 INFO [main] org.apache.atlas.AtlasBaseClient - method=POST path=api/atlas/v2/entity/ contentType=application/json; charset=UTF-8 accept=application/json status=200
2022-02-02T14:31:15,883 INFO [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Updated hive_table entity: name=ads.ads_member_retention_rate@primary, guid=706f5111-f1eb-4c43-b689-ec9824b0a30b
2022-02-02T14:31:16,075 INFO [main] org.apache.atlas.AtlasBaseClient - method=GET path=api/atlas/v2/entity/uniqueAttribute/type/ contentType=application/json; charset=UTF-8 accept=application/json status=200
2022-02-02T14:31:16,076 INFO [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Table ads.ads_new_member_cnt is already registered with id ec6ac531-7faf-4f5f-bd3f-e6a27da84702. Updating entity.
2022-02-02T14:31:16,649 INFO [main] org.apache.atlas.AtlasBaseClient - method=POST path=api/atlas/v2/entity/ contentType=application/json; charset=UTF-8 accept=application/json status=200
2022-02-02T14:31:16,650 INFO [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Updated hive_table entity: name=ads.ads_new_member_cnt@primary, guid=ec6ac531-7faf-4f5f-bd3f-e6a27da84702
2022-02-02T14:31:16,890 INFO [main] org.apache.atlas.AtlasBaseClient - method=GET path=api/atlas/v2/entity/uniqueAttribute/type/ contentType=application/json; charset=UTF-8 accept=application/json status=200
2022-02-02T14:31:16,892 INFO [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Table ads.ads_trade_order_analysis is already registered with id 77f518f8-2cc1-4255-a8f6-8d2d1886e577. Updating entity.
2022-02-02T14:31:17,906 INFO [main] org.apache.atlas.AtlasBaseClient - method=POST path=api/atlas/v2/entity/ contentType=application/json; charset=UTF-8 accept=application/json status=200
2022-02-02T14:31:17,909 INFO [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Updated hive_table entity: name=ads.ads_trade_order_analysis@primary, guid=77f518f8-2cc1-4255-a8f6-8d2d1886e577
2022-02-02T14:31:18,040 INFO [main] org.apache.atlas.AtlasBaseClient - method=GET path=api/atlas/v2/entity/uniqueAttribute/type/ contentType=application/json; charset=UTF-8 accept=application/json status=200
..........
2022-02-02T14:32:05,936 INFO [main] org.apache.atlas.AtlasBaseClient - method=GET path=api/atlas/v2/entity/uniqueAttribute/type/ contentType=application/json; charset=UTF-8 accept=application/json status=404
2022-02-02T14:32:06,040 INFO [main] org.apache.atlas.AtlasBaseClient - method=POST path=api/atlas/v2/entity/ contentType=application/json; charset=UTF-8 accept=application/json status=200
2022-02-02T14:32:06,089 INFO [main] org.apache.atlas.AtlasBaseClient - method=GET path=api/atlas/v2/entity/guid/ contentType=application/json; charset=UTF-8 accept=application/json status=200
2022-02-02T14:32:06,089 INFO [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Created hive_db entity: name=tmp@primary, guid=e5334f76-3bb3-4052-a27d-ada5496aaf1b
2022-02-02T14:32:06,101 INFO [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - No tables to import in database tmp
Hive Meta Data imported successfully!!!

终于成功了
查看web界面
在这里插入图片描述
血缘关系展示

完美应用起来!!!

谷应泰

关注

1
点赞
踩
7

收藏

觉得还不错? 一键收藏
7
评论
Atlas编译安装以及Hive元数据导入

一.解压二.修改pom文件 645 <npm-for-v2.version>3.10.8</npm-for-v2.version> 652 <hadoop.version>2.9.2</hadoop.version>三.创建文件夹,手动放入solr,hbase(不然maven下载时间较长)/opt/lagou/servers/apache-atlas-sources-1.2.0/distro[root@Linux1
复制链接

扫一扫