Zeppelin安装成功后，安装Hive之后启动Zeppelin异常

最新推荐文章于 2023-03-20 22:58:40 发布
最新推荐文章于 2023-03-20 22:58:40 发布
阅读量3k 收藏
点赞数
分类专栏： Spark Hive
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/dax1n/article/details/58630396
版权
Spark 同时被 3 个专栏收录
70 篇文章 0 订阅
订阅专栏
Hive
15 篇文章 2 订阅
订阅专栏
Zeppelin
4 篇文章 0 订阅
订阅专栏


Zeppelin安装参见：http://blog.csdn.net/dax1n/article/details/57079534

异常信如下：


 WARN [2017-03-01 19:27:25,416] ({pool-2-thread-2} Hive.java[<clinit>]:168) - Failed to access metastore. This class should not accessed in runtime.
org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1236)
at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:38)
at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:33)
at org.apache.zeppelin.spark.SparkInterpreter.createSparkSession(SparkInterpreter.java:350)
at org.apache.zeppelin.spark.SparkInterpreter.getSparkSession(SparkInterpreter.java:222)
at org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:815)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:482)
at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1523)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
... 60 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
... 66 more
Caused by: MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
at org.apache.thrift.transport.TSocket.open(TSocket.java:187)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:420)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:236)
at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234)
at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:174)
at org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:166)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:38)
at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:33)
at org.apache.zeppelin.spark.SparkInterpreter.createSparkSession(SparkInterpreter.java:350)
at org.apache.zeppelin.spark.SparkInterpreter.getSparkSession(SparkInterpreter.java:222)
at org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:815)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:482)
at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.thrift.transport.TSocket.open(TSocket.java:182)
... 74 more
)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:466)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:236)
at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
... 71 more
 INFO [2017-03-01 19:27:25,418] ({pool-2-thread-2} HiveMetaStoreClient.java[open]:376) - Trying to connect to metastore with URI thrift://node:9083
 WARN [2017-03-01 19:27:25,418] ({pool-2-thread-2} HiveMetaStoreClient.java[open]:428) - Failed to connect to the MetaStore Server...
 INFO [2017-03-01 19:27:25,419] ({pool-2-thread-2} HiveMetaStoreClient.java[open]:459) - Waiting 1 seconds before next connection attempt.
 INFO [2017-03-01 19:27:26,419] ({pool-2-thread-2} HiveMetaStoreClient.java[open]:376) - Trying to connect to metastore with URI thrift://node:9083
 WARN [2017-03-01 19:27:26,419] ({pool-2-thread-2} HiveMetaStoreClient.java[open]:428) - Failed to connect to the MetaStore Server...
 INFO [2017-03-01 19:27:26,420] ({pool-2-thread-2} HiveMetaStoreClient.java[open]:459) - Waiting 1 seconds before next connection attempt.
 INFO [2017-03-01 19:27:27,420] ({pool-2-thread-2} HiveMetaStoreClient.java[open]:376) - Trying to connect to metastore with URI thrift://node:9083
 WARN [2017-03-01 19:27:27,421] ({pool-2-thread-2} HiveMetaStoreClient.java[open]:428) - Failed to connect to the MetaStore Server...
 INFO [2017-03-01 19:27:27,422] ({pool-2-thread-2} HiveMetaStoreClient.java[open]:459) - Waiting 1 seconds before next connection attempt.
ERROR [2017-03-01 19:27:28,425] ({pool-2-thread-2} Utils.java[invokeMethod]:40) - 
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:38)
at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:33)
at org.apache.zeppelin.spark.SparkInterpreter.createSparkSession(SparkInterpreter.java:350)
at org.apache.zeppelin.spark.SparkInterpreter.getSparkSession(SparkInterpreter.java:222)
at org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:815)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:482)
at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.IllegalArgumentException: Error while instantiating 'org.apache.spark.sql.hive.HiveSessionState':
at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:981)
at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
... 20 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
... 30 more
Caused by: java.lang.IllegalArgumentException: Error while instantiating 'org.apache.spark.sql.hive.HiveExternalCatalog':
at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:169)
at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
... 35 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
... 43 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
... 48 more
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:522)
at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
... 56 more
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1523)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
... 57 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
... 63 more
Caused by: MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
at org.apache.thrift.transport.TSocket.open(TSocket.java:187)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:420)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:236)
at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:192)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:264)
at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:366)
at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:270)
at org.apache.spark.sql.hive.HiveExternalCatalog.<init>(HiveExternalCatalog.scala:65)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.spark.sql.internal.SharedState$.org$apache$spark$sql$internal$SharedState$$reflect(SharedState.scala:166)
at org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:86)
at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
at org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:101)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:101)
at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:100)
at org.apache.spark.sql.internal.SessionState.<init>(SessionState.scala:157)
at org.apache.spark.sql.hive.HiveSessionState.<init>(HiveSessionState.scala:32)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.spark.sql.SparkSession$.org$apache$spark$sql$SparkSession$$reflect(SparkSession.scala:978)
at org.apache.spark.sql.SparkSession.sessionState$lzycompute(SparkSession.scala:110)
at org.apache.spark.sql.SparkSession.sessionState(SparkSession.scala:109)
at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
at org.apache.spark.sql.SparkSession$Builder$$anonfun$getOrCreate$5.apply(SparkSession.scala:878)
at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:878)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:38)
at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:33)
at org.apache.zeppelin.spark.SparkInterpreter.createSparkSession(SparkInterpreter.java:350)
at org.apache.zeppelin.spark.SparkInterpreter.getSparkSession(SparkInterpreter.java:222)
at org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:815)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:482)
at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.thrift.transport.TSocket.open(TSocket.java:182)
... 71 more
)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:466)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:236)
at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
... 68 more
 INFO [2017-03-01 19:27:28,430] ({pool-2-thread-2} SparkInterpreter.java[createSparkSession]:351) - Created Spark session with Hive support
ERROR [2017-03-01 19:27:28,430] ({pool-2-thread-2} Job.java[run]:181) - Job failed
java.lang.NullPointerException
at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:38)
at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:33)
at org.apache.zeppelin.spark.SparkInterpreter.createSparkContext_2(SparkInterpreter.java:380)
at org.apache.zeppelin.spark.SparkInterpreter.createSparkContext(SparkInterpreter.java:369)
at org.apache.zeppelin.spark.SparkInterpreter.getSparkContext(SparkInterpreter.java:144)
at org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:817)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:482)
at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
 INFO [2017-03-01 19:27:28,434] ({pool-2-thread-2} SchedulerFactory.java[jobFinished]:137) - Job remoteInterpretJob_1488367636075 finished by scheduler org.apache.zeppelin.spark.SparkInterpreter1631518462
ERROR [2017-03-01 19:29:50,893] ({dispatcher-event-loop-0} Logging.scala[logError]:70) - Application has been killed. Reason: Master removed our application: KILLED
ERROR [2017-03-01 19:29:50,897] ({dispatcher-event-loop-0} Logging.scala[logError]:91) - Ignoring error
org.apache.spark.SparkException: Exiting due to error from cluster scheduler: Master removed our application: KILLED
at org.apache.spark.scheduler.TaskSchedulerImpl.error(TaskSchedulerImpl.scala:459)
at org.apache.spark.scheduler.cluster.StandaloneSchedulerBackend.dead(StandaloneSchedulerBackend.scala:139)
at org.apache.spark.deploy.client.StandaloneAppClient$ClientEndpoint.markDead(StandaloneAppClient.scala:254)
at org.apache.spark.deploy.client.StandaloneAppClient$ClientEndpoint$$anonfun$receive$1.applyOrElse(StandaloneAppClient.scala:168)
at org.apache.spark.rpc.netty.Inbox$$anonfun$process$1.apply$mcV$sp(Inbox.scala:117)
at org.apache.spark.rpc.netty.Inbox.safelyCall(Inbox.scala:205)
at org.apache.spark.rpc.netty.Inbox.process(Inbox.scala:101)
at org.apache.spark.rpc.netty.Dispatcher$MessageLoop.run(Dispatcher.scala:213)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
 INFO [2017-03-01 19:29:50,901] ({stop-spark-context} AbstractConnector.java[doStop]:306) - Stopped ServerConnector@7d78bd66{HTTP/1.1}{0.0.0.0:4040}
 INFO [2017-03-01 19:29:50,903] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@56a4a72f{/stages/stage/kill,null,UNAVAILABLE}
 INFO [2017-03-01 19:29:50,903] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@1f66222{/jobs/job/kill,null,UNAVAILABLE}
 INFO [2017-03-01 19:29:50,903] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@58f60d97{/api,null,UNAVAILABLE}
 INFO [2017-03-01 19:29:50,903] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@9d278c2{/,null,UNAVAILABLE}
 INFO [2017-03-01 19:29:50,904] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@60769778{/static,null,UNAVAILABLE}
 INFO [2017-03-01 19:29:50,904] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@45515c1d{/executors/threadDump/json,null,UNAVAILABLE}
 INFO [2017-03-01 19:29:50,904] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@6c81ea9a{/executors/threadDump,null,UNAVAILABLE}
 INFO [2017-03-01 19:29:50,904] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@451900c6{/executors/json,null,UNAVAILABLE}
 INFO [2017-03-01 19:29:50,905] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@23558b01{/executors,null,UNAVAILABLE}
 INFO [2017-03-01 19:29:50,905] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@6005e657{/environment/json,null,UNAVAILABLE}
 INFO [2017-03-01 19:29:50,905] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@6eadfa35{/environment,null,UNAVAILABLE}
 INFO [2017-03-01 19:29:50,905] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@57b9e9f1{/storage/rdd/json,null,UNAVAILABLE}
 INFO [2017-03-01 19:29:50,905] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@6b6f4b51{/storage/rdd,null,UNAVAILABLE}
 INFO [2017-03-01 19:29:50,905] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@465aae64{/storage/json,null,UNAVAILABLE}
 INFO [2017-03-01 19:29:50,906] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@218397ac{/storage,null,UNAVAILABLE}
 INFO [2017-03-01 19:29:50,906] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@3cb92e21{/stages/pool/json,null,UNAVAILABLE}
 INFO [2017-03-01 19:29:50,906] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@5348b9a{/stages/pool,null,UNAVAILABLE}
 INFO [2017-03-01 19:29:50,906] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@4651534f{/stages/stage/json,null,UNAVAILABLE}
 INFO [2017-03-01 19:29:50,906] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@75597dc9{/stages/stage,null,UNAVAILABLE}
 INFO [2017-03-01 19:29:50,907] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@537a1a5f{/stages/json,null,UNAVAILABLE}
 INFO [2017-03-01 19:29:50,907] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@2986d6b5{/stages,null,UNAVAILABLE}
 INFO [2017-03-01 19:29:50,907] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@4114828c{/jobs/job/json,null,UNAVAILABLE}
 INFO [2017-03-01 19:29:50,907] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@4ae40925{/jobs/job,null,UNAVAILABLE}
 INFO [2017-03-01 19:29:50,907] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@caf54bc{/jobs/json,null,UNAVAILABLE}
 INFO [2017-03-01 19:29:50,907] ({stop-spark-context} ContextHandler.java[doStop]:865) - Stopped o.s.j.s.ServletContextHandler@17002d90{/jobs,null,UNAVAILABLE}
 INFO [2017-03-01 19:29:50,908] ({stop-spark-context} Logging.scala[logInfo]:54) - Stopped Spark web UI at http://192.168.1.101:4040
 INFO [2017-03-01 19:29:50,970] ({stop-spark-context} Logging.scala[logInfo]:54) - Shutting down all executors
 INFO [2017-03-01 19:29:50,971] ({dispatcher-event-loop-3} Logging.scala[logInfo]:54) - Asking each executor to shut down
 INFO [2017-03-01 19:29:50,980] ({dispatcher-event-loop-2} Logging.scala[logInfo]:54) - MapOutputTrackerMasterEndpoint stopped!
 INFO [2017-03-01 19:29:50,984] ({stop-spark-context} Logging.scala[logInfo]:54) - MemoryStore cleared
 INFO [2017-03-01 19:29:50,984] ({stop-spark-context} Logging.scala[logInfo]:54) - BlockManager stopped
 INFO [2017-03-01 19:29:50,985] ({stop-spark-context} Logging.scala[logInfo]:54) - BlockManagerMaster stopped
 INFO [2017-03-01 19:29:50,987] ({dispatcher-event-loop-3} Logging.scala[logInfo]:54) - OutputCommitCoordinator stopped!
 INFO [2017-03-01 19:29:50,988] ({stop-spark-context} Logging.scala[logInfo]:54) - Successfully stopped SparkContext
 INFO [2017-03-01 19:30:06,228] ({pool-2-thread-4} SchedulerFactory.java[jobStarted]:131) - Job remoteInterpretJob_1488367806228 started by scheduler org.apache.zeppelin.spark.SparkInterpreter1631518462
 INFO [2017-03-01 19:30:07,487] ({pool-2-thread-4} SparkInterpreter.java[createSparkSession]:307) - ------ Create new SparkContext spark://node:7077 -------
 WARN [2017-03-01 19:30:07,488] ({pool-2-thread-4} SparkInterpreter.java[setupConfForSparkR]:551) - sparkr.zip is not found, sparkr may not work.
 INFO [2017-03-01 19:30:07,490] ({pool-2-thread-4} Logging.scala[logInfo]:54) - Running Spark version 2.1.0
 WARN [2017-03-01 19:30:07,490] ({pool-2-thread-4} Logging.scala[logWarning]:66) - Support for Java 7 is deprecated as of Spark 2.0.0
 WARN [2017-03-01 19:30:07,492] ({pool-2-thread-4} Logging.scala[logWarning]:66) - 
SPARK_WORKER_INSTANCES was detected (set to '2').
This is deprecated in Spark 1.0+.


Please instead use:
 - ./spark-submit with --num-executors to specify the number of executors
 - Or set SPARK_EXECUTOR_INSTANCES
 - spark.executor.instances to configure the number of instances in the spark config.
        
解决方案：

由于配置Hive之后，需要将hive-site.xml复制HADOOP_CONF_DIR一份，由于Zeppelin（支持Hive）也需要加载HADOOP_CONF_DIR目录，所以导致无法加载Hive元数据，

导致异常，此时启动Hive的元数据服务在启动Zeppelin即可。

javartisan
关注
  • 0
    点赞
  • 踩
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
一键安装jdk+mysql+hadoop+hive+zeppelin.rar
09-17
给大家提供了完整的jar包、一键化安装、一键启停脚本以及使用说明教程 由于zeppelin文件大小限制，分开上传，大家点进我主页的资源查看
spark整合hive后spark-sql启动报错
天龙八部
12-14 2349
通过spark-sql 来访问hive的元数据，hive元数据信息存储在mysql但是启动的时候是报如下的错误，检查过配置,metastore已经配置，并且启动，用hive启动可以正常使用，但是spark-sql不行，报错信息 Hive: Failed to access metastore. This class should not accessed in runtime. org.apac...
启动spark-sql报错Caused by: MetaException(message:Version information not found
wqhlmark64的博客
09-13 3601
集群中已部署且正常使用了hadoop, hive, spark。打算启动spark-sql来访问hive数据时报错， WARN metadata.Hive: Failed to access metastore. This class should not accessed in runtime. org.apache.hadoop.hive.ql.metadata.HiveExcep
Zeppelin报错 java.lang.RuntimeException: Java HotSpot(TM) 64-Bit Server VM warni
科技向善
10-05 753
把虚拟机内存调大一点，足够撑起Zeppelin里面的服务器内存就可以了。 虚拟机内存2G以上应该可以解决这个问题
代码审查常见代码质量问题
邢超的博客
09-30 2万+
配套的Bug解释模式 为了有针对性的使用这个工具，减少bug的误报，提高使用效率，我们选择了10个左右的bug模式，下面就是对这10个模式的解释。 这些bug可能会引起程序的性能或逻辑问题. 需要说明的是,findbugs能检测的bug pattern远不仅于此,甚至可以定制自己的探测器,因此,这个文档会不断扩充,同时,也欢迎大家不断探索和分享使用实践. 大的分类主要包括以
java runtime 异常_java中的java.lang.RuntimeException异常怎么解决？
weixin_29439509的博客
02-12 2万+
展开全部解决方式如下：1、通配符{1}修改为{0}。2、form提交地址62616964757a686964616fe59b9ee7ad9431333365633865修改下:注册Form地址:登录Form地址:Java语音特征：Java编程语言的风格十分接近C语言、C++语言。Java是一个纯粹的面向对象的程序设计语言，它继承了C++语言面向对象技术的核心。Java舍弃了C语言中容易引起错误的指...
 Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
微电子学与固体电子学-俞驰
05-03 2120
020-05-03 10:18:11 WARN Hive:168 - Failed to access metastore. This class should not accessed in runtime. org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to i...
已解决java.lang.RuntimeException: java.lang.RuntimeException: org.codehaus.plexus.component.repository.
热门推荐
03-20 8万+
已解决java.lang.RuntimeException: java.lang.RuntimeException: org.codehaus.plexus.component.repository.exc异常的正确解决方法，亲测有效！！！
逆向生成pojo以及mapper
小孩与狗的博客
01-30 246
generatorConfig.xml <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE generatorConfiguration PUBLIC "-//mybatis.org//DTD MyBatis Generator Configuration 1.0//EN" "http://mybatis.org/dtd/mybatis-generator-config_1_0.dtd"> <generatorConfig
metastore作用_Hive metastore整体代码分析及详解
weixin_27966471的博客
12-24 1408
从上一篇对Hive metastore表结构的简要分析中，我再根据数据设计的实体对象，再进行整个代码结构的总结。那么我们先打开metadata的目录，其目录结构：可以看到，整个hivemeta的目录包含metastore(客户端与服务端调用逻辑)、events(事件目录包含table生命周期中的检查、权限认证等listener实现)、hooks(这里的hooks仅包含了jdo connection...
Hive实战项目数据文件和Zeppelin源文件
01-26
Hive实战项目数据文件和Zeppelin源文件
Zeppelin源码分析—Zeppelin的设计思想
02-24
支持多种语言，包括Scala、Python、SparkSQL、Hive、Markdown、Shell等。本文从需求出发，探索Zeppelin的架构设计、技术选型、代码的模块划分和依赖关系的最初“出发点”，从而我们可以了解到Zeppelin为什么是这样...
Zeppelin原理简介
01-27
Zeppelin是一个基于Web的notebook，提供交互数据分析和可视化。后台支持接入多种数据处理引擎，如spark，hive等。支持多种语言： Scala(ApacheSpark)、Python(ApacheSpark)、SparkSQL、 Hive、Markdown、Shell等。...
Spark 重分区函数：coalesce和repartition区别与实现，可以优化Spark程序性能
吟啸徐行
12-02 2万+
源码包路径： org.apache.spark.rdd.RDD coalesce函数： 方法注释： Return a new RDDthat is reduced into numPartitions partitions. This results in a narrowdependency, e.g. if you go from 1000 p
Spark On Yarn ERROR spark.SparkContext: Error initializing SparkContext.
吟啸徐行
11-01 1万+
当初搭建Spark On Yarn时候 配置yarn端获取spark运行的jar包时候，配置如下： spark.yarn.jars           hdfs:///sparkrunjar/ 提交程序： spark-submit --class org.apache.spark.examples.SparkPi --master yarn --deploy-mode client
Spark Streaming +Kafka 使用底层API直接读取Kafka的Partition数据，手动更新Offset到Zookeeper集群
吟啸徐行
11-30 1万+
Spark Streaming  +Kafka 使用底层API直接读取Kafka的Partition数据，正常Offset存储在CheckPoint中。但是这样无法实现Kafka监控工具对Kafka的监控，所以手动更新Offset到Zookeeper集群中
Spark Streaming的foreachRDD算子的正确使用
吟啸徐行
03-01 9497
这一块是Spark闭包的问题，不懂自行看文档吧！ 重点：Spark Streaming的foreachRDD运行在Driver端，而foreach和foreachPartion运行在Worker节点。 备注：对数据的向外输出，还是用foreach**算子好，不要用Map**算子，因为Map还要返回一个RDD。
Spark整合kafka0.10.0新特性(二)
吟啸徐行
03-12 6292
接着Spark整合kafka0.10.0新特性(一)开始 import org.apache.kafka.clients.consumer.ConsumerRecord import org.apache.kafka.common.serialization.StringDeserializer import org.apache.spark.streaming.kafka010._
Spark 数据全局排序实现以及RangePartitioner的使用示例
吟啸徐行
09-13 6238
使用Java 随机数类随机生成0到Integer.MAX_VALUE之间的10GB文件，然后使用Spark计算框架进行全局排序。 实现思路：可以局部排序(局部数据是全局数据的某一个范围)最后合并到全同一个文件，保证全局有序，这样可以设置一个reduce任务实现，但是对于更大量的数据容易出现OOM。如果不合并到同一个文件的话，可以将每一个分区有序的数据输出到磁盘。最后借鉴Kafka的数据管理方式建
zeppelin 连接kerberos hive
最新发布
03-14
Zeppelin是一个开源的数据分析和可视化工具，它提供了一个交互式的笔记本界面，可以用于执行和展示数据分析任务。...完成以上步骤后，你应该能够在Zeppelin中连接到Kerberos Hive，并执行相关的数据分析任务。

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值