apache-kylin-4.0.1-bin-spark2问题

1、问题

2022-05-22 09:14:16,233 ERROR [Thread-1] application.JobMonitor : Job failed the 0 times.
java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/protobuf/generated/MasterProtos$MasterService$BlockingInterface
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:264)
	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:228)
	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218)
	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:119)
	at org.apache.hadoop.hive.hbase.HiveHBaseTableInputFormat.getSplitsInternal(HiveHBaseTableInputFormat.java:466)
	at org.apache.hadoop.hive.hbase.HiveHBaseTableInputFormat.getSplits(HiveHBaseTableInputFormat.java:453)
	at org.apache.spark.rdd.HadoopRDD.getPartitions(HadoopRDD.scala:204)
	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:273)
	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:269)
	at scala.Option.getOrElse(Option.scala:121)
	at org.apache.spark.rdd.RDD.partitions(RDD.scala:269)
	at org.apache.kylin.engine.spark.utils.SparkUtils$$anonfun$leafNodePartitionNums$1.apply(SparkUtils.scala:37)
	at org.apache.kylin.engine.spark.utils.SparkUtils$$anonfun$leafNodePartitionNums$1.apply(SparkUtils.scala:37)
	at scala.collection.immutable.List.map(List.scala:273)
	at org.apache.kylin.engine.spark.utils.SparkUtils$.leafNodePartitionNums(SparkUtils.scala:37)
	at org.apache.kylin.engine.spark.utils.SparkUtils.leafNodePartitionNums(SparkUtils.scala)
	at org.apache.kylin.engine.spark.job.ResourceDetectBeforeCubingJob.doExecute(ResourceDetectBeforeCubingJob.java:87)
	at org.apache.kylin.engine.spark.application.SparkApplication.execute(SparkApplication.java:308)
	at org.apache.kylin.engine.spark.application.SparkApplication.execute(SparkApplication.java:94)
	at org.apache.spark.application.JobWorker$$anon$2.run(JobWorker.scala:55)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingInterface
	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 24 more

把hbase目录下的hbase-common-1.2.0-cdh5.12.0.jar、hbase-client-1.2.0-cdh5.12.0.jar、hbase-protocol-1.2.0-cdh5.12.0.jar、hbase-server-1.2.0-cdh5.12.0.jar拷贝到spark的jar目录下

2、问题

Caused by: com.google.protobuf.ServiceException: java.lang.NoClassDefFoundError: com/yammer/metrics/core/Gauge
	at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:239)
	at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:331)
	at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:37250)
	at org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:394)
	... 10 more
Caused by: java.lang.NoClassDefFoundError: com/yammer/metrics/core/Gauge
	at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:224)
	... 13 more
Caused by: java.lang.ClassNotFoundException: com.yammer.metrics.core.Gauge
	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 14 more

拷贝 hbase目录下的/opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/lib/hbase/lib/…/…/…/jars/metrics-core-2.2.0.jar到spark 的jar目录下

3、SparkSQL查询与HBase关联的Hive外部表，查询失败
在使用SparkSQL查询时，报错：error in initSerDe: java.ClassNotFoundException Class org.apache.hadoop.hive.hbase.HBaseSerDe not found
解决方法

hbase-client-1.2.0-cdh5.12.2.jar
hbase-common-1.2.0-cdh5.12.2.jar
hbase-protocol-1.2.0-cdh5.12.2.jar
hbase-server-1.2.0-cdh5.12.2.jar
htrace-core4-4.1.0-incubating.jar
protobuf-java-2.5.0.jar
guava-14.0.1.jar
hive-hbase-handler-1.1.0-cdh5.12.2.jar