2021-11-16

**

Kettle CDH6.3.1 链接hive 报错 org.apache.hive.service.cli.thrift.TCLIService$Iface

**

查阅了资料,Kettle是稍微需要和CDH版本进行匹配的。
因为我们现有的集群是CDH 6.3.1的所以需要匹配 Kettle 8.X 的版本
1.首先是我们的CDH版本。

2.其次是Kettle版本

我测试了下， 这俩个版本是比较匹配的。
3.接下来解压完成Kettle之后需要修改一些Kettle配置

1. 先去修改 D:\kettleData\data-integration\plugins\pentaho-big-data-plugin\plugin.properties
   
   为什么这么修改？是为了方便解决后面的错误

2. 接下来需要把CDH的客户端文件下载下来替换相应的conf文件
   D:\kettleData\data-integration\plugins\pentaho-big-data-plugin\hadoop-configurations\cdh61
   这个目录的conf 文件
   

3.替换完成之后需要把相应的jar包也替换了，（也是CDH hive/lib/目录下的hive开头的和 com开头的）
D:\kettleData\data-integration\plugins\pentaho-big-data-plugin\hadoop-configurations\cdh61\lib


hive我这里没放完，你们放完就可以了。
4.把hive的lib文件夹中的beeline jdbc 放入
D:\kettleData\data-integration\lib 中。（在这步是为了解决报数据库链接驱动找不到的情况）
5.这里会报错少类，回到我们第一个图片让修改的配置哪里。使用kettle连接hive2就会报这个错误
下面展示一些 内联代码片。

org.pentaho.di.core.exception.KettleDatabaseException:
Error occurred while trying to connect to the database

Error connecting to database: (using class org.apache.hive.jdbc.HiveDriver)
org/apache/hive/service/cli/thrift/TCLIService$Iface


org.pentaho.di.core.exception.KettleDatabaseException:
Error occurred while trying to connect to the database

Error connecting to database: (using class org.apache.hive.jdbc.HiveDriver)
org/apache/hive/service/cli/thrift/TCLIService$Iface


at org.pentaho.di.core.database.Database.normalConnect(Database.java:477)
at org.pentaho.di.core.database.Database.connect(Database.java:373)
at org.pentaho.di.core.database.Database.connect(Database.java:344)
at org.pentaho.di.core.database.Database.connect(Database.java:334)
at org.pentaho.di.core.database.DatabaseFactory.getConnectionTestReport(DatabaseFactory.java:83)
at org.pentaho.di.core.database.DatabaseFactory.getConnectionTestResults(DatabaseFactory.java:112)
at org.pentaho.di.core.database.DatabaseMeta.testConnectionSuccess(DatabaseMeta.java:2843)
at org.pentaho.ui.database.event.DataHandler.testDatabaseConnection(DataHandler.java:631)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.pentaho.ui.xul.impl.AbstractXulDomContainer.invoke(AbstractXulDomContainer.java:313)
at org.pentaho.ui.xul.impl.AbstractXulComponent.invoke(AbstractXulComponent.java:157)
at org.pentaho.ui.xul.impl.AbstractXulComponent.invoke(AbstractXulComponent.java:141)
at org.pentaho.ui.xul.swt.tags.SwtButton.access$500(SwtButton.java:43)
at org.pentaho.ui.xul.swt.tags.SwtButton$4.widgetSelected(SwtButton.java:137)
at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source)
at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
at org.eclipse.jface.window.Window.runEventLoop(Window.java:820)
at org.eclipse.jface.window.Window.open(Window.java:796)
at org.pentaho.di.ui.xul.KettleDialog.show(KettleDialog.java:89)
at org.pentaho.di.ui.xul.KettleDialog.show(KettleDialog.java:56)
at org.pentaho.di.ui.core.database.dialog.XulDatabaseDialog.open(XulDatabaseDialog.java:117)
at org.pentaho.di.ui.core.database.dialog.DatabaseDialog.open(DatabaseDialog.java:61)
at org.pentaho.di.ui.trans.step.BaseStepDialog.showDbDialogUnlessCancelledOrValid(BaseStepDialog.java:813)
at org.pentaho.di.ui.trans.step.BaseStepDialog$EditConnectionListener.widgetSelected(BaseStepDialog.java:1521)
at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source)
at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
at org.pentaho.big.data.kettle.plugins.hive.trans.HiveOutputDialog.open(HiveOutputDialog.java:686)
at org.pentaho.di.ui.spoon.delegates.SpoonStepsDelegate.editStep(SpoonStepsDelegate.java:120)
at org.pentaho.di.ui.spoon.Spoon.editStep(Spoon.java:8814)
at org.pentaho.di.ui.spoon.trans.TransGraph.editStep(TransGraph.java:3287)
at org.pentaho.di.ui.spoon.trans.TransGraph.mouseDoubleClick(TransGraph.java:785)
at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source)
at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
at org.pentaho.di.ui.spoon.Spoon.readAndDispatch(Spoon.java:1384)
at org.pentaho.di.ui.spoon.Spoon.waitForDispose(Spoon.java:7949)
at org.pentaho.di.ui.spoon.Spoon.start(Spoon.java:9331)
at org.pentaho.di.ui.spoon.Spoon.main(Spoon.java:710)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.pentaho.commons.launcher.Launcher.main(Launcher.java:92)
Caused by: org.pentaho.di.core.exception.KettleDatabaseException:
Error connecting to database: (using class org.apache.hive.jdbc.HiveDriver)
org/apache/hive/service/cli/thrift/TCLIService$Iface

at org.pentaho.di.core.database.Database.connectUsingClass(Database.java:590)
at org.pentaho.di.core.database.Database.normalConnect(Database.java:461)
... 56 more
Caused by: java.lang.NoClassDefFoundError: org/apache/hive/service/cli/thrift/TCLIService$Iface
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:208)
at org.pentaho.di.core.database.Database.connectUsingClass(Database.java:572)
... 57 more
Caused by: java.lang.ClassNotFoundException: org.apache.hive.service.cli.thrift.TCLIService$Iface
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 61 more

ketlle用到的是D:\kettleData\data-integration\plugins\pentaho-big-data-plugin\hadoop-configurations\mapr60\lib 下的所有jar包
6、将hive/lib下 hive-开头的所有的包 和 hadoop-common-***.jar【从服务端的hadoop获取jar包】 拷贝到 D:\kettleData\data-integration\plugins\pentaho-big-data-plugin\hadoop-configurations\mapr60\lib 下
7、将hive/lib下 hive-开头的所有的包 和 hadoop-common-***.jar【从服务端的hadoop获取jar包】 拷贝到 D:\kettleData\data-integration\lib 下