kettle-java代码执行hive相关ktr时报错： database type with plugin id [HIVE2] couldn't be found!

最新推荐文章于 2024-07-26 15:08:55 发布
最新推荐文章于 2024-07-26 15:08:55 发布
阅读量4.3k 收藏 3
点赞数 1
分类专栏： kettle

kettle-java代码执行hive相关ktr时报错： database type with plugin id [HIVE2] couldn't be found!

转

2018年08月13日 16:47:30 lisery1993 阅读数：305

1.在java项目中执行hive相关的ktr:

  1.  

  2. KettleEnvironment.init();

  3.  

  4. TransMeta transMeta = new TransMeta("C:\\Users\\lixiang\\Desktop\\hivetorizhi.ktr");

  5. // 设置执行模式

  6. TransExecutionConfiguration config = new TransExecutionConfiguration();

  7. config.setExecutingClustered(true);

  8. config.setExecutingLocally(false);

  9. config.setExecutingRemotely(false);

  10. config.setClusterPosting(true);

  11. config.setClusterPreparing(true);

  12. config.setClusterStarting(true);

  13. TransSplitter transSplitter = Trans.executeClustered(transMeta, config);

  14. System.out.println(transSplitter.getCarteObjectMap());

  15. System.out.println(transSplitter.getMaster());

  16. System.out.println(transSplitter.getSlaves()[0]);

  17. System.out.println(transSplitter.getSlaves()[1].getStepNames()[0]);

  18. }

2.报错：

 

  1. Exception in thread "main" org.pentaho.di.core.exception.KettleXMLException:

  2. 错误从XML文件读取转换

  3.  

  4. 错误从XML文件读取转换

  5.  

  6. Unable to load database connection info from XML node

  7.  

  8. Unable to create new database interface

  9.  

  10. database type with plugin id [HIVE2] couldn't be found!

  11.  

  12. at org.pentaho.di.trans.TransMeta.loadXML(TransMeta.java:3402)

  13. at org.pentaho.di.trans.TransMeta.<init>(TransMeta.java:2758)

  14. at org.pentaho.di.trans.TransMeta.<init>(TransMeta.java:2710)

  15. at org.pentaho.di.trans.TransMeta.<init>(TransMeta.java:2687)

  16. at org.pentaho.di.trans.TransMeta.<init>(TransMeta.java:2667)

  17. at org.pentaho.di.trans.TransMeta.<init>(TransMeta.java:2632)

  18. at org.pentaho.di.trans.TransMeta.<init>(TransMeta.java:2595)

  19. at com.kettle.KettleExamle.getRemoteServer(KettleExamle.java:55)

  20. at com.kettle.KettleExamle.main(KettleExamle.java:140)

  21. Caused by: org.pentaho.di.core.exception.KettleXMLException:

  22. 错误从XML文件读取转换

  23.  

  24. Unable to load database connection info from XML node

  25.  

  26. Unable to create new database interface

  27.  

  28. database type with plugin id [HIVE2] couldn't be found!

  29.  

  30. at org.pentaho.di.trans.TransMeta.loadXML(TransMeta.java:3384)

  31. ... 8 more

  32. Caused by: org.pentaho.di.core.exception.KettleXMLException:

  33. Unable to load database connection info from XML node

  34.  

  35. Unable to create new database interface

  36.  

  37. database type with plugin id [HIVE2] couldn't be found!

  38.  

  39. at org.pentaho.di.core.database.DatabaseMeta.<init>(DatabaseMeta.java:1013)

  40. at org.pentaho.di.trans.TransMeta.loadXML(TransMeta.java:2976)

  41. ... 8 more

  42. Caused by: org.pentaho.di.core.exception.KettleXMLException:

  43. Unable to create new database interface

  44.  

  45. database type with plugin id [HIVE2] couldn't be found!

  46.  

  47. at org.pentaho.di.core.database.DatabaseMeta.<init>(DatabaseMeta.java:977)

  48. ... 9 more

  49. Caused by: org.pentaho.di.core.exception.KettleDatabaseException:

  50. database type with plugin id [HIVE2] couldn't be found!

  51.  

  52. at org.pentaho.di.core.database.DatabaseMeta.findDatabaseInterface(DatabaseMeta.java:584)

  53. at org.pentaho.di.core.database.DatabaseMeta.getDatabaseInterface(DatabaseMeta.java:558)

  54. at org.pentaho.di.core.database.DatabaseMeta.<init>(DatabaseMeta.java:974)

  55. ... 9 more

3.报错分析：kettle程序在读取database type的过程中读不到hive2这个类型。

查看kettle-core-7.1.0.0-12.jar中的kettle-database-types.xml文件，此文件定义了database type ,其中没hive2.。

 

  1. <database-types>

  2.  

  3. <database-type id="AS/400">

  4. <description>AS/400</description>

  5. <classname>org.pentaho.di.core.database.AS400DatabaseMeta</classname>

  6. </database-type>

  7.  

  8. <database-type id="CACHE">

  9. <description>Intersystems Cache</description>

  10. <classname>org.pentaho.di.core.database.CacheDatabaseMeta</classname>

  11. </database-type>

  12.  

  13. <database-type id="DB2">

  14. <description>IBM DB2</description>

  15. <classname>org.pentaho.di.core.database.DB2DatabaseMeta</classname>

  16. </database-type>

  17.  

  18. <database-type id="DBASE">

  19. <description>dBase III, IV or 5</description>

  20. <classname>org.pentaho.di.core.database.DbaseDatabaseMeta</classname>

  21. </database-type>

  22.  

  23. <database-type id="DERBY">

  24. <description>Apache Derby</description>

  25. <classname>org.pentaho.di.core.database.DerbyDatabaseMeta</classname>

  26. </database-type>

  27.  

  28. <database-type id="EXASOL4">

  29. <description>Exasol 4</description>

  30. <classname>org.pentaho.di.core.database.Exasol4DatabaseMeta</classname>

  31. </database-type>

  32.  

  33. <database-type id="EXTENDB">

  34. <description>ExtenDB</description>

  35. <classname>org.pentaho.di.core.database.ExtenDBDatabaseMeta</classname>

  36. </database-type>

  37.  

  38. <database-type id="FIREBIRD">

  39. <description>Firebird SQL</description>

  40. <classname>org.pentaho.di.core.database.FirebirdDatabaseMeta</classname>

  41. </database-type>

  42.  

  43. <database-type id="GENERIC">

  44. <description>Generic database</description>

  45. <classname>org.pentaho.di.core.database.GenericDatabaseMeta</classname>

  46. </database-type>

  47.  

  48. <database-type id="GREENPLUM">

  49. <description>Greenplum</description>

  50. <classname>org.pentaho.di.core.database.GreenplumDatabaseMeta</classname>

  51. </database-type>

  52.  

  53. <database-type id="INFINIDB">

  54. <description>Calpont InfiniDB</description>

  55. <classname>org.pentaho.di.core.database.InfiniDbDatabaseMeta</classname>

  56. </database-type>

  57.  

  58. <database-type id="SQLBASE">

  59. <description>Gupta SQL Base</description>

  60. <classname>org.pentaho.di.core.database.GuptaDatabaseMeta</classname>

  61. </database-type>

  62.  

  63. <database-type id="H2">

  64. <description>H2</description>

  65. <classname>org.pentaho.di.core.database.H2DatabaseMeta</classname>

  66. </database-type>

  67.  

  68. <database-type id="HYPERSONIC">

  69. <description>Hypersonic</description>

  70. <classname>org.pentaho.di.core.database.HypersonicDatabaseMeta</classname>

  71. </database-type>

  72.  

  73. <database-type id="INFOBRIGHT">

  74. <description>Infobright</description>

  75. <classname>org.pentaho.di.core.database.InfobrightDatabaseMeta</classname>

  76. </database-type>

  77.  

  78. <database-type id="INFORMIX">

  79. <description>Informix</description>

  80. <classname>org.pentaho.di.core.database.InformixDatabaseMeta</classname>

  81. </database-type>

  82.  

  83. <database-type id="INGRES">

  84. <description>Ingres</description>

  85. <classname>org.pentaho.di.core.database.IngresDatabaseMeta</classname>

  86. </database-type>

  87.  

  88. <database-type id="VECTORWISE">

  89. <description>Ingres VectorWise</description>

  90. <classname>org.pentaho.di.core.database.VectorWiseDatabaseMeta</classname>

  91. </database-type>

  92.  

  93. <database-type id="INTERBASE">

  94. <description>Borland Interbase</description>

  95. <classname>org.pentaho.di.core.database.InterbaseDatabaseMeta</classname>

  96. </database-type>

  97.  

  98. <database-type id="KINGBASEES">

  99. <description>KingbaseES</description>

  100. <classname>org.pentaho.di.core.database.KingbaseESDatabaseMeta</classname>

  101. </database-type>

  102.  

  103. <database-type id="LucidDB">

  104. <description>LucidDB</description>

  105. <classname>org.pentaho.di.core.database.LucidDBDatabaseMeta</classname>

  106. </database-type>

  107.  

  108. <database-type id="MARIADB">

  109. <description>MariaDB</description>

  110. <classname>org.pentaho.di.core.database.MariaDBDatabaseMeta</classname>

  111. </database-type>

  112.  

  113. <database-type id="MONETDB">

  114. <description>MonetDB</description>

  115. <classname>org.pentaho.di.core.database.MonetDBDatabaseMeta</classname>

  116. </database-type>

  117.  

  118. <database-type id="MSACCESS">

  119. <description>MS Access</description>

  120. <classname>org.pentaho.di.core.database.MSAccessDatabaseMeta</classname>

  121. </database-type>

  122.  

  123. <database-type id="MSSQLNATIVE">

  124. <description>MS SQL Server (Native)</description>

  125. <classname>org.pentaho.di.core.database.MSSQLServerNativeDatabaseMeta</classname>

  126. </database-type>

  127.  

  128. <database-type id="MSSQL">

  129. <description>MS SQL Server</description>

  130. <classname>org.pentaho.di.core.database.MSSQLServerDatabaseMeta</classname>

  131. </database-type>

  132.  

  133. <database-type id="MYSQL">

  134. <description>MySQL</description>

  135. <classname>org.pentaho.di.core.database.MySQLDatabaseMeta</classname>

  136. </database-type>

  137.  

  138. <database-type id="MONDRIAN">

  139. <description>Native Mondrian</description>

  140. <classname>org.pentaho.di.core.database.MondrianNativeDatabaseMeta</classname>

  141. </database-type>

  142.  

  143. <database-type id="NEOVIEW">

  144. <description>Neoview</description>

  145. <classname>org.pentaho.di.core.database.NeoviewDatabaseMeta</classname>

  146. </database-type>

  147.  

  148. <database-type id="NETEZZA">

  149. <description>Netezza</description>

  150. <classname>org.pentaho.di.core.database.NetezzaDatabaseMeta</classname>

  151. </database-type>

  152.  

  153. <database-type id="ORACLE">

  154. <description>Oracle</description>

  155. <classname>org.pentaho.di.core.database.OracleDatabaseMeta</classname>

  156. </database-type>

  157.  

  158. <database-type id="ORACLERDB">

  159. <description>Oracle RDB</description>

  160. <classname>org.pentaho.di.core.database.OracleRDBDatabaseMeta</classname>

  161. </database-type>

  162.  

  163. <database-type id="POSTGRESQL">

  164. <description>PostgreSQL</description>

  165. <classname>org.pentaho.di.core.database.PostgreSQLDatabaseMeta</classname>

  166. </database-type>

  167.  

  168. <database-type id="REDSHIFT">

  169. <description>Redshift</description>

  170. <classname>org.pentaho.di.core.database.RedshiftDatabaseMeta</classname>

  171. </database-type>

  172.  

  173. <database-type id="REMEDY-AR-SYSTEM">

  174. <description>Remedy Action Request System</description>

  175. <classname>org.pentaho.di.core.database.RemedyActionRequestSystemDatabaseMeta</classname>

  176. </database-type>

  177.  

  178. <database-type id="SAPDB">

  179. <description>MaxDB (SAP DB)</description>

  180. <classname>org.pentaho.di.core.database.SAPDBDatabaseMeta</classname>

  181. </database-type>

  182.  

  183. <database-type id="SQLITE">

  184. <description>SQLite</description>

  185. <classname>org.pentaho.di.core.database.SQLiteDatabaseMeta</classname>

  186. </database-type>

  187.  

  188. <database-type id="SYBASE">

  189. <description>Sybase</description>

  190. <classname>org.pentaho.di.core.database.SybaseDatabaseMeta</classname>

  191. </database-type>

  192.  

  193. <database-type id="SYBASEIQ">

  194. <description>SybaseIQ</description>

  195. <classname>org.pentaho.di.core.database.SybaseIQDatabaseMeta</classname>

  196. </database-type>

  197.  

  198. <database-type id="TERADATA">

  199. <description>Teradata</description>

  200. <classname>org.pentaho.di.core.database.TeradataDatabaseMeta</classname>

  201. </database-type>

  202.  

  203. <database-type id="UNIVERSE">

  204. <description>UniVerse database</description>

  205. <classname>org.pentaho.di.core.database.UniVerseDatabaseMeta</classname>

  206. </database-type>

  207.  

  208. <database-type id="VERTICA">

  209. <description>Vertica</description>

  210. <classname>org.pentaho.di.core.database.VerticaDatabaseMeta</classname>

  211. </database-type>

  212.  

  213. <database-type id="VERTICA5">

  214. <description>Vertica 5+</description>

  215. <classname>org.pentaho.di.core.database.Vertica5DatabaseMeta</classname>

  216. </database-type>

  217.  

  218. </database-types>

4.报错解决：

hive的api的连接方式是通过jdbc连接的。具体思路与postgresql的连接思路相同。因此仿照postgresql的连接方法自定义一个database-type为hive2。因此，在kettle-core-7.1.0.0-12.jar的kettle-database-types.xml文件中添加

 

  1. <database-type id="HIVE2">

  2. <description>HIVE2</description>

  3. <classname>org.pentaho.di.core.database.Hive2SQLDatabaseMeta</classname>

  4. </database-type>

其中会有对应的实现类org.pentaho.di.core.database.Hive2SQLDatabaseMeta。

仿照postgresql的实现类org.pentaho.di.core.database.PostgreSQLDatabaseMeta来写即可，只需将连接的url以及jdbc driver名称更换为hive相关。

将org.pentaho.di.core.database.Hive2SQLDatabaseMeta类编译为.class文件放进kettle-core-7.1.0.0-12.jar的org\pentaho\di\core\database路径下

6.再次执行ktr：

执行成功：结果：

 

  1. {/hivetorizhi (master)=1cc5f5bb-b0c4-4938-96f7-85363f88564d, /hivetorizhi (local:slave1-8081)=09ea3633-f5bd-4e43-9352-62b958255469, /hivetorizhi (local:slave2-8082)=35e00dc2-f570-4f26-8540-156fb84d9524}

  2. /hivetorizhi (master)

  3. /hivetorizhi (local:slave2-8082)

  4.  
fengfengchen95
关注
专栏目录
kettle 7 在linux执行,Kettle 在Linux下执行pan.sh 出现报错 [HIVE2] couldn't be found
weixin_29710393的博客
05-16 995
源码如下：[dcp@etlsvr1 pdi-ce-7.0.0.0-25]$ ./pan.sh -file=lnr.ktr#######################################################################WARNING:no libwebkitgtk-1.0 detected, some features will be unavail...
kettle-linux中执行hive的ktr报错：Error connecting to database: using class org.apache.hive.jdbc.HiveDriver
lisery的博客
07-19 1万+
1.将windows环境中产生的ktr文件上传到linux环境中执行 [root@lx02 data-integration]# ./pan.sh -file=/opt/cm/hadoop/pdi-ce-7.1.0.0-12/data-integration/workspace/hivetorizhi.ktr &gt;&gt; /opt/cm/hadoop/pdi-ce-7.1.0.0-12/...
kettle执行hive相关ktr时报错： database type with plugin id [HIVE2] couldn't be found!
04-02
java代码执行hive相关ktr时报错： database type with plugin id [HIVE2] couldn't be found! 解决：kettle-core-7.1.0.0-12.jar适配hive后的包。具体步骤请查看https://blog.csdn.net/m0_37618809/article/details/81632342
kettle连接东华InterSystems IRIS数据库
最新发布
weixin_46349545的博客
07-26 280
kettle cache数据库 InterSystems
kettle-java代码执行hive相关ktr时报错： database type with plugin id [HIVE2] couldn‘t be found!
lisery的博客
08-13 6675
1.在java项目中执行hive相关的ktr: KettleEnvironment.init(); TransMeta transMeta = new TransMeta("C:\\Users\\lixiang\\Desktop\\hivetorizhi.ktr"); // 设置执行模式 TransExecutionConfiguration config = new T...
Kettle7.0 报错database type with plugin id [HIVE2] couldn't be found!
weixin_33928137的博客
11-21 2229
2019独角兽企业重金招聘Python工程师标准>>> ...
KettleXMLException: 错误从XML文件读取转换 (error during reading transform from the xml file )...
weixin_33982670的博客
09-11 3565
2019独角兽企业重金招聘Python工程师标准>>> ...
Kettle 连接hive2
afunyusong的专栏
11-17 2739
Data Integration (kettle) 7.0 连接Hivekettle 5.1.0 连接 Hadoop hive 2 (hive 1.2.1)
kettle执行hive相关ktr时报错
05-06
总的来说，解决"Kettle执行hive相关ktr时报错"的问题需要综合考虑多个方面，包括但不限于Hive和Kettle的配置、环境变量、权限、认证、库的兼容性、文件路径和元数据一致性。通过逐一排查这些潜在问题，通常能找出...
kettle-core-7.1.0.0-12.jar
09-19
java代码执行hive相关ktr时报错： database type with plugin id [HIVE2] couldn't be found! 解决：kettle-core-7.1.0.0-12.jar适配hive后的包。具体步骤请查看...
（Hive输出）pentaho-big-data-kettle-plugins-hive-6.1.0.1-196
07-24
（Hive输出）ETLDesigner\common\system\karaf\system\pentaho\pentaho-big-data-kettle-plugins-hive\6.1.0.1-196\下的文件。 （Hive输出）pentaho-big-data-kettle-plugins-hive-6.1.0.1-196。
eclipse集成kettle_java 调用kettle报错，求大神解答
weixin_29516495的博客
01-14 286
2017-06-25 回答java调用kettle数据库类型资源库中的ktr 此问题在1个月前或许已经接触，单是一直木有怎么用到，就被耽搁至今；问题的解决要来源于网络，其实我还想说问题的解决更多的是要靠我们自己的思想，不过多的言情，我们接下来直接进入主题吧！ 环境：kettle-spoon 4.2.0,oracle11g，myeclipse6.5，sqlserver2008 前提：在kettle图...
Kettle连接数据库[MySQL]报错
m0_74972727的博客
10-07 1478
【代码】Kettle连接数据库[MySQL]报错。
java 调用kettle资源库报错
milian_12的博客
02-17 798
java 调用kettle资源库报错，资源库连接失败：Error occured while trying to connect to the database。Error connecting to database: (using class org.gjt.mm.mysql.Driver)
Kettle连接MySQL报错：错误连接数据库 [mysql] : org.pentaho.di.core.exception.KettleDatabaseException的解决办法
qq_42166308的博客
07-03 2501
在Windows系统里面安装Kettle后打算连接MySQL的时候突然报错： 错误连接数据库 [mysql] : org.pentaho.di.core.exception.KettleDatabaseException: Error occurred while trying to connect to the database Driver class ‘org.gjt.mm.mysql.Driver’ could not be found, make sure the ‘MySQL’ driver (
db2中kettle报错 Field [XXX] is required and couldn‘t be found 解决方法
BHSZZY的博客
08-01 4991
有一个需求，要从db2的某个表抽取数据到mysql的某个表中。上面明明配置了asname，但是还报错没有name，就很奇葩。
job kettle 导出_kettle从hive2导出数据正常，但是用kettle manager启动就报错
weixin_42371574的博客
01-27 188
该楼层疑似违规已被系统折叠隐藏此楼查看此楼kettle连接hive2，并导出hive2数据，在kettle运行正常，但是用kettle manager启动job就报错：2017/03/31 14:20:33 - ETL - ERROR (version 5.4.0.1-130, build 1 from 2015-06-14_12-34-55 by buildguy) : 当读共享文件时发生错误...
解决kettle数据处理中报 Field [XXXXX] is required and couldn‘t be found错误
热门推荐
u010479989的博客
08-24 2万+
Kettle帮助文档下载 刚接触kettle，在做数据处理时候一直提示Field [FIELD_ID] is required and couldn't be found!， FIELD_ID是标识字段，通过自定义常量赋值，模型建立好后一直报错 最后发现原因是核心对象使用错误，使用增加常量后不报错 ...
java：从XML文件读取转换Unable to load step info from XML step nodejava.util.ConcurrentModificationException
it佳佳的博客
12-04 1642
问题现象 java 调用Kettle时候，提示 错误从XML文件读取转换 Unable to load step info from XML step nodejava.util.ConcurrentModificationException at java.lang.Thread.run (Thread.java:748) at java.util.concurrent.ThreadPool...
java调kettle的kjb脚本时报错：Can't run transformation due to plugin missing
05-25
这个错误通常是由于 Kettle 中某些插件没有正确安装或配置所致。您可以尝试以下几个步骤来解决这个问题： 1. 确认您的 Kettle 安装中是否缺少必要的插件，可以在 Kettle 的安装目录下的 plugins 文件夹中查看是否...
评论 4
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值