kettle mysql导入数据到hdfs报错:Hadoop file output.0 - File name field [null] couldn‘t be found in the input

解决此问题之前要首先检查hdfs连接是否有问题,如果如下图所示位置的连接标错说明windows下hadoop的环境变量未配置:

此时在黑窗口执行 hadoop version如果出现以下界面则hadoop环境变量配置成功

如果此时解析运行该转换会出现以下错误:

2024/09/12 13:25:53 - Hadoop file output.0 - ERROR (version 9.1.0.0-324, build 9.1.0.0-324 from 2020-09-07 05.09.05 by buildguy) : Unexpected error
2024/09/12 13:25:53 - Hadoop file output.0 - ERROR (version 9.1.0.0-324, build 9.1.0.0-324 from 2020-09-07 05.09.05 by buildguy) : org.pentaho.di.core.exception.KettleStepException:
2024/09/12 13:25:53 - Hadoop file output.0 - File name field [null] couldn't be found in the input stream!
2024/09/12 13:25:53 - Hadoop file output.0 -
2024/09/12 13:25:53 - Hadoop file output.0 - 	at org.pentaho.di.trans.steps.textfileoutput.TextFileOutput.getOutputFileName(TextFileOutput.java:265)
2024/09/12 13:25:53 - Hadoop file output.0 - 	at org.pentaho.di.trans.steps.textfileoutput.TextFileOutput.writeRowToFile(TextFileOutput.java:352)
2024/09/12 13:25:53 - Hadoop file output.0 - 	at org.pentaho.di.trans.steps.textfileoutput.TextFileOutput.writeRowTo(TextFileOutput.java:468)
2024/09/12 13:25:53 - Hadoop file output.0 - 	at org.pentaho.di.trans.steps.textfileoutput.TextFileOutput.processRow(TextFileOutput.java:461)
2024/09/12 13:25:53 - Hadoop file output.0 - 	at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)
2024/09/12 13:25:53 - Hadoop file output.0 - 	at java.lang.Thread.run(Thread.java:750)
2024/09/12 13:25:53 - Hadoop file output.0 - 完成处理 (I=0, O=0, R=1, W=0, U=0, E=1)
2024/09/12 13:25:53 - 表输入.0 - Finished reading query, closing connection.
2024/09/12 13:25:53 - hadoop-driver - 转换被检测
2024/09/12 13:25:53 - hadoop-driver - 转换正在杀死其他步骤!
2024/09/12 13:25:53 - 表输入.0 - 完成处理 (I=34, O=0, R=0, W=32, U=0, E=0)
2024/09/12 13:25:53 - hadoop-driver - ERROR (version 9.1.0.0-324, build 9.1.0.0-324 from 2020-09-07 05.09.05 by buildguy) : 错误被检测到!
2024/09/12 13:25:53 - Spoon - 转换完成!!
2024/09/12 13:25:53 - hadoop-driver - ERROR (version 9.1.0.0-324, build 9.1.0.0-324 from 2020-09-07 05.09.05 by buildguy) : 错误被检测到!
2024/09/12 13:25:53 - hadoop-driver - ERROR (version 9.1.0.0-324, build 9.1.0.0-324 from 2020-09-07 05.09.05 by buildguy) : 错误被检测到!

然后我们重启kettle再次在来南京Hdfs的页面点击测试图一所示的位置连接成功则hdfs就连接成功了。然后我们运行创建好的转换(mysql表输入和hadoopoutput输出)此时发现出现了以下错误:

2024/09/12 18:18:06 - Hadoop file output.0 - ERROR (version 9.1.0.0-324, build 9.1.0.0-324 from 2020-09-07 05.09.05 by buildguy) : Unexpected error
2024/09/12 18:18:06 - Hadoop file output.0 - ERROR (version 9.1.0.0-324, build 9.1.0.0-324 from 2020-09-07 05.09.05 by buildguy) : org.pentaho.di.core.exception.KettleException:
2024/09/12 18:18:06 - Hadoop file output.0 - Error opening new file : java.lang.RuntimeException: com.ctc.wstx.exc.WstxIOException: Invalid UTF-8 middle byte 0x3f (at char #1450, byte #20)
2024/09/12 18:18:06 - Hadoop file output.0 -

 显而易见就是我们文件的格式除了问题,此时我们回到kettle的启动文件下:

在如下图所示的位置加入我们解析utf-8的代码(位置不是特别好找建议索引查找一下):

"-Dfile.encoding=UTF-8"

此时再运行我们的转换即可

  • 2
    点赞
  • 2
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值