hive 调用python脚步报错

最新推荐文章于 2024-04-05 01:00:00 发布

keilson

最新推荐文章于 2024-04-05 01:00:00 发布

阅读量6.1k

点赞数

分类专栏： hive 大数据文章标签： hive

本文链接：https://blog.csdn.net/u014793282/article/details/54894977

版权

在使用Hive进行数据处理时遇到错误：Error 20003，原因是尝试关闭运行自定义脚本的操作符时出错。问题发生在Hive的ScriptOperator关闭过程中，可能由于未将Python脚本添加到分布式缓存导致。解决方法是通过Hive的`ADD FILE`命令将Python脚本路径添加到缓存，例如：`hive> add file oss://LTA.py;` 和 `hive> add file /demo.py;`。

摘要由CSDN通过智能技术生成

Error: java.lang.RuntimeException: Hive Runtime Error while closing operators: [Error 20003]: An error occurred when trying to close the Operator running your custom script.
at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.close(ExecReducer.java:287)
at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:453)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: [Error 20003]: An error occurred when trying to close the Operator running your custom script.

最低0.47元/天解锁文章