pig脚本常见错误

1、ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1200: String index out of range: 36

原因:结尾多了个空格

video_read_allnet = LOAD '/user/hdphailiyang/statistics/kyk_close/video_read_allnet/${date}/p*';
-- `awk XXX ` XXX后面不能有空格
video_read_allnet = STREAM video_read_allnet THROUGH `awk -F'[\t|]' '{print $1"\t"$2"\t"$3}'`;
2、ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1000: Error during parsing.
   Encountered " <PATH> "book=load "" at line 2, column 1.
   Was expecting one of:
       <EOF> 
       "cat" ...
       "clear" ...
       "fs" ...
       "sh" ...
       "cd" ...
       "cp" ...
       "copyFromLocal" ...
       "copyToLocal" ...
       "dump" ...
       "\\d" ...
       "describe" ...
       "\\de" ...
       "aliases" ...
       "explain" ...
       "\\e" ...
       "help" ...
       "history" ...
       "kill" ...
       "ls" ...
       "mv" ...
       "mkdir" ...
       "pwd" ...
       "quit" ...
       "\\q" ...
       "register" ...
       "rm" ...
       "rmf" ...
       "set" ...
       "illustrate" ...
       "\\i" ...
       "run" ...
       "exec" ...
       "scriptDone" ...
       "" ...
       "" ...
       <EOL> ...
       ";" ...

原因:等号两边要留空格

A=LOAD 'XXX'  => A = LOAD 'XXX'
3、ERROR org.apache.pig.Main - ERROR 2997: Encountered IOException. org.apache.pig.tools.parameters.ParseException: Encountered “” at line 1, column 9

我的脚本第一行是这个

SET default_parallel 500;

原因:
用shell脚本调用的时候,-p 参数有空,该参数我前面删了,但是脚本里没有去掉对应的 -p参数,所以报错。

4、Invalid field projection. Projected field [group_res1::pv] does not exist in schema: group:chararray,group_res1:bag{:tuple(uin:chararray,pv:long)}.
group_res1 = GROUP join_res1 BY (uin, docid);

group_res1 = FOREACH group_res1 GENERATE group.$0 AS uin, COUNT($1) AS pv;

uin_max1 = GROUP group_res1 BY uin;

# 这一行报错
uin_max1 = FOREACH uin_max1 GENERATE FLATTEN(group) AS uin, MAX(group_res1::pv) AS pv;

这里pv是在group_res1里的tuple数据结构里,不能用双冒号,应该用点号去获取,修改如下:

uin_max1 = FOREACH uin_max1 GENERATE FLATTEN(group) AS uin, MAX(group_res1.pv) AS pv;
  • 1
    点赞
  • 3
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值