pig一段时间没用,生疏了些,遇到几个错误,小问题,但记录之。
1.错误1
INFO org.apache.pig.impl.util.Utils - Default bootup file /tmp/.pigbootup not found
2014-05-06 14:52:59,644 [main] ERROR org.apache.pig.Main - ERROR 2999: Unexpected internal error. null
Details at logfile: /home/hdfs/project/lostrate/pig_1399359179385.log
bash-3.2$ more /home/hdfs/project/lostrate/pig_1399359179385.log
Error before Pig is launched
----------------------------
ERROR 2999: Unexpected internal error. null
java.lang.NullPointerException
at java.io.StringReader.<init>(StringReader.java:33)
at org.apache.pig.tools.parameters.PreprocessorContext.loadParamVal(PreprocessorContext.java:
258)
at org.apache.pig.impl.PigContext.doParamSubstitutionOutputToFile(PigContext.java:425)
at org.apache.pig.Main.runParamPreprocessor(Main.java:775)
at org.apache.pig.Main.run(Main.java:440)
at org.apache.pig.Main.main(Main.java:156)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
对应pig命令行是
/home/hdfs/pig-0.12.0/bin/pig -w -d 2 -c -f /home/hdfs/project/lostrate/count_install_new.pig -p -p stat_day=20140505
看到没,-p 后漏写了一个参数,导致这个问题,补上就对了
2.问题2
n] ERROR org.apache.pig.PigServer - exception during parsing: Error during parsing. Could not resolve com.twitter.elephantbird.pig.store.LzoPigStorage using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
Failed to parse: Pig script failed to parse:
<file /home/hdfs/project/lostrate/count_install_new.pig, line 11, column 56> pig script failed to validate: org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not resolve com.twitter.elephantbird.pig.store.LzoPigStorage using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:196)
at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1676)
at org.apache.pig.PigServer$Graph.access$000(PigServer.java:1409)
at org.apache.pig.PigServer.parseAndBuild(PigServer.java:342)
at org.apache.pig.PigServer.executeBatch(PigServer.java:367)
at org.apache.pig.PigServer.executeBatch(PigServer.java:353)
at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:140)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:202)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
at org.apache.pig.Main.run(Main.java:478)
at org.apache.pig.Main.main(Main.java:156)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by:
<file /home/hdfs/project/lostrate/count_install_new.pig, line 11, column 56> pig script failed to validate: org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not resolve com.twitter.elephantbird.pig.store.LzoPigStorage using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
at org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder.java:1260)
at org.apache.pig.parser.LogicalPlanBuilder.buildFuncSpec(LogicalPlanBuilder.java:1245)
at org.apache.pig.parser.LogicalPlanGenerator.func_clause(LogicalPlanGenerator.java:5069)
at org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3426)
at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1536)
at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1013)
at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:553)
at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)
at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:188)
... 16 more
Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not resolve com.twitter.elephantbird.pig.store.LzoPigStorage using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
at org.apache.pig.impl.PigContext.resolveClassName(PigContext.java:653)
at org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder.java:1257)
... 24 more
2014-05-06 15:14:20,140 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1070: Could not resolve com.twitter.elephantbird.pig.store.LzoPigStorage using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
又来欺负人了,以前没出现过这问题啊。
由于要用到lzo,结果开头只加载了一个jar文件,应该加载两个
REGISTER 'elephant-bird-core-3.0.8.jar';
REGISTER 'elephant-bird-pig-3.0.8.jar';
解决。