Hive的坑

公司集群做了一个迁移。但不久以后，发现，从hive的cli执行命令到从phphiveadmin执行命令，都出现错误。

错误如下:

-----
Diagnostic Messages for this Task:
java.io.IOException: java.lang.reflect.InvocationTargetException
        at org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
        at org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
        at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader(HadoopShimsSecure.java:367)
        at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.<init>(HadoopShimsSecure.java:314)
        at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileInputFormatShim.getRecordReader(HadoopShimsSecure.java:428)
        at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:540)
        at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:197)
        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:418)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
        at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
        at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader(HadoopShimsSecure.java:353)
        ... 11 more
Caused by: java.io.IOException: Compressed length 828431912 exceeds max block size 67108864 (probably corrupt file)
        at com.hadoop.compression.lzo.LzopInputStream.getCompressedData(LzopInputStream.java:286)
        at com.hadoop.compression.lzo.LzopInputStream.decompress(LzopInputStream.java:256)
        at org.apache.hadoop.io.compress.DecompressorStream.read(DecompressorStream.java:76)
        at java.io.InputStream.read(InputStream.java:82)
        at org.apache.hadoop.util.LineReader.readLine(LineReader.java:134)
        at org.apache.hadoop.util.LineReader.readLine(LineReader.java:187)
        at com.hadoop.mapred.DeprecatedLzoLineRecordReader.<init>(DeprecatedLzoLineRecordReader.java:64)
        at com.hadoop.mapred.DeprecatedLzoTextInputFormat.getRecordReader(DeprecatedLzoTextInputFormat.java:158)
        at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.<init>(CombineHiveRecordReader.java:65)
        ... 16 more

分析一:

是某台节点的LZO有问题，导致LZO错误。但某些HQL却能执行正确。排除这个可能。

分析二:

某台tt有问题，导致出错。排除掉故障节点，问题依旧。

最后在看MR日志的时候，发现有这么一行:

/tmp not exsit

然后发现hive机器上的磁盘在搬运中损坏，导致hadoop系统文件输出错误。最后hive编译完毕MR,以

hadoop 的client提交任务时候，无法正常生成临时文件。导致这个错误。

最后，换机器装hive,立刻解决。