HIVE1.1.0报错记录

运行环境：

HADOOP 2.2.0

ZOOKEEPER 3.4.6

HIVE 1.1.0

现象：

配置好hive-site.xml后执行Create Table正常,执行insert报错，日志如下：

Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.hdfs.DFSClient.getKeyProvider()Lorg/apache/hadoop/crypto/key/KeyProvider;

at org.apache.hadoop.hive.shims.Hadoop23Shims$HdfsEncryptionShim.<init>(Hadoop23Shims.java:1152)

at org.apache.hadoop.hive.shims.Hadoop23Shims.createHdfsEncryptionShim(Hadoop23Shims.java:1279)

at org.apache.hadoop.hive.ql.session.SessionState.getHdfsEncryptionShim(SessionState.java:392)

at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.isPathEncrypted(SemanticAnalyzer.java:1756)

at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getStagingDirectoryPathname(SemanticAnalyzer.java:1875)

at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:1689)

at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:1427)

at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genResolvedParseTree(SemanticAnalyzer.java:10132)

at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10147)

at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:192)

at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:222)

at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:421)

at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:307)

at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1112)

at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1160)

at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1049)

at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1039)

at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:207)

at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:159)

at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:370)

at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:754)

at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675)

at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:615)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

问题原因：

HIVE 1.1.0兼容性问题，需要修改源码重编译后修复问题详见https://issues.apache.org/jira/browse/HIVE-9957

修改源码详情：

diff --git a/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java b/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java
index 7c1887a20d53abd92bca8cb5f9b78030a38f9657..82833a33876e25bf75ae5dad0533c4c4a4ab6cbc 100644
--- a/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java
+++ b/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java
@@ -1281,7 +1281,12 @@ private int compareKeyStrength(String keyname1, String keyname2) throws IOExcept
   public HadoopShims.HdfsEncryptionShim createHdfsEncryptionShim(FileSystem fs, Configuration conf) throws IOException {
     URI uri = fs.getUri();
     if ("hdfs".equals(uri.getScheme())) {
-      return new HdfsEncryptionShim(uri, conf);
+      try {	
+        return new HdfsEncryptionShim(uri, conf);
+      } catch (NoSuchMethodError e) {
+        // ignore error as encryption is not supported.
+        // let this method return the unsupported encryption shim instead
+      }
     }
     return new HadoopShims.NoopHdfsEncryptionShim();
   }

删除 "-" 后面的内容

增加 "+"后面的内容

重编译步骤：

1.svn co http://svn.apache.org/repos/asf/hive/trunk hive  //下载源码
  

2.根据上述修改源码详情内容修改 hive/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java

3.CD hive

  mvn clean install -Phadoop-2,dist
4.cd packaging/target/apache-hive-{version}-SNAPSHOT-bin/apache-hive-{version}-SNAPSHOT-bin//编译好后存放目录