今天出现了一个怪问题,就是hive在运行几个SQL的时候,始终无法正常运行,导致方法阻塞,然后CPU也100%。
通过打印几个进程的方法栈发现:
"Thread-8" id=20 idx=0x50 tid=3647 prio=5 alive, native_blocked
at org/apache/hadoop/mapred/lib/CombineFileInputFormat.getMoreSplits(CombineFileInputFormat.java:339)
at org/apache/hadoop/mapred/lib/CombineFileInputFormat.getSplits(CombineFileInputFormat.java:204)
at org/apache/hadoop/hive/shims/HadoopShimsSecure$CombineFileInputFormatShim.getSplits(HadoopShimsSecure.java:386)
at org/apache/hadoop/hive/shims/HadoopShimsSecure$CombineFileInputFormatShim.getSplits(HadoopShimsSecure.java:352)
at org/apache/hadoop/hive/ql/io/CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:387)
at org/apache/hadoop/mapred/JobClient.writeOldSplits(JobClient.java:989)
at org/apache/hadoop/mapred/JobClient.writeSplits(JobClient.java:981)
at org/apache/hadoop/mapred/JobClient.access$600(JobClient.java:174)
at org/apache/hadoop/mapred/JobClient$2.run(JobClient.java:897)
at org/apache/hadoop/mapred/JobClient$2.run(JobClient.java:850)
at jrockit/vm/AccessController.doPrivileged(AccessController.java:254)
at javax/security/auth/Subject.doAs(Subject.java:396)
at org/apache/hadoop/security/UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org/apache/hadoop/mapred/JobClient.submitJobInternal(JobClient.java:850)
at org/apache/hadoop/mapred/JobClient.submitJob(JobClient.java:824)
at org/apache/hadoop/hive/ql/exec/ExecDriver.execute(ExecDriver.java:435)
at org/apache/hadoop/hive/ql/exec/MapRedTask.execute(MapRedTask.java:137)
at org/apache/hadoop/hive/ql/exec/Task.executeTask(Task.java:134)
at org/apache/hadoop/hive/ql/exec/TaskRunner.runSequential(TaskRunner.java:57)
at org/apache/hadoop/hive/ql/exec/TaskRunner.run(TaskRunner.java:47)
at jrockit/vm/RNI.c2java(JJJJJ)V(Native Method)
都是在此方法进行阻塞。
最近一直碰到这种怪问题啊,继续努力解决!