hive日常报错之 serious problem serious problem,block 丢失

最新推荐文章于 2024-08-16 22:41:01 发布

weixin_34323858

最新推荐文章于 2024-08-16 22:41:01 发布

阅读量650

点赞数

文章标签：大数据 java python

原文链接：https://my.oschina.net/ilovetao/blog/1572768

版权

2019独角兽企业重金招聘Python工程师标准>>>

https://community.hortonworks.com/questions/71589/after-upgrading-to-hdp-253-using-beeline-with-a-si.html

报错如下参考链接如上感觉是 orc的文件优化导致的这个错误解决办法是

set hive.exec.orc.split.strategy=BI;

加上这么一个参数即可

Vertex failed, vertexName=Map 5, vertexId=vertex_1507250665075_0317_1_00, diagnostics=[Vertex vertex_1507250665075_0317_1_00 [Map 5] killed/failed due to:ROOT_INPUT_INIT_FAILURE, Vertex Input: erp_orders_union0124 initializer failed, vertex=vertex_1507250665075_0317_1_00 [Map 5], java.lang.RuntimeException: serious problem
09-10-2017 08:52:55 CST append ERROR - 	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1172)
09-10-2017 08:52:55 CST append ERROR - 	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getSplits(OrcInputFormat.java:1199)
09-10-2017 08:52:55 CST append ERROR - 	at org.apache.hadoop.hive.ql.io.HiveInputFormat.addSplitsForGroup(HiveInputFormat.java:307)
09-10-2017 08:52:55 CST append ERROR - 	at org.apache.hadoop.hive.ql.io.HiveInputFormat.getSplits(HiveInputFormat.java:409)
09-10-2017 08:52:55 CST append ERROR - 	at org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:155)
09-10-2017 08:52:55 CST append ERROR - 	at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:273)
09-10-2017 08:52:55 CST append ERROR - 	at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:266)
09-10-2017 08:52:55 CST append ERROR - 	at java.security.AccessController.doPrivileged(Native Method)
09-10-2017 08:52:55 CST append ERROR - 	at javax.security.auth.Subject.doAs(Subject.java:422)
09-10-2017 08:52:55 CST append ERROR - 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
09-10-2017 08:52:55 CST append ERROR - 	at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:266)
09-10-2017 08:52:55 CST append ERROR - 	at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:253)
09-10-2017 08:52:55 CST append ERROR - 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
09-10-2017 08:52:55 CST append ERROR - 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
09-10-2017 08:52:55 CST append ERROR - 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
09-10-2017 08:52:55 CST append ERROR - 	at java.lang.Thread.run(Thread.java:745)
09-10-2017 08:52:55 CST append ERROR - Caused by: java.util.concurrent.ExecutionException: org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-1265061168-192.168.5.206-1490093965591:blk_1074553674_813300 file=/apps/hive/warehouse/erp2.db/erp_orders_union0124/000000_0
09-10-2017 08:52:55 CST append ERROR - 	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
09-10-2017 08:52:55 CST append ERROR - 	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
09-10-2017 08:52:55 CST append ERROR - 	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1167)
09-10-2017 08:52:55 CST append ERROR - 	... 15 more
09-10-2017 08:52:55 CST append ERROR - Caused by: org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-1265061168-192.168.5.206-1490093965591:blk_1074553674_813300 file=/apps/hive/warehouse/erp2.db/erp_orders_union0124/000000_0
09-10-2017 08:52:55 CST append ERROR - 	at org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:1004)
09-10-2017 08:52:55 CST append ERROR - 	at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:648)
09-10-2017 08:52:55 CST append ERROR - 	at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:898)
09-10-2017 08:52:55 CST append ERROR - 	at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:955)
09-10-2017 08:52:55 CST append ERROR - 	at java.io.DataInputStream.readFully(DataInputStream.java:195)
09-10-2017 08:52:55 CST append ERROR - 	at org.apache.hadoop.hive.ql.io.orc.ReaderImpl.extractMetaInfoFromFooter(ReaderImpl.java:370)
09-10-2017 08:52:55 CST append ERROR - 	at org.apache.hadoop.hive.ql.io.orc.ReaderImpl.(ReaderImpl.java:317)
09-10-2017 08:52:55 CST append ERROR - 	at org.apache.hadoop.hive.ql.io.orc.OrcFile.createReader(OrcFile.java:238)
09-10-2017 08:52:55 CST append ERROR - 	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.populateAndCacheStripeDetails(OrcInputFormat.java:1057)
09-10-2017 08:52:55 CST append ERROR - 	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.callInternal(OrcInputFormat.java:969)
09-10-2017 08:52:55 CST append ERROR - 	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.access$1800(OrcInputFormat.java:810)
09-10-2017 08:52:55 CST append ERROR - 	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator$1.run(OrcInputFormat.java:960)
09-10-2017 08:52:55 CST append ERROR - 	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator$1.run(OrcInputFormat.java:957)
09-10-2017 08:52:55 CST append ERROR - 	at java.security.AccessController.doPrivileged(Native Method)
09-10-2017 08:52:55 CST append ERROR - 	at javax.security.auth.Subject.doAs(Subject.java:422)
09-10-2017 08:52:55 CST append ERROR - 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
09-10-2017 08:52:55 CST append ERROR - 	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.call(OrcInputFormat.java:957)
09-10-2017 08:52:55 CST append ERROR - 	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.call(OrcInputFormat.java:810)
09-10-2017 08:52:55 CST append ERROR - 	... 4 more
09-10-2017 08:52:55 CST append ERROR - ]
09-10-2017 08:52:55 CST append ERROR - Vertex killed, vertexName=Reducer 4, vertexId=vertex_1507250665075_0317_1_03, diagnostics=[Vertex received Kill in INITED state., Vertex vertex_1507250665075_0317_1_03 [Reducer 4] killed/failed due to:OTHER_VERTEX_FAILURE]
09-10-2017 08:52:55 CST append ERROR - Vertex killed, vertexName=Reducer 3, vertexId=vertex_1507250665075_0317_1_02, diagnostics=[Vertex received Kill in INITED state., Vertex vertex_1507250665075_0317_1_02 [Reducer 3] killed/failed due to:OTHER_VERTEX_FAILURE]
09-10-2017 08:52:55 CST append ERROR - Vertex killed, vertexName=Map 1, vertexId=vertex_1507250665075_0317_1_01, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:87, Vertex vertex_1507250665075_0317_1_01 [Map 1] killed/failed due to:OTHER_VERTEX_FAILURE]
09-10-2017 08:52:55 CST append ERROR - DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:3

加上之后不报这个错了改成block missing

之后想起来最近hdfs死过一回应该就是那一回导致block missing

然后百度 hdfs fsck /

之后 hdfs fsck -delete / 删除丢失的块(因为我这儿的数据不怎么重要所以直接删除)

再次执行 hive sql语句 ok 搞定

转载于:https://my.oschina.net/ilovetao/blog/1572768