服务器掉电重启后无法修复,ES节点服务器异常掉电重启导致shard不能启动的问题修复...

[2018-04-18 16:29:25,950][WARN][indices.cluster] [elasticsearch_43_ssd] [ocslog-2018.04.18][0] failed to start shard

org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException: [ocslog-2018.04.18][0] failed to recover shard

at org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:287)

at org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:132)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745)

Caused by: org.elasticsearch.index.translog.TranslogCorruptedException: translog corruption while reading from stream

at org.elasticsearch.index.translog.ChecksummedTranslogStream.read(ChecksummedTranslogStream.java:70)

at org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:257)

... 4 more

Caused by: java.io.EOFException

at org.elasticsearch.common.io.stream.InputStreamStreamInput.readBytes(InputStreamStreamInput.java:53)

at org.elasticsearch.index.translog.BufferedChecksumStreamInput.readBytes(BufferedChecksumStreamInput.java:55)

at org.elasticsearch.common.io.stream.StreamInput.readBytesReference(StreamInput.java:86)

at org.elasticsearch.common.io.stream.StreamInput.readBytesReference(StreamInput.java:74)

at org.elasticsearch.index.translog.Translog$Index.readFrom(Translog.java:495)

at org.elasticsearch.index.translog.ChecksummedTranslogStream.read(ChecksummedTranslogStream.java:68)

... 5 more

[2018-04-18 16:29:25,959][WARN][cluster.action.shard] [elasticsearch_43_ssd] [ocslog-2018.04.18][0] sending failed shard for [ocslog-2018.04.18][0], node[hROI0lMDQvqFNa2pcdlrGg], [P], s[INITIALIZING], indexUUID [opNRuZb0QS-UseDQlAadfA], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[ocslog-2018.04.18][0] failed to recover shard]; nested: TranslogCorruptedException[translog corruption while reading from stream]; nested: EOFException; ]]

[2018-04-18 16:29:25,959][WARN][cluster.action.shard] [elasticsearch_43_ssd] [ocslog-2018.04.18][0] received shard failed for [ocslog-2018.04.18][0], node[hROI0lMDQvqFNa2pcdlrGg], [P], s[INITIALIZING], indexUUID [opNRuZb0QS-UseDQlAadfA], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[ocslog-2018.04.18][0] failed to recover shard]; nested: TranslogCorruptedException[translog corruption while reading from stream]; nested: EOFException; ]]

[2018-04-18 16:29:26,304][WARN][cluster.action.shard] [elasticsearch_43_ssd] [ocslog-2018.04.18][0] received shard failed for [ocslog-2018.04.18][0], node[hROI0lMDQvqFNa2pcdlrGg], [P], s[INITIALIZING], indexUUID [opNRuZb0QS-UseDQlAadfA], reason [master [elasticsearch_43_ssd][hROI0lMDQvqFNa2pcdlrGg][ocsbak][inet[/192.168.0.43:9300]]{tag=hot, max_local_storage_nodes=1, master=true} marked shard as initializing, but shard is marked as failed, resend shard failure]

[2018-04-18 16:29:26,624][WARN][cluster.action.shard] [elasticsearch_43_ssd] [ocslog-2018.04.18][0] received shard failed for [ocslog-2018.04.18][0], node[hROI0lMDQvqFNa2pcdlrGg], [P], s[INITIALIZING], indexUUID [opNRuZb0QS-UseDQlAadfA], reason [master [elasticsearch_43_ssd][hROI0lMDQvqFNa2pcdlrGg][ocsbak][inet[/192.168.0.43:9300]]{tag=hot, max_local_storage_nodes=1, master=true} marked shard as initializing, but shard is marked as failed, resend shard failure]

[2018-04-18 16:29:27,174][WARN][cluster.action.shard] [elasticsearch_43_ssd] [ocslog-2018.04.18][0] received shard failed for [ocslog-2018.04.18][0], node[hROI0lMDQvqFNa2pcdlrGg], [P], s[INITIALIZING], indexUUID [opNRuZb0QS-UseDQlAadfA], reason [master [elasticsearch_43_ssd][hROI0lMDQvqFNa2pcdlrGg][ocsbak][inet[/192.168.0.43:9300]]{tag=hot, max_local_storage_nodes=1, master=true} marked shard as initializing, but shard is marked as failed, resend shard failure]