elasticsearch7.5.1集群断电故障解决方案

RecoveryFailedException，IndexShardRecoveryException，CorruptIndexException，BufferedChecksumIndexInput

错误信息如下

{
"state": "UNASSIGNED",
"primary": true,
"node": null,
"relocating_node": null,
"shard": 3,
"index": "my_index",
"recovery_source": { "type": "EXISTING_STORE","bootstrap_new_history_uuid": false},
"unassigned_info": {
"reason": "ALLOCATION_FAILED",
"at": "2021-04-12T11:19:46.920Z",
"failed_attempts": 5,
"failed_nodes": [
"LwWiAwmdQCiEibtiF7oqxQ"
],
"delayed": false,
"details": "failed shard on node [LwWiAwmdQCiEibtiF7oqxQ]: failed recovery, failure RecoveryFailedException[[my_index][3]: Recovery failed on {reading_10.10.10.15_node2}{LwWiAwmdQCiEibtiF7oqxQ}{-yc1_HSGRBav2OisXrPhlw}{10.10.10.15}{10.10.10.15:9402}{dil}{ml.machine_memory=539647844352, xpack.installed=true, ml.max_open_jobs=20}]; nested: IndexShardRecoveryException[failed recovery]; nested: ElasticsearchException[java.io.IOException: failed to read /home/wsn/es/es7.5/node_2/data/nodes/0/indices/QGft9wywTOeSNjcsz_UUHA/3/_state/retention-leases-91171.st]; nested: IOException[failed to read /home/wsn/es/es7.5/node_2/data/nodes/0/indices/QGft9wywTOeSNjcsz_UUHA/3/_state/retention-leases-91171.st]; nested: IOException[org.apache.lucene.index.CorruptIndexException: codec footer mismatch (file truncated?): actual footer=892219961 vs expected footer=-1071082520 (resource=BufferedChecksumIndexInput(SimpleFSIndexInput(path="/home/wsn/es/es7.5/node_2/data/nodes/0/indices/QGft9wywTOeSNjcsz_UUHA/3/_state/retention-leases-91171.st")))]; nested: CorruptIndexException[codec footer mismatch (file truncated?): actual footer=892219961 vs expected footer=-1071082520 (resource=BufferedChecksumIndexInput(SimpleFSIndexInput(path="/home/wsn/es/es7.5/node_2/data/nodes/0/indices/QGft9wywTOeSNjcsz_UUHA/3/_state/retention-leases-91171.st")))]; ",
"allocation_status": "deciders_no"
}
}

解决方案

POST _cluster/reroute
{
  "commands": [
    {
      "allocate_stale_primary": {
        "index": "my_index",
        "shard": 3,
        "node": "reading_10.10.10.15_node2",
        "accept_data_loss": true
      }
    }
  ]
}