关于es删除nodes文件夹后启动失败的问题

初次接手kafka+elk，不小心删除了es的data文件夹下的nodes文件夹。
然后，es健康状态报红，我尝试重启es集群，失败。提取主要报错，如下：

注：报错未全部显示，为了便于观察我仅提取主要错误展示
报错1
[WARN ][o.e.g.MetaStateService   ] [es] [[.monitoring-kibana-6-2019.03.18/n6kDbooRRhuME3I5ntfQcw]]: failed to write index state
报错2
[WARN ][o.e.g.MetaStateService   ] [es] [[.monitoring-kibana-6-2019.03.19/MghRcWkvTcydQzJIW1SfbA]]: failed to write index state
报错3
[WARN ][o.e.g.MetaStateService   ] [es] [[.monitoring-kibana-6-2019.03.20/MWtk6qR4T9a6IUaJ76dd9A]]: failed to write index state
报错4
[WARN ][o.e.g.MetaStateService   ] [es] [[.monitoring-kibana-6-2019.03.21/rwn9gq3PTqe6zNmsBmEDBA]]: failed to write index state
报错5
[WARN ][o.e.g.MetaStateService   ] [es] [[.monitoring-kibana-6-2019.03.22/cOwSHci8SwKDhv9Ce33mSg]]: failed to write index state
报错6
[WARN ][o.e.g.MetaStateService   ] [es] [[.monitoring-kibana-6-2019.03.23/GwieLpYQTVq-K0I3XO2b_Q]]: failed to write index state
报错7
[WARN ][o.e.g.MetaStateService   ] [es] [[.monitoring-kibana-6-2019.03.24/15AcdE3sS0a-5TC-pvw8gg]]: failed to write index state
报错8
[WARN ][o.e.g.MetaStateService   ] [es] [[.monitoring-kibana-6-2019.03.25/JwtM2CFgQACRDe50yXpdCw]]: failed to write index state
报错9
[WARN ][o.e.g.MetaStateService   ] [es] [[.monitoring-es-6-2019.03.21/3iEQGvf1T06RZpbrUrmjkw]]: failed to write index state
报错10
[WARN ][o.e.g.MetaStateService   ] [es] [[.monitoring-es-6-2019.03.22/Vd3rtLeNTX-1huKkQEHtHQ]]: failed to write index state
报错11
[WARN ][o.e.g.MetaStateService   ] [es] [[.monitoring-es-6-2019.03.23/e_6tPR-qR02L7C0sY4DN1g]]: failed to write index state
报错12
[WARN ][o.e.g.MetaStateService   ] [es] [[.monitoring-es-6-2019.03.24/RWZshlmBQWe9bEtobpEsWw]]: failed to write index state
报错13
[WARN ][o.e.g.MetaStateService   ] [es] [[.monitoring-es-6-2019.03.25/CvRjL5UsTcWTSW4t2uhx6g]]: failed to write index state
报错14
[WARN ][o.e.g.MetaStateService   ] [es] [[.kibana_1/TD4gKls0TuSEvfsSh6zkQw]]: failed to write index state
报错15
[WARN ][o.e.g.MetaStateService   ] [es] [[.watches/91mld4Y0QoiOYqzxJRzPeA]]: failed to write index state
报错16
[WARN ][o.e.c.s.ClusterApplierService] [es] failed to notify ClusterStateListener
报错17
[WARN ][o.e.g.MetaStateService   ] [es] [_global]: failed to write global state

不难发现：错误1~15都是在报XXX索引-写入索引状态失败。
所以我就把除了.kibana_1和.watches以外的所有索引搜删掉了
再次启动，此时只剩下错误16，参考https://discuss.elastic.co/t/distress-elasticsearch-does-not-start/152288
我在本地虚拟机搭建了一个全新的es，然后把它的nodes文件夹拷贝过来，然后启动成功了。
事后我发现并不需要完全拷贝过来，删掉nodes文件夹后删掉所有索引，单个启动，就ok了

**

小结：遇到es集群报红，重启es集群即可。若删除了nodes文件，则需要删除所有索引，然后逐个启动就ok了。

**

最后，附两处社区相关问题提问
V2EX：https://www.v2ex.com/t/548200#reply3
stackoverflow： https://stackoverflow.com/questions/55330873/es-failed-to-write-index-state-caused-by-underlying-file-changed-by-an-external