记一次ELK从5.5.x升级到6.0.0的过程

前言:


1.部署的ELK架构为Elasticsearch(以下简称ES)+Logstash+Kibana+Filebeat

2.Filebeat部署在需要收集日志的节点上,负责收集日志。接着交由Logstash和ES过滤分析,然后传输并集中在Kibana系统上进行可视化展示

3.非集群部署none-cluster

4.其中Elasticsearch,Kibana均为由5.5.2 → 6.0.0 ;而Logstash即由5.5.2 → 6.0.1;Filebeat 5.5.2 → 5.6.5 → 6.0.1

4.以下命令过长,可以保存成sh脚本文件并添加执行权限执行


1.Eleasticsearch部分:

(参考官网文档:点击我


在ES服务器上通过curl发起PUT请求修改集群配置参数:

(1)停止碎片分配

curl -XPUT '10.0.7.227:9200/_cluster/settings?pretty' -H 'Content-Type: application/json' -d'
{
  "persistent": {
    "cluster.routing.allocation.enable": "none"
  }
}
'


(2)暂停索引,然后刷新同步状态

curl -XPOST '10.0.7.227:9200/_flush/synced?pretty'


(3)暂停所有节点,此处由于部署时候没有做成系统级别服务,直接kill杀死进程

kill -9 $(cat pid)


(4)升级软件版本

解压缩tar或者zip包到一个新目录下,不要覆盖旧版本的目录

然后修改配置文件config/elasticsearch.yml中的path.data和path.logs指向原来的数据和日志目录


(5)升级所有插件到兼容版本


(6)启动ES,若部署为集群则启动所有节点的ES

使用以下命令查看节点状态

curl -XGET 'localhost:9200/_cat/health?pretty'
curl -XGET 'localhost:9200/_cat/nodes?pretty'

等待节点状态显示为黄色(完成本地数据恢复的状态)


(8)恢复碎片分配,恢复正常使用

curl -H "Content-Type:application/json" -XPUT 10.0.7.227:9200/_cluster/settings -d '{
  "persistent" : {
     "cluster.routing.allocation.enable": "all"
}
}'

执行结果如下




2.Kibana部分:

(官网升级文档)


(1)解压zip或者tar包到新目录,不要覆盖旧版本的配置和数据目录


(2)迁移配置文件目录config到新版本的目录下,并且检查和升级配置文件相关项目,删除废弃项


(3)迁移ES上和Kibana相关的index项设置


设置.kibana的index空间为只读

curl -XPUT '10.0.7.227:9200/.kibana/_settings?pretty' -H 'Content-Type: application/json' -d'
{
  "index.blocks.write": true
}
'


创建新的index空间.kibana-6

curl -XPUT '10.0.7.227:9200/.kibana-6?pretty' -H 'Content-Type: application/json' -d'
{
  "settings" : {
    "number_of_shards" : 1,
    "index.mapper.dynamic": false
  },
  "mappings" : {
    "doc": {
      "properties": {
        "type": {
          "type": "keyword"
        },
        "updated_at": {
          "type": "date"
        },
        "config": {
          "properties": {
            "buildNum": {
              "type": "keyword"
            }
          }
        },
        "index-pattern": {
          "properties": {
            "fieldFormatMap": {
              "type": "text"
            },
            "fields": {
              "type": "text"
            },
            "intervalName": {
              "type": "keyword"
            },
            "notExpandable": {
              "type": "boolean"
            },
            "sourceFilters": {
              "type": "text"
            },
            "timeFieldName": {
              "type": "keyword"
            },
            "title": {
              "type": "text"
            }
          }
        },
        "visualization": {
          "properties": {
            "description": {
              "type": "text"
            },
            "kibanaSavedObjectMeta": {
              "properties": {
                "searchSourceJSON": {
                  "type": "text"
                }
              }
            },
            "savedSearchId": {
              "type": "keyword"
            },
            "title": {
              "type": "text"
            },
            "uiStateJSON": {
              "type": "text"
            },
            "version": {
              "type": "integer"
            },
            "visState": {
              "type": "text"
            }
          }
        },
        "search": {
          "properties": {
            "columns": {
              "type": "keyword"
            },
            "description": {
              "type": "text"
            },
            "hits": {
              "type": "integer"
            },
            "kibanaSavedObjectMeta": {
              "properties": {
                "searchSourceJSON": {
                  "type": "text"
                }
              }
            },
            "sort": {
              "type": "keyword"
            },
            "title": {
              "type": "text"
            },
            "version": {
              "type": "integer"
            }
          }
        },
        "dashboard": {
          "properties": {
            "description": {
              "type": "text"
            },
            "hits": {
              "type": "integer"
            },
            "kibanaSavedObjectMeta": {
              "properties": {
                "searchSourceJSON": {
                  "type": "text"
                }
              }
            },
            "optionsJSON": {
              "type": "text"
            },
            "panelsJSON": {
              "type": "text"
            },
            "refreshInterval": {
              "properties": {
                "display": {
                  "type": "keyword"
                },
                "pause": {
                  "type": "boolean"
                },
                "section": {
                  "type": "integer"
                },
                "value": {
                  "type": "integer"
                }
              }
            },
            "timeFrom": {
              "type": "keyword"
            },
            "timeRestore": {
              "type": "boolean"
            },
            "timeTo": {
              "type": "keyword"
            },
            "title": {
              "type": "text"
            },
            "uiStateJSON": {
              "type": "text"
            },
            "version": {
              "type": "integer"
            }
          }
        },
        "url": {
          "properties": {
            "accessCount": {
              "type": "long"
            },
            "accessDate": {
              "type": "date"
            },
            "createDate": {
              "type": "date"
            },
            "url": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 2048
                }
              }
            }
          }
        },
        "server": {
          "properties": {
            "uuid": {
              "type": "keyword"
            }
          }
        },
        "timelion-sheet": {
          "properties": {
            "description": {
              "type": "text"
            },
            "hits": {
              "type": "integer"
            },
            "kibanaSavedObjectMeta": {
              "properties": {
                "searchSourceJSON": {
                  "type": "text"
                }
              }
            },
            "timelion_chart_height": {
              "type": "integer"
            },
            "timelion_columns": {
              "type": "integer"
            },
            "timelion_interval": {
              "type": "keyword"
            },
            "timelion_other_interval": {
              "type": "keyword"
            },
            "timelion_rows": {
              "type": "integer"
            },
            "timelion_sheet": {
              "type": "text"
            },
            "title": {
              "type": "text"
            },
            "version": {
              "type": "integer"
            }
          }
        },
        "graph-workspace": {
          "properties": {
            "description": {
              "type": "text"
            },
            "kibanaSavedObjectMeta": {
              "properties": {
                "searchSourceJSON": {
                  "type": "text"
                }
              }
            },
            "numLinks": {
              "type": "integer"
            },
            "numVertices": {
              "type": "integer"
            },
            "title": {
              "type": "text"
            },
            "version": {
              "type": "integer"
            },
            "wsState": {
              "type": "text"
            }
          }
        }
      }
    }
  }
}
'

将index由.kibana重定向到.kibana-6

curl -XPOST 'localhost:9200/_reindex?pretty' -H 'Content-Type: application/json' -d'
{
  "source": {
    "index": ".kibana"
  },
  "dest": {
    "index": ".kibana-6"
  },
  "script": {
    "inline": "ctx._source = [ ctx._type : ctx._source ]; ctx._source.type = ctx._type; ctx._id = ctx._type + \":\" + ctx._id; ctx._type = \"doc\"; ",
    "lang": "painless"
  }
}
'

将.kibana-6这个index同义为.kibana,并删除旧的index,即.kibana

curl -XPOST 'localhost:9200/_aliases?pretty' -H 'Content-Type: application/json' -d'
{
  "actions" : [
    { "add":  { "index": ".kibana-6", "alias": ".kibana" } },
    { "remove_index": { "index": ".kibana" } }
  ]
}
'


(4)同(2),迁移数据目录data到新版的目录里


(5)更新插件plugin到兼容版本


(6)停止旧的Kibana,kill杀死进程等


(7)启动新版本



3.Logstash部分

(官网文档)

官网文档没有很好的说明具体升级步骤,目前直接覆盖data和config目录,尝试启动,成功,没有报错


4.Filebeat部分

(官网文档)


直接下载rpm包升级(rpm -Uvh xxxxxx.rpm)


由于官网文档提到,Filebeat 6.0较为兼容 5.6,所以先升级到Filebeat 5.6,然后再升级到Filebeat 6.0


由5.5升级到5.6的时候一切正常


由5.6升级到6.0的时候提示配置文件已被备份,证明配置文件产生了变化,出现新的配置项或者有旧的配置项被弃用


这时候需要对比一下新旧配置文件filebeat.yml.rpmnew和filebeat.yml,并将旧的配置更新到新的配置文件里面去,并覆盖原文件filebeat.yml


此处尝试直接启用原来的配置文件filebeat.yml,然后启动service filebeat start


©️2020 CSDN 皮肤主题: 大白 设计师:CSDN官方博客 返回首页