#### 前言
logstash的部署方式采用docker
#### 准备工作
##### 时区文件
保证容器服务的时间与宿主机的时间一致
```
cat > /etc/timezone <<-EOF
Asia/Shanghai
EOF
```
##### logstash.yml文件
```
cat > /data/logstash/config/logstash.yml <<-EOF
http.host: 0.0.0.0
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.hosts: url/ip:9200 # 建议: 云主机上部署,url采用内网的url,同理,ip也是采用内网的ip
xpack.monitoring.elasticsearch.username: elasticsearch_username
xpack.monitoring.elasticsearch.password: elasticsearch_password
EOF
```
##### logstash.conf文件
```
cat > /data/logstash/pipeline/logstash.conf <<-EOF
input {
beats {
port => 5044 # 本机部署的logstash端口,注:是容器暴露在宿主机的端口
codec => plain { charset => "UTF-8" } # 由于从filebeat段推送到logstash的日志文件不是json格式的,同时存在中文字符,故采用plain格式,并对数据进行UTF-8编码转换
}
}
filter {
grok {
match => { "message" => "\[(?<log_time>\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\s\+\d{4})\]\s+\[(?<log_status>\d+)\]\s+\[(?<log_level>[a-zA-Z]*)\]\s+(?<log_msg>.*)" }
}
}
output {
elasticsearch {
action => "index"
hosts => ["url/ip:9200"] # 建议: 云主机上部署,url采用内网的url,同理,ip也是采用内网的ip
index => "logstash-dev-img-%{+YYYYMMdd}" # logstash-dev-img,此内容,可自定义
user => "elasticsearch_username"
password => "elasticsearch_password"
}
}
EOF
```
注释:用于json格式化文件的input写法
```
input {
tcp {
port => 5044
codec => "json_lines" # 每行读取json序列化数据
}
}
```
#### 部署方式1:sh脚本形式部署docker服务
```
cat > docker-logstash-root.sh <<-EOF
#!/usr/bin/env bash
docker run -d \
--privileged=true \
-u root \
--name logstash \
--restart always \
-p 5044:5044 \
-v /etc/localtime:/etc/localtime \
-v /etc/timezone:/etc/timezone \
-v /data/logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml \
-v /data/logstash/pipeline/logstash.conf:/usr/share/logstash/pipeline/logstash.conf \
docker.elastic.co/logstash/logstash:7.6.2
EOF
```
> bash docker-logstash-root.sh
#### 部署方式2:docker-compose形式部署docker服务
```
cat > docker-compose.yml <<-EOF
version: "3.5"
services:
logstash:
image: docker.elastic.co/logstash/logstash:7.6.2
container_name: logstash
hostname: logstash-
privileged: true
user: root
ports:
- 5044:5044
volumes:
- /etc/localtime:/etc/localtime:ro
- /etc/timezone:/etc/timezone:ro
- /data/logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml:ro
- /data/logstash/pipeline/logstash.conf:/usr/share/logstash/pipeline/logstash.conf:ro
restart: always
tty: true
EOF
```
> docker-compose up -d