[ELK安装篇]：基于Docker虚拟容器化(主要LogStash)

一：前置准备-(参考之前博客)：

1.1：准备Elasticsearch和Kibana环境：

1.1.1：地址：https://blog.csdn.net/Abraxs/article/details/128517777

二：Docker安装LogStash(数据收集引擎，具有实时管道功能)：

2.1：拉取logstash镜像：

[root@vboxnode3ccccccttttttchenyang ~]# docker pull logstash:6.6.0
6.6.0: Pulling from library/logstash
a02a4930cb5d: Pull complete
83cf3eaa08e1: Pull complete
162671d94cae: Pull complete
c678994d9b6f: Pull complete
af6d15336df1: Pull complete
b808ea4c2d1f: Pull complete
dc25014ab499: Pull complete
16159f779f38: Pull complete
521d45965a24: Pull complete
30fa9d097a91: Pull complete
066458677680: Pull complete
Digest: sha256:08bc3d552c6ec2d47e3970e063dfe800e3467ba1ef80fb87f37902daba9a560b
Status: Downloaded newer image for logstash:6.6.0
docker.io/library/logstash:6.6.0
[root@vboxnode3ccccccttttttcyang ~]# docker images
REPOSITORY               TAG       IMAGE ID       CREATED         SIZE
wurstmeister/kafka       latest    2dd91ce2efe1   18 months ago   508MB
influxdb                 2.0       33f568b26cd9   19 months ago   342MB
grafana/grafana          latest    9b957e098315   19 months ago   275MB
logstash                 6.6.0     8f45a7702366   4 years ago     751MB
wurstmeister/zookeeper   latest    3f43f72cb283   4 years ago     510MB

2.2：根据logstash镜像生成容器：

命令如下
docker run -d -p 9600:9600 -p 4560:4560 --name logstash logstash:6.6.0

[root@vboxnode3ccccccttttttcyang ~]# docker run -d -p 9600:9600 -p 4560:4560 --name logstash logstash:6.6.0
ae0fbdac223afd98a6b00cfb30f312d58217b725342848ad59370d9e5f7a18db

2.3：查看docker所有容器：

[root@vboxnode3ccccccttttttcyang ~]# docker ps
CONTAINER ID   IMAGE                    COMMAND                   CREATED         STATUS             PORTS                                                                                            NAMES
ae0fbdac223a   logstash:6.6.0           "/usr/local/bin/dock…"   3 seconds ago   Up 1 second        0.0.0.0:4560->4560/tcp, :::4560->4560/tcp, 0.0.0.0:9600->9600/tcp, :::9600->9600/tcp, 5044/tcp   logstash
29753343c1b0   wurstmeister/zookeeper   "/bin/sh -c '/usr/sb…"   2 months ago    Up About an hour   22/tcp, 2888/tcp, 3888/tcp, 0.0.0.0:2181->2181/tcp, :::2181->2181/tcp  

三：开放防火墙LogStash对应的端口：

[root@vboxnode3ccccccttttttcyang ~]# firewall-cmd --permanent -zone=public --add-port=9600/tcp
usage: see firewall-cmd man page
firewall-cmd: error: unrecognized arguments: -zone=public
[root@vboxnode3ccccccttttttcyang ~]# firewall-cmd --permanent --zone=public --add-port=9600/tcp
success
[root@vboxnode3ccccccttttttcyang ~]# firewall-cmd --permanent --zone=public --add-port=4560/tcp
success
[root@vboxnode3ccccccttttttcyang ~]# firewall-cmd --reload
success
[root@vboxnode3ccccccttttttcyang ~]# firewall-cmd --permanent --zone=public --list-port
3306/tcp 8848/tcp 8091/tcp 8879/tcp 8887/tcp 8083/tcp 8086/tcp 8090/tcp 8099/tcp 9600/tcp 4560/tcp

四：修改LogStash配置[logstash.(yml / config)]：

4.1：进入容器修改logstash.yml配置文件：

4.1.1：也可以尝试通过挂在目录在宿主机配置文件操作

docker exec -it ae0fbdac223a /bin/sh

[root@vboxnode3ccccccttttttchenyang ~]# docker exec -it ae0fbdac223a /bin/sh
sh-4.2$ ls
bin     CONTRIBUTORS  Gemfile       lib          logstash-core             modules     pipeline  vendor
config  data          Gemfile.lock  LICENSE.txt  logstash-core-plugin-api  NOTICE.TXT  tools     x-pack
sh-4.2$ cd config/
sh-4.2$ ls
jvm.options  log4j2.properties  logstash-sample.conf  logstash.yml  pipelines.yml  startup.options
sh-4.2$
sh-4.2$ vi logstash.yml

配置文件默认信息：

4.1.2：修改默认配置>：改成如下配置

http.host: "0.0.0.0"
xpack.monitoring.elasticsearch.url: http://192.yourIp.103:9200
# 如果有密码执行下面
xpack.monitoring.elasticsearch.username: elastic 
xpack.monitoring.elasticsearch.password: 123456

4.2：进入容器修改pipeline文件夹下的logstash.conf配置文件：

[root@vboxnode3ccccccttttttchenyang ~]# docker exec -it ae0fbdac223a /bin/sh
sh-4.2$ ls
bin     CONTRIBUTORS  Gemfile       lib          logstash-core             modules     pipeline  vendor
config  data          Gemfile.lock  LICENSE.txt  logstash-core-plugin-api  NOTICE.TXT  tools     x-pack
sh-4.2$ cd config/
sh-4.2$ ls
jvm.options  log4j2.properties  logstash-sample.conf  logstash.yml  pipelines.yml  startup.options
sh-4.2$ cd ..
sh-4.2$ ls
bin     CONTRIBUTORS  Gemfile       lib          logstash-core             modules     pipeline  vendor
config  data          Gemfile.lock  LICENSE.txt  logstash-core-plugin-api  NOTICE.TXT  tools     x-pack
sh-4.2$ cd pipeline/
sh-4.2$ ls
logstash.conf
sh-4.2$ vi logstash.conf
sh-4.2$
sh-4.2$ pwd
/usr/share/logstash/pipeline

4.2.1：配置文件默认信息：

input {
  beats {
    port => 5044
  }
}

output {
  stdout {
    codec => rubydebug
  }
}

4.2.2：修改填充默认配置文件信息：

input {
  tcp {
    mode => "server"
    #logstash配置地址
    host => "0.0.0.0"
    port => 4560
    #日志文件json输出
    codec => json_lines
 }
}
output {
  elasticsearch {
	action => "index"
    # ES服务地址，也可以是数组多地址
    hosts  => ["192.168.56.102:9200"]
    # Es index > 类似DB数据库
    index  => "index-logstash"
  }	
  stdout {
    codec => rubydebug
  }
}

4.3：重启logstash容器[docker restart containerid]