基于docker 搭建elk环境搜集nginx日志

**

基于docker 搭建elk环境搜集nginx日志

**
相关拓扑图

一、获取相关镜像，镜像版本号最好保持一致，避免后期问题。本章版本号用的是7.2.0

1. docker pull elasticsearch:版本号
2. docker pull logstash:版本号
3. docker pull kibana:版本号

二、部署Elasticsearch

创建相关挂着目录
mkdir -p /home/elasticsearch/config
mkdir -p /home/elasticsearch/data
echo “http.host: 0.0.0.0”>>/home/elasticsearch/config/elasticsearch.yml
chmod -R 777 /home/elasticsearch/

运行容器

docker run --privileged=true --name elasticsearch -p 9200:9200 \
 -p 9300:9300 \
 -e "discovery.type=single-node" \
 -e ES_JAVA_OPTS="-Xms64m -Xmx128m" \
 -v /home/elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml \
 -v /home/elasticsearch/data:/usr/share/elasticsearch/data \
 -v /home/elasticsearch/plugins:/usr/share/elasticsearch/plugins \
 -d elasticsearch:7.2.0

配置跨域（可以不用配置）
修改/home/elasticsearch/config/elasticsearch.yml文件

cluster.name: "docker-cluster"
network.host: 0.0.0.0
# 访问ID限定，0.0.0.0为不限制，生产环境请设置为固定IP
transport.host: 0.0.0.0
# elasticsearch节点名称
node.name: node-1
# elasticsearch节点信息
cluster.initial_master_nodes: ["node-1"]
# 下面的配置是关闭跨域验证（可以不开启）
http.cors.enabled: true
http.cors.allow-origin: "*"

检查部署结果
curl http://localhost

三、部署Kibana

kibana不用做啥配置，只需运行容器时link一下es就好。

docker run --name kibana --link elasticsearch:elasticsearch -p 5601:5601 -d kibana:7.2.0

检查一下效果：http://ip:5601

四、部署Logstash

创建相关挂载目录
mkdir -p /home/elk/logstash/pipeline/pipeline/
mkdir -p /home/elk/logs
编辑配置文件入下：vi /home/elk/logstash/pipeline/pipeline/logstash.conf

docker run -d -p 5044:5044 --name logstash -v /home/elk/logs:/opt/logs -v /home/elk/logstash/pipeline/pipeline/logstash.conf:/etc/logstash.conf --link elasticsearch:elasticsearch logstash:7.2.0 logstash -f /etc/logstash.conf

注：/home/elk/logs为nginx 日志的存放路径
可以使用 ps aux|grep nginx 命令来查看nginx配置路径
如果更改过nginx日志存放目录，请先检查nginx再重启生效。
/usr/local/nginx/sbin/nginx -t #检查
/usr/local/nginx/sbin/nginx -s reload #重启
nginx.conf配置文件如下：

#user  nobody;
worker_processes  1;

#error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;

#pid        logs/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       mime.types;
    default_type  application/octet-stream;

    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';

    #access_log  logs/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  65;

    #gzip  on;

    log_format json '{"@timestamp":"$time_iso8601",'
                    '"@version":1,'
                    '"host":"$server_addr",'
                    '"client":"$remote_addr",'
                    '"size":$body_bytes_sent,'
                    '"responsetime":$request_time,'
                    '"domain":"$host",'
                    '"url":"$uri",'
                    '"status":"$status"}';
 
    access_log /home/elk/logs/access.log  json;

    server {
        listen       80;
        server_name  localhost;

        #charset koi8-r;

        #access_log  logs/host.access.log  main;

        location / {
            root   html;
            index  index.html index.htm;
        }

        #error_page  404              /404.html;

        # redirect server error pages to the static page /50x.html
        #
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }

        # proxy the PHP scripts to Apache listening on 127.0.0.1:80
        #
        #location ~ \.php$ {
        #    proxy_pass   http://127.0.0.1;
        #}

        # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
        #
        #location ~ \.php$ {
        #    root           html;
        #    fastcgi_pass   127.0.0.1:9000;
        #    fastcgi_index  index.php;
        #    fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
        #    include        fastcgi_params;
        #}

        # deny access to .htaccess files, if Apache's document root
        # concurs with nginx's one
        #
        #location ~ /\.ht {
        #    deny  all;
        #}
    }


    # another virtual host using mix of IP-, name-, and port-based configuration
    #
    #server {
    #    listen       8000;
    #    listen       somename:8080;
    #    server_name  somename  alias  another.alias;

    #    location / {
    #        root   html;
    #        index  index.html index.htm;
    #    }
    #}


    # HTTPS server
    #
    #server {
    #    listen       443 ssl;
    #    server_name  localhost;

    #    ssl_certificate      cert.pem;
    #    ssl_certificate_key  cert.key;

    #    ssl_session_cache    shared:SSL:1m;
    #    ssl_session_timeout  5m;

    #    ssl_ciphers  HIGH:!aNULL:!MD5;
    #    ssl_prefer_server_ciphers  on;

    #    location / {
    #        root   html;
    #        index  index.html index.htm;
    #    }
    #}

}

五、访问kibana

创建索引

查看日志

如果使用集成镜像可以参考以下链接搭建环境：

相关文章链接: link