docker配置ELK环境(win7)

---------------------elasticsearch------------------------------
1.下载镜像
docker pull elasticsearch:7.9.3

2.创建挂载的目录
-v /f/docker/chench/elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
-v /f/docker/chench/elasticsearch/data:/usr/share/elasticsearch/data
-v /f/docker/chench/elasticsearch/plugins:/usr/share/elasticsearch/plugins
echo “http.host: 0.0.0.0” >> /f/docker/chench/elasticsearch/config/elasticsearch.yml

3.创建容器并启动
docker run -it --privileged=true --restart always --name elasticsearch -p 9301:9301 -p 9302:9302 -e “discovery.type=single-node” -e ES_JAVA_OPTS="-Xms128m -Xmx256m" -v /f/docker/chench/elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml -v /f/docker/chench/elasticsearch/data:/usr/share/elasticsearch/data -v /f/docker/chench/elasticsearch/plugins:/usr/share/elasticsearch/plugins -d elasticsearch:7.9.3

其中elasticsearch.yml是挂载的配置文件，data是挂载的数据，plugins是es的插件，如ik，而数据挂载需要权限，需要设置data文件的权限为可读可写,需要下边的指令。
chmod -R 777 要修改的路径

-e “discovery.type=single-node” 设置为单节点
特别注意：
-e ES_JAVA_OPTS="-Xms256m -Xmx256m" \ 设置ES的初始内存和最大内存，否则导致过大启动不了ES

docker exec -it 6c9314710 /bin/bash
docker stop elasticsearch
docker start elasticsearch
docker restart elasticsearch
docker logs --tail 1000 -f elasticsearch

http://192.168.99.100:9301/

---------------------Kibana---------------------
docker pull kibana:7.9.3

创建挂载的目录
-v /f/docker/chench/kibana/config/kibana.yml:/usr/share/kibana/config/kibana.yml
-v /f/docker/chench/kibana/data:/usr/share/kibana/data

创建配置文件,并添加下面内容>> /f/docker/chench/kibana/config/kibana.yml
server.port: 9306
server.host: 0.0.0.0
elasticsearch.hosts: [ “http://192.168.99.100:9301” ]
i18n.locale: “zh-CN”
–太小内存会导致启动失败
docker run -itd --privileged=true --restart always -m 1G --name kibana -e ELASTICSEARCH_HOSTS=http://192.168.99.100:9301 -p 9306:9306 -v /f/docker/chench/kibana/config/kibana.yml:/usr/share/kibana/config/kibana.yml -v /f/docker/chench/kibana/data:/usr/share/kibana/data -d kibana:7.9.3

docker exec -it 8397aa0af4a /bin/bash
docker stop kibana
docker start kibana
docker restart kibana
docker logs --tail 1000 -f kibana
然后访问页面
http://192.168.99.100:9306/app/kibana

---------------------logstash---------------------
docker pull logstash:7.9.3

安装需要挂载config目录，并需要4个文件

logstash.yml

http.host: “0.0.0.0”
path.logs: “/usr/share/logstash/logs”
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.hosts: [“http://192.168.99.100:9301”]

pipelines.yml

• pipeline.id: “chench-logstash”
  path.config: “/usr/share/logstash/config/*.conf”
  pipeline.workers: 2

log4j2.properties

status = error
name = LogstashPropertiesConfig
property.filename = /usr/share/logstash/logs

appender.console.type = Console
appender.console.name = plain_console
appender.console.layout.type = PatternLayout
appender.console.layout.pattern = [%d{ISO8601}][%-5p][%-25c] %m%n

appender.json_console.type = Console
appender.json_console.name = json_console
appender.json_console.layout.type = JSONLayout
appender.json_console.layout.compact = true
appender.json_console.layout.eventEol = true

appender.rolling.type = RollingFile
appender.rolling.name = RollingFile
appender.rolling.fileName = ${filename}/logstash.log
appender.rolling.filePattern = ${filename}/logstash-%d{yyyy-MM-dd}-%i.log.gz
appender.rolling.layout.type = PatternLayout
appender.rolling.layout.pattern = %d %p %C{1.} [%t] %m%n
appender.rolling.policies.type = Policies
appender.rolling.policies.time.type = TimeBasedTriggeringPolicy
appender.rolling.policies.time.interval = 1
appender.rolling.policies.time.modulate = true
appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
appender.rolling.policies.size.size=10MB
appender.rolling.strategy.type = DefaultRolloverStrategy
appender.rolling.strategy.max = 1

rootLogger.level = info
rootLogger.appenderRef.rolling.ref = RollingFile
rootLogger.appenderRef.console.ref = ${sys:ls.log.format}_console

这个复制修改日志存放地址就可以了

logstash.conf

input {
  gelf {
    port => 9110
    use_udp => true
    port_udp => 9110
    use_tcp => true
    port_tcp => 9120
    codec => json {
      charset => "UTF-8"
    }
  }
}
filter {
  mutate {
    remove_field => ["@timestamp","@version","server"]
  }
}
output {
  stdout {
    codec => rubydebug
  }
  elasticsearch {
    hosts => ["192.168.99.100:9301"]
    index => "log4j2-%{appName}"
    #-%{+YYYY-MM-dd}
    #document_id => "%{traceId}"
  }
}

这里我用的log4j2配置+logstash-gelf udp 9110端口推送日志

log4j2.xml

!-- 192.168.99.100为logstash主机IP，9110为logstash端口 -->
    	<!-- 输出日志到Logstash 中做日志收集 host="tcp:localhost" UDP不保证可靠性,运行速度较快-->
    	<Gelf name="logstash-gelf" host="udp:192.168.99.100" port="9110" version="1.1" extractStackTrace="true"
          filterStackTrace="true" mdcProfiling="true" includeFullMdc="true" maximumMessageSize="8192"
          originHost="%host{fqdn}" additionalFieldTypes="fieldName1=String,fieldName2=Double,fieldName3=Long">
	      <Field name="timestamp" pattern="%d{yyyy-MM-dd HH:mm:ss,SSS}" />
	      <Field name="level" pattern="%level" />
	      <Field name="className" pattern="%C" />
	      <Field name="server" pattern="%host" />
	      <Field name="server.fqdn" pattern="%host{fqdn}" />
	      <Field name="message" pattern="%message" />
	      <!-- 配置程序名,用于index 
	      <Field name="appName" literal="${FILE_NAME}" />-->
	      <!-- 配置程序名,用于index -->
	      <Field name="appName" pattern="${FILE_NAME}-%d{yyyy-MM-dd}" />
	      <!-- ThreadContext.put("traceId", "1"); mdc设置值 -->
	      <Field name="traceId" mdc="traceId" />
	    </Gelf>

<appender-ref ref="logstash-gelf" />

docker run -d --privileged=true --restart=always -p 9110:9110/udp -p 9120:9120 -p 5044:5044 -p 9600:9600 --name logstash -v /f/docker/chench/logstash/config/:/usr/share/logstash/config/ -v /f/docker/chench/logstash/data/:/usr/share/logstash/data/ -v /f/docker/chench/logstash/logs/:/usr/share/logstash/logs/ logstash:7.9.3

docker logs --tail 1000 -f logstash