ElasticSearch docker环境部署

ElasticSearch

ElasticSearch下载镜像

docker pull docker.elastic.co/elasticsearch/elasticsearch:7.10.0

ElasticSearch创建挂载的目录

mkdir -p /mydata/elasticsearch/config
mkdir -p /mydata/elasticsearch/data
echo "http.host: 0.0.0.0" >> /mydata/elasticsearch/config/elasticsearch.yml

下载elasticsearch
docker pull docker.elastic.co/elasticsearch/elasticsearch:7.10.0
​
运行elasticsearch容器
docker run --name elasticsearch -p 9200:9200 -p 9400:9400  -e "discovery.type=single-node" -e ES_JAVA_OPTS="-Xms256m -Xmx256m" -v /elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml -v /elasticsearch/data:/usr/share/elasticsearch/data -v /elasticsearch/plugins:/usr/share/elasticsearch/plugins -d docker.elastic.co/elasticsearch/elasticsearch:7.10.0
​
docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:7.10.0

其中elasticsearch.yml是挂载的配置文件，data是挂载的数据，plugins是es的插件，如ik，而数据挂载需要权限，需要设置data文件的权限为可读可写,需要下边的指令。 chmod -R 777 要修改的路径

-e "discovery.type=single-node" 设置为单节点 特别注意： -e ES_JAVA_OPTS="-Xms256m -Xmx256m" \ 测试环境下，设置ES的初始内存和最大内存，否则导致过大启动不了ES

kibana

https://www.elastic.co/guide/en/kibana/7.9/docker.html#docker

下载kibana

docker pull docker.elastic.co/kibana/kibana:7.9.3

运行kibana容器

docker run --name kibana -e ELASTICSEARCH_HOSTS=http://IP地址:9200 -e "SERVER_NAME=jm-kbn" -e "I18N_LOCALE=zh-CN" -p 5601:5601 -d docker.elastic.co/kibana/kibana:7.9.3

server.port: 5601 server.host: 0.0.0.0 elasticsearch.hosts: [ "http://192.168.149.128:9200" ] i18n.locale: "Zh-CN"

Logstash

https://www.elastic.co/guide/en/logstash/7.9/docker-config.html

下载Logstash

docker pull docker.elastic.co/logstash/logstash:7.9.3

Logstash配置

mkdir -p /usr/local/src/docker_logstash
mkdir -p /usr/local/src/docker_logstash/logs
  
vi log4j2.properties
logger.elasticsearchoutput.name = logstash.outputs.elasticsearch
logger.elasticsearchoutput.level = debug

logstash.yml配置文件

http.host: "0.0.0.0"
path.config : /usr/local/src/docker_logstash/config.conf
path.logs : /usr/local/src/docker_logstash/logs

pipelines.yml配置文件

pipeline.id: table1
path.config: "/usr/local/src/docker_logstash/config.conf"

config.conf配置文件内容

  input {
    jdbc {
    jdbc_driver_library => "/usr/share/logstash/config/conf.d/mysql-connector-java-8.0.19.jar"
    jdbc_driver_class => "com.mysql.jdbc.Driver"
    jdbc_connection_string => "jdbc:mysql://IP地址/aos-log?useUnicode=true&characterEncoding=utf-8&serverTimezone=Asia/Shanghai"
    jdbc_user => "root"
    jdbc_password => "admin"
    #schedule => "* * * * *"
    statement => " SELECT * FROM app_userlog_20201016 u order by data_time desc"
    #tracking_column => "updateTime"
    #tracking_column_type => "timestamp"
    # 是否将 字段(column) 名称转小写
    lowercase_column_names => false
    }
    }
    output {
    elasticsearch {
    hosts => "http://IP地址:9200/"    codec => json
    index => "my_log"
    }
    stdout {}
  }
​

启动Logstash

docker run -d -p 5044:5044 -p 9600:9600 -it --name logstash -v /usr/local/src/docker_logstash:/usr/share/logstash/config docker.elastic.co/logstash/logstash:7.9.3  /bin/bash

启动后进入容器

cd /usr/share/logstash/bin
​
/bin/logstash -f /config/config.conf

启动成功后

http://IP地址:5601/

进入kibana目录中选择Stack Management管理索引信息，在kabana索引模式中创建索引模式，