【ElasticSearch入门】1、elasticsearch、kibana、logstash的安装

【ElasticSearch入门】1、elasticsearch、kibana、logstash的安装

一、安装

https://www.elastic.co/cn/downloads/elasticsearch

下载对应版本后，执行bin/elasticsearch 运行es

运行es之后，输入localhost:9200 查看es是否运行成功

运行成功！

安装分词插件。

bin/elasticsearch-plugin install analysis-icu

查看已安装插件

bin/elasticsearch-plugin list

安装kibana

注意：kibana版本要与es相同

https://www.elastic.co/downloads/kibana

启动kibana

bin/kibana

测试kibana

http://localhost:5601/

二、使用docker运行elasticsearch

使用docker可以使运行环境与本机隔离，从而使你可以通过多个docker容器取运行多个es实例。

1. 安装docker https://www.docker.com/products/docker-desktop

2. 在文件夹中创建一个开源的ELK配置

   version: '2.2'
   services:
     cerebro:
       image: lmenezes/cerebro:0.8.3
       container_name: cerebro
       ports:
         - "9000:9000"
       command:
         - -Dhosts.0.host=http://elasticsearch:9200
       networks:
         - es7net
     kibana:
       image: docker.elastic.co/kibana/kibana:7.1.0
       container_name: kibana7
       environment:
         - I18N_LOCALE=zh-CN
         - XPACK_GRAPH_ENABLED=true
         - TIMELION_ENABLED=true
         - XPACK_MONITORING_COLLECTION_ENABLED="true"
       ports:
         - "5601:5601"
       networks:
         - es7net
     elasticsearch:
       image: docker.elastic.co/elasticsearch/elasticsearch:7.1.0
       container_name: es7_01
       environment:
         - cluster.name=geektime
         - node.name=es7_01
         - bootstrap.memory_lock=true
         - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
         - discovery.seed_hosts=es7_01,es7_02
         - cluster.initial_master_nodes=es7_01,es7_02
       ulimits:
         memlock:
           soft: -1
           hard: -1
       volumes:
         - es7data1:/usr/share/elasticsearch/data
       ports:
         - 9200:9200
       networks:
         - es7net
     elasticsearch2:
       image: docker.elastic.co/elasticsearch/elasticsearch:7.1.0
       container_name: es7_02
       environment:
         - cluster.name=geektime
         - node.name=es7_02
         - bootstrap.memory_lock=true
         - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
         - discovery.seed_hosts=es7_01,es7_02
         - cluster.initial_master_nodes=es7_01,es7_02
       ulimits:
         memlock:
           soft: -1
           hard: -1
       volumes:
         - es7data2:/usr/share/elasticsearch/data
       networks:
         - es7net
   
   
   volumes:
     es7data1:
       driver: local
     es7data2:
       driver: local
   
   networks:
     es7net:
       driver: bridge
      
   

3. 本地启动docker，mac记得运行docker桌面版,进入设置Resources调大内存 防止oom
   

4、文件夹内执行 docker-compose up

依次执行localhost:9200 5601 9000

启动成功！！！

三、安装logstash与测试数据导入

logstash版本要与es相同

https://www.elastic.co/cn/downloads/logstash

下载测试数据集

https://grouplens.org/datasets/movielens/

下载

ml-latest-small.zip 即可

在logstash 的bin目录下新建logstash.conf

input {
  file {
    path => "/Users/yiruan/dev/elk7/logstash-7.0.1/bin/movies.csv"  目录改成你的
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}
filter {
  csv {
    separator => ","
    columns => ["id","content","genre"]
  }

  mutate {
    split => { "genre" => "|" }
    remove_field => ["path", "host","@timestamp","message"]
  }

  mutate {

    split => ["content", "("]
    add_field => { "title" => "%{[content][0]}"}
    add_field => { "year" => "%{[content][1]}"}
  }

  mutate {
    convert => {
      "year" => "integer"
    }
    strip => ["title"]
    remove_field => ["path", "host","@timestamp","message","content"]
  }

}
output {
   elasticsearch {
     hosts => "http://localhost:9200"
     index => "movies"
     document_id => "%{id}"
   }
  stdout {}
}

运行

#启动Elasticsearch实例，然后启动 logstash，并制定配置文件导入数据
sudo bin/logstash -f /YOUR_PATH_of_logstash.conf 
#改成你的配置文件地址

等一等 导入成功