安装LogStash并导入数据

环境

1. Centos7
2. ElasticSearch 7.1

去官网下载logstash

https://artifacts.elastic.co/downloads/logstash/logstash-7.1.0.zip 链接贴在这里了

压缩文件

unzip logstash-7.1.0.zip

下载最 MovieLens 最小测试数据

https://grouplens.org/datasets/movielens/

编写logstash 配置文件

input {
  file {
    path => "/opt/esStudy/2.4logstash/movielens/ml-latest-small/movies.csv"  // 这个路径为 你 下载 movies.cvs 的位置
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}
filter {
  csv {
    separator => ","
    columns => ["id","content","genre"]
  }

  mutate {
    split => { "genre" => "|" }
    remove_field => ["path", "host","@timestamp","message"]
  }

  mutate {

    split => ["content", "("]
    add_field => { "title" => "%{[content][0]}"}
    add_field => { "year" => "%{[content][1]}"}
  }

#  mutate {

#    gsub => [
#
#      "year", "\\)", ""
#    ]
#  }


  mutate {
    convert => {
      "year" => "integer"
    }
    strip => ["title"]
    remove_field => ["path", "host","@timestamp","message","content"]
  }
   

进入到logstash 的bin 目录。运行

./logstash -f movies/logstash.conf

-f 指定配置文件 logstash.conf 是上面编写的配置文件

数据导入成功截图

出现上线就说明你的数据导入成功啦！