1 下载
在官网下载和elasticsearch相同版本的logstash
2.安装
上传到对应目录
解压
tar -zxvf logstash-7.1.0.tar.gz
3.集成elasticsearch
创建配置文件 我放在config 目录下
配置文件内容如下
input {
file {
# 这里是我的测试文件的路径
path => "/exfts/elk/logstash/file/movies.csv"
start_position => "beginning"
sincedb_path => "/dev/null"
}
}
filter {
csv {
separator => ","
columns => ["id","content","genre"]
}
mutate {
split => { "genre" => "|" }
remove_field => ["path", "host","@timestamp","message"]
}
mutate {
split => ["content", "("]
add_field => { "title" => "%{[content][0]}"}
add_field => { "year" => "%{[content][1]}"}
}
mutate {
convert => {
"year" => "integer"
}
strip => ["title"]
remove_field => ["path", "host","@timestamp","message","content"]
}
}
output {
elasticsearch {
#elasticsearch 的地址
hosts => "http://localhost:9200"
index => "movies"
document_id => "%{id}"
}
stdout {}
}
4.启动
bin/logstash -f /YOUR_PATH_of_logstash.conf
我这里是
./bin/logstash -f ./config/movies.conf
然后可以到相关插件查看数据是否导入成功
我这里用的是cerebro,相关安装可以看上一章节