-
下载es, kibana和logstash,注意三者最好使用相同版本
es下载地址
kibana下载地址
logstash下载地址 -
进行安装:我使用的是mac系统,所以将下载的压缩包解压即可。
-
启动es:进到es目录,然后执行:
./bin/elasticsearch
- 浏览器访问9200端口,发现已经启动:
- 下载测试数据,从https://grouplens.org/datasets/movielens/下载ml-latest-small.zip,并解压。
- 增加logstash配置文件,path是测试文件路径,根据实际情况改写:
input {
file {
path => "/Users/xxxx/Downloads/logstash-7.10.2/movies.csv"
start_position => "beginning"
sincedb_path => "/dev/null"
}
}
filter {
csv {
separator => ","
columns => ["id","content","genre"]
}
mutate {
split => { "genre" => "|" }
remove_field => ["path", "host","@timestamp","message"]
}
mutate {
split => ["content", "("]
add_field => { "title" => "%{[content][0]}"}
add_field => { "year" => "%{[content][1]}"}
}
mutate {
convert => {
"year" => "integer"
}
strip => ["title"]
remove_field => ["path", "host","@timestamp","message","content"]
}
}
output {
elasticsearch {
hosts => "http://localhost:9200"
index => "movies"
document_id => "%{id}"
}
stdout {}
}
- 启动logstash,将测试数据导入到es,进入到logstash安装目录后执行命令:
sudo ./bin/logstash -f logstash.conf
- 命令行会打印导入的数据:
- 进入到kibana安装目录执行命令启动kibana:
./bin/kibana
- 通过浏览器访问5601端口,进入kibana:
- 可以看到es中已经有数据了,但是我们没有创建索引所以在discover中没有显示。
- 点击创建按钮进行索引创建,索引名称支持正则,需要能够匹配到logstash配置文件中output的index:
- 创建后可以看到索引包含哪些字段:
- 回到discover,已经可以看到导入的数据了:
遇到的坑
- es启动报:max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
- 解决方法:切换到root用户修改配置/etc/sysctl.conf,在尾行添加
vm.max_map_count=262144
并执行命令 sysctl -p
- 安装glibc:https://blog.csdn.net/changcsw/article/details/79761620
- 安装autoconf:https://blog.csdn.net/testcs_dn/article/details/45437903
- 安装libstdc: http://php-note.com/article/1734.html