logstash同步mongodb数据到elasticsearch

最新推荐文章于 2024-07-11 11:45:24 发布

weixin_30384217

最新推荐文章于 2024-07-11 11:45:24 发布

阅读量598

点赞数

文章标签：数据库大数据

原文链接：http://www.cnblogs.com/a-du/p/8298418.html

版权

一、安装logstash

二、安装mongodb插件

cd D:\Software\ELK5.5.0\logstash-5.5.0\bin

logstash-plugin install logstash-output-mongodb

logstash-plugin install logstash-input-mongodb

要先装output再装input，不然会报mongo版本冲突

D:\Software\ELK5.5.0\logstash-5.5.0\bin\mongo\productinfo.conf配置文件如下：

input {
  mongodb {
    uri => "mongodb://username:password@192.168.1.111:7001/Mall_search?ssl=false"
    placeholder_db_dir => "D:\opt\logstash-mongodb"
    placeholder_db_name => "eb_search_mongo.db"
    collection => "ProductInfo_ObjectId"
    batch_size => 5
    type => "product"    
  }
}
filter {
mutate {
remove_field => [ "_id" ]
}
}

output {
    elasticsearch {
        hosts => ["192.168.1.58:9200"]
        index => "sync_mongo_productinfo_201801166666677"
        document_id => "%{ProductID}"        
    }
    stdout { codec => rubydebug }
}