logstash input es output jdbc

最新推荐文章于 2024-08-19 20:18:19 发布

潇潇一夜风兼雨

最新推荐文章于 2024-08-19 20:18:19 发布

阅读量1.4k

点赞数

分类专栏： es

本文链接：https://blog.csdn.net/u011590876/article/details/86138355

版权

es 专栏收录该内容

3 篇文章 0 订阅

订阅专栏

记录一下踩坑过程，input elasticsearch 的官方有文档，不用多说，output jdbc 需要安装插件，安装命令./bin/logstash-plugin install logstash-output-jdbc

主要是sql问题，格式为"%{参数名}"这样，为空的字段会直接把%{参数名}" 这个入库，需要在sql中替换成空

input {
elasticsearch {
hosts => "ip:端口"
index => "goods"
query => '{"query":{"bool":{"must":[{"term":{"type":"user_order"}}]}}}'
size => 500
scroll => "5m"
docinfo => true
}
}
output {
jdbc{
driver_jar_path => "/opt/logstash/logstash-jdbc/mysql-connector-java-5.1.46.jar"
driver_class => "com.mysql.jdbc.Driver"
connection_string => "jdbc:mysql://ip:端口/库名?useUnicode=true&characterEncoding=UTF-8&useSSL=false&rewriteBatchedStatements=true&user=用户名&password=密码"
statement => ["INSERT INTO user_order_info_copy(user_order_id,logistics_id,logistics_corporation,create_time) values (?,?,?,from_unixtime(?/1000,'%Y-%m-%d %H:%i:%S'))","%{userOrderId}","%{logisticsId}","%{logisticsCorporation}","%{createTime}"]
}
}
直接运行./bin/logstash -f ./plugin_conf/配置文件名称.conf 没毛病