1.安装logstash
2.修改logstash 文件夹下Gemfile文件
将source改为:https://gems.ruby-china.org
3.进入bin目录
执行logstash-plugin install logstash-input-jdbc
出现如下信息
Validating logstash-input-jdbcInstalling logstash-input-jdbc
Installation successful
表示安装成功
4.在bin目录中新建mysql目录,放入mysql-connector-java-5.1.30.jar包
5.在bin目录中新建jdbc.sql文件
SELECT * from person
改成如下配置:
input {
jdbc {
# mysql jdbc connection string to our backup databse
jdbc_connection_string => "jdbc:mysql://127.0.0.1:3306/test"
# the user we wish to excute our statement as
jdbc_user => "root"
jdbc_password => "root"
# the path to our downloaded jdbc driver 下面根据自己的具体路径添加
jdbc_driver_library => "mysql/mysql-connector-java-5.1.30.jar"
# the name of the driver class for mysql
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_paging_enabled => "true"
jdbc_page_size => "50000"
statement_filepath => "jdbc.sql"
schedule => "* * * * *"
type => "jdbc"
}
}
filter {
json {
source => "message"
remove_field => ["message"]
}
}
output {
elasticsearch {
hosts => ["127.0.0.1:9200"]
index => "mysql01"
document_id => "%{id}"
}
stdout {
codec => json_lines
}
}
logstash.bat -f logstash.conf
出现如下界面:成功的界面。
下面说下踩过的坑
1,有的是说6.0以上不用安装 logstash-input-jdbc 插件,我试了不用安装。
1,新建的.config,.sql文件必须是utf_8 without bom 格式,否则会出现错误。