最新优化网站,想了想使用ec做列表查询,估计速度会提个一倍,但是使用了什么同步数据呢?
经过网上查资料,一顿操作,最总选定了logstash这个软件
下面是配置
input {
stdin {
}
jdbc {
jdbc_connection_string => "jdbc:mysql://192.168.1.135:3306/chipsmall_test"
jdbc_user => "docker"
jdbc_password => "docker"
jdbc_driver_library => "/www/mysql-connector-java-8.0.28.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_paging_enabled => "true"
jdbc_page_size => "1"
use_column_value => "true"
tracking_column => "update_at"
#"numeric", "timestamp"
tracking_column_type => "numeric"
statement => "select * from snsii_user where update_at > :sql_last_value"
schedule => "* * * * *"
last_run_metadata_path => "/www/update_at-es"
}
}
filter {
mutate {
# convert => [ "publish_time", "string" ]
}
json {
source => "message"
remove_field => ["message"]
}
}
output {
elasticsearch {
hosts => ["http://192.168.1.135:9200"]
user => "elastic"
password => "str123456"
index => "user"
document_id => "%{id}"
}
stdout {
codec => json_lines
}
}
刚开始用的字段Update_at 发现sql_last_value值一直不跟新,why,why,why?
卡了我一天,后来偶然看到日志,查询出来的数据字段都是小写,改成小写试试?
欧耶!
竟然可以了!
吐槽!!!!!!!!!