在csv文件里的格式
“field1”,“field2”,“field3”,"line1
line2
line3
",“field5”
可以看到上面field4里面有被换行成几行来显示了。
这是我们用logstash去读取:
input {
file {
path => [ "file.csv" ]
start_position => "beginning"
sincedb_path => "/dev/null"
}
}
filter {
csv {
separator => ","
skip_header => "true"
columns => ["field1","field2","field3","field14,"field5"]
}
}
output {
stdout {}
elasticsearch{
hosts => ["https://ip:9200"]
ssl => true
cacert => "/etc/logstash/certs/rootCA.crt"
api_key => "api_key "
index => "alert-%{+YYYY-MM-dd}"
}
}
这时候会遇到下面的错误信息
exception=>#<CSV::MalformedCSVError: Unclosed quoted field on line 1
解决办法:
在file代码块里面加入下面代码:
codec => multiline {
pattern => "^\"XXX"
negate => "true"
what => "previous"
}
^是从行的开头开始,
"XXX是匹配每一行开头的字符串
这样就可以解析了,我实测过,可以解析成功。