logstash 7.X系列之：logstash采集csv文件并存储至es

超超超超子

于 2021-09-29 16:10:16 发布

阅读量863

点赞数

分类专栏：大数据系列文章标签：大数据 elk logstash

本文链接：https://blog.csdn.net/g5703129/article/details/120550696

版权

该配置文件定义了一个Logstash pipeline，用于从CSV文件中读取数据。文件路径为'D:/date/rs-hf.csv'，使用平原编码并以UTF-8字符集解析。配置中指定了CSV字段及其数据类型，并将所有字段转换为字符串类型。数据经过处理后被索引到Elasticsearch中，索引名称根据日期动态生成。同时，数据也会输出到标准输出以便于调试。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

input {
  file {
    type => "csv_log_1"
    path => ["D:/date/rs-hf.csv"]  
    start_position => "beginning"
    codec => plain{
      charset=>"UTF-8"
    }
  }
}
filter {
	#字段以及数据类型
  csv {
    separator => ","
    columns =>   ["city","house_address","house_region","house_area","house_id","total_price","unit_price","house_layout","house_floor","house_rental_area","house_structure","house_inner_area","house_building_type","house_orientation","house_building_structure","house_decoration","house_elevator_sytle","house_elevator","house_listing_time","house_transaction_type","house_last_time","house_useage","house_years","house_property","house_mortgage_info","house_book","house_

最低0.47元/天解锁文章