logstash输出到elasticsearch多索引

目标：将json格式的两类日志输出到elasticsearch两类索引

1. 安装logstash。

2. 编写logstash处理配置文件，创建一个test.conf文件，内容如下：

input {
     file {
        path => "/home/vagrant/logstash/logstash-2.2.2/dbpool-logs/dev/common-sql-*.log"
        start_position => "beginning"
        type => "sql"
        codec => json {
            charset => "UTF-8"
        }        
     }
     file {
        path => "/home/vagrant/logstash/logstash-2.2.2/dbpool-logs/dev/common-transaction-*.log"
        start_position => "beginning"
        type => "transaction"
        codec => json {
            charset => "UTF-8"
        }        
     }
}
output { 
    if "_grokparsefailure" in [tags] {
    }else{
        if [type] == "sql"{
                elasticsearch { 
                         hosts => ["http://192.168.33.10:9200"]
                         index => "common-sql-%{+YYYY.MM.dd}"
                }
       }
       if [type] == "transaction"{
                elasticsearch { 
                         hosts => ["http://192.168.33.10:9200"]
                         index => "common-transaction-%{+YYYY.MM.dd}"
                }
       }
    }   
}

或者

input {
     file {
        path => "/home/vagrant/logstash/logstash-2.2.2/dbpool-logs/dev/common-sql-*.log"
        start_position => "beginning"
        type => "sql"
        codec => json {
            charset => "UTF-8"
        }        
     }
     file {
        path => "/home/vagrant/logstash/logstash-2.2.2/dbpool-logs/dev/common-transaction-*.log"
        start_position => "beginning"
        type => "transaction"
        codec => json {
            charset => "UTF-8"
        }        
     }
}
output { 
    if "_grokparsefailure" in [tags] {
    }else{
        if [type] == "sql"{
                elasticsearch { 
                         hosts => ["http://192.168.33.10:9200"]
                         index => "common-%{type}-%{+YYYY.MM.dd}"
                }
       }
    }   
}

• 指定输入日志的路径，按通配符匹配。分为两类：sql和transaction。

• 根据type分别输出到elasticsearch不同的索引。

3. 安装elasticsearch。

4. 启动elasticsearch，./bin/elasticsearch -d ，默认端口为9200。

5. 启动logstash开始处理，./bin/logstash -f conf/test.conf。

6. 完成。

========广告时间========

鄙人的新书《Tomcat内核设计剖析》已经在京东销售了，有需要的朋友可以到 https://item.jd.com/12185360.html 进行预定。感谢各位朋友。

为什么写《Tomcat内核设计剖析》

=========================