logstash elasticsearch output插件的bulk提交

logstash的output设置为elasticsearch时,实际上logstash在向es中插入数据时也是bulk提交的,和指提交相关的设置有以下两个参数:

flush_size

Value type is number
Default value is 500
This plugin uses the bulk index API for improved indexing performance. To make efficient bulk API calls, we will buffer a certain number of events before flushing that out to Elasticsearch. This setting controls how many events will be buffered before sending a batch of events. Increasing the flush_size has an effect on Logstash’s heap size. Remember to also increase the heap size using LS_HEAP_SIZE if you are sending big documents or have increased the flush_size to a higher value.

表示多少条flush一起提交到es.

idle_flush_time

Value type is number
Default value is 1
The amount of time since last flush before a flush is forced.

This setting helps ensure slow event rates don’t get stuck in Logstash. For example, if your flush_size is 100, and you have received 10 events, and it has been more than idle_flush_time seconds since the last flush, Logstash will flush those 10 events automatically.

This helps keep both fast and slow log streams moving along in near-real-time.

表示距离上次flush的时间之后多少秒自动flush提交一次

这两个参数一起用的意思就是:
累计缓冲event条数达到flush_size值会flush一次
距离上次flush的时间之后idle_flush_time秒后也会flush一次

满足上面任意两个条件logstash都会flush提交到es

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 3
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 3
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值