logstash集成mongo

解压
tar -zxvf logstash-6.2.4.tar.gz

进入bin目录下
bin/logstash -e ‘input { stdin { } } output { stdout {} }’

输入zjl
回车了看到以下就可以了
{
“@version” => “1”,
“host” => “arreat01”,
“message” => “zjl”,
“@timestamp” => 2018-12-26T09:48:36.020Z
}

vim config/logstash.yml

pipeline.workers: 8
pipeline.output.workers: 8
pipeline.batch.size: 10000
pipeline.batch.delay: 10

vim config/jvm.options
-Xms2g
-Xmx2g

离线安装插件
进入logstash 下bin目录 查看已安装的插件：

./logstash-plugin list

没有logstash-input-mongodb插件那么：

./logstash-plugin install logstash-input-mongodb

logstash的bin目录下执行
打成离线包的命令
./logstash-plugin prepare-offline-pack --overwrite --output logstash-input-mongodb.zip logstash-input-mongodb
上传到服务器执行命令
./logstash-plugin install file:///home/upsmart/soft/logstash-input-mongodb.zip

ok啦！！！

upsmart@arreat01:~/soft/logstash/logstash-6.5.4/bin$ ./logstash -f logstash.conf
warning: ignoring JAVA_OPTS=-Xmx8192m; pass JVM parameters via LS_JAVA_OPTS
Sending Logstash logs to /home/upsmart/soft/logstash/logstash-6.5.4/logs which is now configured via log4j2.properties
[2018-12-27T14:31:37,141][WARN ][logstash.config.source.multilocal] Ignoring the ‘pipelines.yml’ file because modules or command line options are specified
[2018-12-27T14:31:37,166][FATAL][logstash.runner ] Logstash could not be started because there is already another instance using the configured data directory. If you wish to run multiple instances, you must change the “path.data” setting.
[2018-12-27T14:31:37,178][ERROR][org.logstash.Logstash ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

报错

原因：之前运行的instance有缓冲，保存在path.data里面有.lock文件，删除掉就可以。
解决办法：

在 logstash.yml 文件中找到 Data path 的路径(默认在安装目录的data目录下）

ls -a查看是否存在 .lock 文件，如果存在把它删除

logstash_sqlite.db

placeholder_db_dir => '/opt/logstash-mongodb/'
placeholder_db_name =>'logstash_sqlite.db'

Error registering plugin {:pipeline_id=>“main”, :plugin=>"<LogStash::Inputs::MongoDB uri=>“mongodb://192.168.88.85:27017/sys_request”, placeholder_db_dir=>"/opt/logstash-mongodb/", placeholder_db_name=>“sys_request.db”, collection=>“sys_request_info”, id=>“c6741908217fda859a83049daaff5d8ac50f7bf5a3618c9bbb64cdf2c86db3f8”, enable_metric=>true, codec=><LogStash::Codecs::Plain id=>“plain_ca927bcd-1866-4fa6-a853-9bf656a32ef5”, enable_metric=>true, charset=>“UTF-8”>, batch_size=>30, since_table=>“logstash_since”, since_column=>"_id", since_type=>“id”, parse_method=>“flatten”, isodate=>false, retry_delay=>3, generateId=>false, unpack_mongo_id=>false, message=>“Default message…”, interval=>1>", :error=>“Java::JavaSql::SQLException: [SQLITE_ERROR] SQL error or missing database (no such table: since_table)”, :thread=>"#<Thread:0x58640393 run>"}

ObjectId(“5822cb4250c1fd585019e4a1”)