NoMethodError：undefined method to_ISO8601 for nil:NilClass-CSDN博客

打开微信扫一扫，关注微信公众号【数据与算法联盟】
转载请注明出处： http://blog.csdn.net/gamer_gyt
博主微博： http://weibo.com/234654758
Github： https://github.com/thinkgamer

环境介绍

docker 1.12.1
elasticsearch 2.4.1
logstash 2.4.0
kibana 4.6.1

docker容器中运行elk服务，并用logstash 解析日志，出现错误

这里写图片描述

查看日志发现是字符格式解析失败，如下信息所示

Received an event that has a different character encoding than you configured. {:text=>"2015/11/3 8:33,\\\"\\xB7\\xA2\\xCF\\xD6\\xD3?\\xA7\\xC8?\\xFDUSP,?\\xD3\\xC3\\xD5?\\xC5bj,\\xB4\\xD3su\\xB5\\xC7??\\xB1\\xEA\\xD6\\xF7\\xBB\\xFA132.77.138.21\\xA3\\xAC\\xD0\\xE8?\\xB9\\xD8?\\xA1\\xA3\\\"\\r", :expected_charset=>"UTF-8", :level=>:warn}
Error parsing csv {:field=>"message", :source=>"2015/11/3 8:33,\\\"\\xB7\\xA2\\xCF\\xD6\\xD3?\\xA7\\xC8?\\xFDUSP,?\\xD3\\xC3\\xD5?\\xC5bj,\\xB4\\xD3su\\xB5\\xC7??\\xB1\\xEA\\xD6\\xF7\\xBB\\xFA132.77.138.21\\xA3\\xAC\\xD0\\xE8?\\xB9\\xD8?\\xA1\\xA3\\\"\\r", :exception=>#<CSV::MalformedCSVError: Illegal quoting in line 1.>, :level=>:warn}

                  to_s at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-event-2.2.4-java/lib/logstash/event.rb:105
                encode at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-codec-line-2.1.2/lib/logstash/codecs/line.rb:56
               receive at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-stdout-2.0.6/lib/logstash/outputs/stdout.rb:55
         multi_receive at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.4-java/lib/logstash/outputs/base.rb:83
                  each at org/jruby/RubyArray.java:1613
         multi_receive at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.4-java/lib/logstash/outputs/base.rb:83
  worker_multi_receive at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.4-java/lib/logstash/output_delegator.rb:130
         multi_receive at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.4-java/lib/logstash/output_delegator.rb:114
          output_batch at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.4-java/lib/logstash/pipeline.rb:293
                  each at org/jruby/RubyHash.java:1342
          output_batch at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.4-java/lib/logstash/pipeline.rb:293
           worker_loop at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.4-java/lib/logstash/pipeline.rb:224
         start_workers at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.2.4-java/lib/logstash/pipeline.rb:193

解决办法

想了好久也没找到解决办法，网上也是各种百度谷歌，最后突发奇想是不是docker安装时是最小化安装，然后好多package loss了，结果还真是

总结一下原因 ：docker是最小化安装，logstash对文本数据进行解析时，如果没有指定codec，默认采用得是utf-8,而docker由于是最小化安装，缺少对应得package，所以解析时会报错

查看docker容器当前的系统运行语言环境

root@0f2baa6ee463:/var/log/logstash# locale
LANG=
LANGUAGE=
LC_CTYPE="POSIX"
LC_NUMERIC="POSIX"
LC_TIME="POSIX"
LC_COLLATE="POSIX"
LC_MONETARY="POSIX"
LC_MESSAGES="POSIX"
LC_PAPER="POSIX"
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=

查看所支持的语言

root@0f2baa6ee463:/home# locale -a
C
C.UTF-8
POSIX

开始解决问题

/* 生成 en_US.UTF-8 locale文件 CentOS没有locale-gen命令*/
$localedef -v -c -i en_US -f UTF-8 en_US.UTF-8

$export LC_ALL="en_US.UTF-8"
bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8): No such file or directory

最后在查看新系统的语言环境

root@0f2baa6ee463:/home# locale
LANG=
LANGUAGE=
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=en_US.UTF-8
root@0f2baa6ee463:/home# locale -a
C
C.UTF-8
en_US.utf8
POSIX