Log Monitor : 簡便的分散式 log 管理工具

JavaEye 的 CVU 版友最近發表了一篇 Log Monitor - 把分散在各台server上的log集中起来monitor 的程式，主要功能是可以將分散在各個機器的 Rails log 給集合起來。主要是使用 net-ssh 連到各個 server，然後將指定好的 log 給 tail -f 起來。經過試用後發現程式簡單方便修改，但是輸出目前為止又很堪用，實在感謝他。License 是 CC 2.5 姓名，相同方式。


config 寫法
config出現在第六行裡面的 modules.conf ，Modules.conf 寫法如下

log 所在的 server ip or domain name：
__username：ssh 的使用者名稱
__password：ssh user 的 password
__logs:
____/log/所在/目錄/log檔名:
______name: [在 log monitor 顯示的名稱]
______greps:
________error.log:
__________- ERROR
________info.log:
__________- INFO

程式寫法

### license: http://creativecommons.org/licenses/by-sa/2.5/deed.zh
### by: caoweiyuan#gmail.com
require 'net/ssh'

require 'yaml'
modules = YAML.load(File.open('modules.conf'))
### sample config
# engine-1:
#   username: admin
#   password: ******
#   logs:
#     /opt/apps/production/tomcat/logs/app.log:
#       name: [module-1]
#       greps:
#         error.log:
#           - ERROR
#         info.log:
#           - INFO

def do_grep(name, line, greps)
log = "[#{name}] - #{line}"

# default output to screen
puts log

# grep patterns to sepecific files
for log_file, patterns in greps
  pattern = (patterns.class == Array) ? patterns.join("|") : patterns
  File.open('log/' + log_file, 'a') {|f| f.puts log} if line =~ /#{pattern}/
# TODO: auto mail for errors
  # TODO: horizontal log for different products, and delete after each process over.
end
end

def do_tail( session, log_file, log_conf )
session.open_channel do |channel|
  channel.on_data do |ch, data|
    data.each_line do |line|
      do_grep log_conf['name'], line, log_conf['greps']
end
end
  channel.exec "tail -1f #{log_file}"
end
end

for key, value in modules
t = Thread.new(key, value) do |host, host_conf|
  Net::SSH.start( host, host_conf['username'], host_conf['password'] ) do |session|
puts "#{host} logged in."
for log_file, log_conf in host_conf['logs']
puts "start watching #{log_file}"
# open channels for multiple log files
do_tail session, log_file, log_conf
end # log
session.loop # important: loop to keep reading channel outputs
end # session
end # thread
end # host

# TODO: implement a formal deamon
loop do
# deamonize
sleep 1
end