ELK读取网络设备日志的方式

最新推荐文章于 2024-07-05 19:17:36 发布

yuezhilangniao

最新推荐文章于 2024-07-05 19:17:36 发布

阅读量2.5k

点赞数 1

分类专栏： ELK+APM监控文章标签： ELK

原文链接：https://www.opscaff.com/2018/05/08/elk-%E4%BA%A4%E6%8D%A2%E6%9C%BA%E6%97%A5%E5%BF%97%E5%88%86%E6%9E%90/

版权

ELK+APM监控专栏收录该内容

17 篇文章 3 订阅

订阅专栏

原文：https://www.opscaff.com/2018/05/08/elk-%E4%BA%A4%E6%8D%A2%E6%9C%BA%E6%97%A5%E5%BF%97%E5%88%86%E6%9E%90/

步骤：1 搭建ELK filebeat收集日志，logstash做正则解析，标准数据存入es，符合规则的调用python脚本告警。

2 交换机配置发送日志给目标ip：端口，ELK那边做好监听收集即可。

例子，

思科交换机：

logging host 10.100.18.18 transport udp port 5002

H3C：

info-center enable

info-center source default channel 2 trap state off

// 必要，不然日志会出现不符合级别的 alert 日志

info-center loghost 10.100.18.18 port 5003

huawei ：
info-center enable
info-center loghost 10.100.18.18
info-center timestamp log short-date
info-center timestamp trap short-date

# 可以批量操作交换机 python脚本

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from netmiko import ConnectHandler
from openpyxl import load_workbook
import threading

def conf_syslog(ip):

	cisco_881 = {
	‘device_type’: ‘cisco_ios’,
	‘ip’: ip,
	‘username’: ‘admin’,
	‘password’: ‘admin@123’,
	}

	net_connect = ConnectHandler(**cisco_881)

	commands = [‘logging on’,
	‘logging host 10.100.18.18 transport udp port 5002’,
	‘end’,
	‘write memory’]

	output = net_connect.send_config_set(commands)

	return output

def get_host():

	wb = load_workbook(filename=‘hosts.xlsx’)
	sheetnames = wb.sheetnames
	ws = wb[sheetnames[0]]

	for i in range(2, ws.max_row +1):
		ip = ws.cell(row=i, column=1).value
		res = threading.Thread(target=conf_syslog, args=(ip,))
		res.start()

	return res

print(get_host())

# grok原文根据不同厂商已经写好 NB

Logstash 的配置：
不同厂商的日志 gork我都写好了，复制过去就能用。

input{
    tcp {port =>
				5002 type =>
				"Cisco"}
    udp {port =>
				514 type =>
				"HUAWEI"}
    udp {port =>
				5002 type =>
				"Cisco"}
    udp {port =>
				5003 type =>
				"H3C"}
}
filter {
    if [type] ==
				"Cisco"{
    grok{
    match => { "message"
				=>
				"<%{BASE10NUM:syslog_pri}>%{NUMBER:log_sequence}: .%{SYSLOGTIMESTAMP:timestamp}: %%{DATA:facility}-%{POSINT:severity}-%{CISCO_REASON:mnemonic}: %{GREEDYDATA:message}" }
    match => { "message"
				=>
				"<%{BASE10NUM:syslog_pri}>%{NUMBER:log_sequence}: %{SYSLOGTIMESTAMP:timestamp}: %%{DATA:facility}-%{POSINT:severity}-%{CISCO_REASON:mnemonic}: %{GREEDYDATA:message}" }
    add_field => {"severity_code"
				=>
				"%{severity}"}
    overwrite => ["message"]
    }    
}
    else
				if [type] ==
				"H3C"{
    grok {
    match => { "message"
				=>
				"<%{BASE10NUM:syslog_pri}>%{SYSLOGTIMESTAMP:timestamp} %{YEAR:year} %{DATA:hostname} %%%{DATA:vvmodule}/%{POSINT:severity}/%{DATA:digest}: %{GREEDYDATA:message}" }
    remove_field => [ "year" ]
    add_field => {"severity_code"
				=>
				"%{severity}"}
    overwrite => ["message"]
    }
}

				else
				if [type] ==
				"HUAWEI"{
    grok {
       match => { "message"
				=>
				"<%{BASE10NUM:syslog_pri}>%{SYSLOGTIMESTAMP:timestamp} %{DATA:hostname} %%%{DATA:ddModuleName}/%{POSINT:severity}/%{DATA:Brief}:%{GREEDYDATA:message}"}
       match => { "message"
				=>
				"<%{BASE10NUM:syslog_pri}>%{SYSLOGTIMESTAMP:timestamp} %{DATA:hostname} %{DATA:ddModuleName}/%{POSINT:severity}/%{DATA:Brief}:%{GREEDYDATA:message}"}
       remove_field => [ "timestamp" ]
    add_field => {"severity_code"
				=>
				"%{severity}"}
    overwrite => ["message"]
    }
}
 

mutate {
        gsub => [
        "severity", "0", "Emergency",
        "severity", "1", "Alert",
        "severity", "2", "Critical",
        "severity", "3", "Error",
        "severity", "4", "Warning",
        "severity", "5", "Notice",
        "severity", "6", "Informational",
        "severity", "7", "Debug"
			
        ]
    }
}
output{
    elasticsearch {
        index =>
				"syslog-%{+YYYY.MM.dd}"
			
        hosts => ["your_ipaddress:9200"]
    }
}