使用nifi巡检系统，出现异常及时告警

1、模版获取

https://github.com/qkfm-97/nifi-example/blob/main/ops/check_system.xml

2、任务说明

该流程主要是对运维平时巡检的系统或者重要服务进行自动巡检，如果出现接口或者系统不能访问时就产生告警。

将要巡检的系统url或者重要接口写入system_check_list表中，示例如下

如果访问出现异常就往短信表插入一条数据，短信定时器（另外的程序）会根据内容发送短信

短信表和巡检列表的表结构如下

-- Create table 巡检列表
create table SYSTEM_CHECK_LIST
(
  system_id       VARCHAR2(100),
  system_name     VARCHAR2(1000),
  check_time      DATE,
  check_parameter VARCHAR2(4000),
  check_method    VARCHAR2(200),
  service_url     VARCHAR2(1000),
  service_name    VARCHAR2(1000),
  result_status   VARCHAR2(10),
  result_code     VARCHAR2(1000),
  result_msg      VARCHAR2(1000),
  remark          VARCHAR2(4000),
  unid            VARCHAR2(32),
  check_protocol  VARCHAR2(100)
);
-- Add comments to the columns 
comment on column SYSTEM_CHECK_LIST.system_id
  is '系统唯一标识';
comment on column SYSTEM_CHECK_LIST.system_name
  is '系统名称';
comment on column SYSTEM_CHECK_LIST.check_time
  is '检测时间';
comment on column SYSTEM_CHECK_LIST.check_parameter
  is '检测参数';
comment on column SYSTEM_CHECK_LIST.check_method
  is '检测方法POST、GET';
comment on column SYSTEM_CHECK_LIST.service_url
  is '服务url地址';
comment on column SYSTEM_CHECK_LIST.service_name
  is '服务名称';
comment on column SYSTEM_CHECK_LIST.result_status
  is '结果状态';
comment on column SYSTEM_CHECK_LIST.result_code
  is '结果代码';
comment on column SYSTEM_CHECK_LIST.result_msg
  is '结果信息';
comment on column SYSTEM_CHECK_LIST.remark
  is '备注';
comment on column SYSTEM_CHECK_LIST.unid
  is '主键';
comment on column SYSTEM_CHECK_LIST.check_protocol
  is '检测协议http、socket';


--短信表
-- Create table
create table MESSAGE_LIST
(
  unid           VARCHAR2(128) not null,
  msg_type       VARCHAR2(2),
  msg_state      VARCHAR2(2),
  msg_send_time  DATE,
  msg_phone      VARCHAR2(12),
  msg_createtime DATE,
  msg_content    VARCHAR2(4000),
  msg_desc       VARCHAR2(4000)
);
-- Add comments to the columns 
comment on column MESSAGE_LIST.unid
  is '主键';
comment on column MESSAGE_LIST.msg_type
  is '短信类型1-系统巡检 2-大数据组件检测';
comment on column MESSAGE_LIST.msg_state
  is '发送状态0-未发送，1-已发送，2-发送失败';
comment on column MESSAGE_LIST.msg_send_time
  is '发送时间';
comment on column MESSAGE_LIST.msg_phone
  is '手机号码';
comment on column MESSAGE_LIST.msg_createtime
  is '创建时间';
comment on column MESSAGE_LIST.msg_content
  is '短信内容';
comment on column MESSAGE_LIST.msg_desc
  is '短信发送失败等备注';

3、流程截图

4、重点说明

这边https的请求做了特殊处理，由于NiFi调用HTTPS接口时需要证书，统一巡检操作起来较为不便，因此我们采用了特殊处理方式。通过一个Python脚本创建了一个HTTP接口，该接口负责访问HTTPS接口并跳过证书验证。Python脚本如下：

from flask import Flask, request, jsonify
import requests

app = Flask(__name__)

@app.route('/invoke', methods=['POST'])
def invoke():
    # 从请求中获取JSON数据
    data = request.json
    if not data or 'url' not in data or 'method' not in data or 'param' not in data:
        return jsonify({"error": "Invalid input"}), 400

    url = data['url']
    method = data['method'].upper()
    params = data['param']

    try:
        if method == 'GET':
            response = requests.get(url, params=params, verify=False)
        elif method == 'POST':
            response = requests.post(url, json=params, verify=False)
        else:
            return jsonify({"error": "Unsupported HTTP method"}), 400

        return jsonify({
            "status_code": response.status_code,
            "headers": dict(response.headers),
            "content": response.content.decode('utf-8')
        })
    except Exception as e:
        return jsonify({"error": str(e)}), 500

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000)