三种类型—异常检测

最新推荐文章于 2024-05-22 08:35:51 发布

only-guo

最新推荐文章于 2024-05-22 08:35:51 发布

阅读量288

点赞数

文章标签：深度学习神经网络机器学习

本文链接：https://blog.csdn.net/weixin_45153493/article/details/120370722

版权

本文概述了异常检测的三种主要方法：基于统计学的一维数据检测，如sigma原则和箱线图；无监督分类技术，包括孤立森林、LOF和OneClassSVM；以及基于重构误差的PCA和自编码器进行异常检测。

摘要由CSDN通过智能技术生成

异常检测

异常检测方法概述

异常检测方法概述

基于统计学（一维数据）
1.1sigma原则
1.2箱线图原则
无监督分类
2.1 孤立森林：outlier detection: 训练集中有异常样本
2.2 LOF ：局部异常因子算法，outlier detection
2.3 OneClassSVM ：novelty detection，训练集中没有异常样本
基于重构误差
3.1 PCA “Therefore, outlier scores can be obtained as the sum of the projected distance of a sample on all eigenvectors.”
3.2 自编码器 “Similar to PCA, AE could be used to detect outlying objects in the data by calculating the reconstruction errors”


from pyod.models.knn import KNN
from pyod.models.lof import LOF
from pyod.models.iforest import IForest
from pyod.models.pca import PCA
from pyod.utils.utility import standardizer
from pyod.utils.data import generate_data
import pandas as pd
from pyod.utils.data import evaluate_print
from pyod.utils.example import visualize
import matplotlib.pyplot as plt
import numpy as np
from pyod.models.combination import aom, moa, average, maximization, median
from pyod.utils.utility import score_to_label
import math
from pyod.models.auto_encoder import AutoEncoder


def single_detector(X_train,detector_list,detector_list_names):
    X_train_norm, X_test_norm = standardizer(X_train, X_test)
    y_train_pred_all = pd.DataFrame()

    for name, clf in zip(detector_list_names, detector_list):
        clf.fit(X_train_norm)
        y_train_pred = clf.labels_
        y_train_pred_all[name] = y_train_pred

    num = len(detector_list)
    n_col = 4
    n_row = math.ceil(num / n_col)

    fig = plt.figure(figsize=(n_row * 5 + 3, n_col * 5))
    colors