吴恩达·Machine Learning || chap15 Anomaly detection简记

15 Anomaly detection

15-1 Problem motivation

Anomaly detection example

Aircraft engine features:

​ $x_1$=heat generated

​ $x_2$=vibration intensity

​ $\cdots$

Dataset: { x ( 1 ) , x ( 2 ) , ⋯   , x ( m ) } \{ x ^ { ( 1 ) } , x ^ { ( 2 ) } , \cdots , x ^ { ( m ) } \} {x(1),x(2),⋯,x(m)}

New engine :$x_{test}$

Density estimation

Dataset: { x ( 1 ) , x ( 2 ) , ⋯   , x ( m ) } \{ x ^ { ( 1 ) } , x ^ { ( 2 ) } , \cdots , x ^ { ( m ) } \} {x(1),x(2),⋯,x(m)}
Is $x_{test}anomalous?$

---

Example

Fraud detection:
$x^{(i)}$= features of user i’s activities
Model p(x) from data
Identify unusual users by checking which have $p(x)<ϵ$

Manufacturing

Monitoring computers in a data center

​ $x^{(i)}$= features of machine i
​ $x_1$ =memory use , $x_2$=number of disk accesses/sec
​ $x_3$ =CPU load , $x_4$ =CPU load/network traffic

15-2 Gaussian distribution

Gaussian (Normal) distribution

​ Say $x\in \mathbb{R}$. If $x$ is a distribution Gaussian with mean $\mu$,variance ${\sigma }^2$

​ $x\sim N(\mu ,{\sigma }^2)$

​ $p(x;\mu ,{\sigma }^2)=\frac{1}{\sqrt{2\pi }\sigma }{e}^{(-\frac{(x-\mu {)}^2}{2{\sigma }^2})}$

​ σ larger，image wider

Parameter estimation

Dataset: { x ( 1 ) , x ( 2 ) , ⋯   , x ( m ) } \{ x ^ { ( 1 ) } , x ^ { ( 2 ) } , \cdots , x ^ { ( m ) } \} {x(1),x(2),⋯,x(m)} $x^{(i)}\in \mathbb{R}$

15-3 Algorithm

Density estimation

Training set: $x^{(1)},\cdots ,x^{(m)}$

Each example is $x\in {\mathbb{R}}^n$

Anomaly detection algorithm

1. Choose features $x_i$ that you think might be indicative of
   anomalous examples.

2. Fit parameters

   ${\mu }_1,\cdots ,{\mu }_n,{\sigma }_1^2,\cdots ,{\sigma }_n^2$

   $${\mu }_j=\frac{1}{m}\sum _{i=1}^m{x}_j^{(i)}$$

   $${\sigma }_j^2=\frac{1}{m}\sum _{i=1}^m(x_i^{(i)}-{\mu }_j{)}^2$$

3. Given new example x, compute p(x):

$$p(x)=\prod _{j=1}^{n}p(x_j;u_j,{\sigma }_j^2)=\prod _{j=1}^n\frac{1}{\sqrt{2\pi }{\sigma }_j}exp(-\frac{(x_j-u_j{)}^2}{2{\sigma }_j})$$

Anomaly if $p(x)<ϵ$

Anomaly detection example

15-4 Developing and evaluating an anomaly detection system

The importance of real-number evaluation

When developing a learning algorithm(choosing features, etc. ), making decisions is much easier if we have a way of evaluating our learning algorithm

Assume we have some labeled data of anomalous and non anomalous examples. (y=0 if normal, y=1 if anomalous

Training set: $x^{(1)},x^{(2)},\cdots ,x^{(m)}$(assume normal examples/not anomalous)

Cross validation set :$$(x_{cv}^{(1)},y_{cv}^{(1)}),\cdots ,(x_{cv}^{(m_{cv})},y_{cv}^{(m_{cv})})$$

Test set :$$(x_{test}^{(1)},y_{test}^{(1)}),\cdots ,(x_{test}^{(m_{test})},y_{test}^{(m_{test})})$$

Aircraft engines motivation example

10000 good (normal) engines

20 flawed engines (anomalous)

Training set : 6000 good engines

CV: 2000 good engines(y=0),10 anomalous (y=1)

Test: 2000 good engines(y=0),10 anomalous(y=1)

or Alternative

Algorithm evaluation

Fit model $p(x)$ on training set { x ( 1 ) , ⋯   , x ( m ) } \{x^{(1)},\cdots,x^{(m)}\} {x(1),⋯,x(m)}

On a cross validation/test example predict
$$y=\{\begin{array}{l}1ifp(x)<ϵ(anomaly)\\ 0ifp(x)\ge ϵ(normal)\end{array}$$
Possible evaluation metrics:

• True positive, false positive, false negative, true negative
• Precision/Recall
• $F_1$ -score

Can also use cross validation set to choose parameter $ϵ$

15-5 Anomaly detection vs. supervised learning

15-6 Choosing what features to use

Non-gaussian features

make the data look a bit more Gaussian

Error analysis for anomaly detection

Want $p(x)$ large for normal examples $x$
$p(x)$ small for anomalous examples a
Most common problem:
$p(x)$ is comparable (say, both large) for normal
and anomalous examples

**Monitoring computers in a data center **

Choose features that might take on unusually large or small values in the event of an anomaly
$x_1$= memory use of computer
$x_1$=number of disk accesses/sec
$x_1$=CPU load
$x_1$=network traffic

15-7 Multivariate Gaussian distribution

Motivating example: Monitoring machines in a data center

Multivariate Gaussian (Normal) distribution

$x\in {\mathbb{R}}^n$.Don’t model $p(x_1),p(x_2),\cdots ,etc.$separately.
Model $p(x)$ all in one go.
Parameters:$\mu \in {\mathbb{R}}^n,\Sigma \in {\mathbb{R}}^{n\times n}(covariancematrix)$

Multivariate Gaussian (Normal) examples

15-8 Anomaly detection using the multivariate Gaussian distribution

Multivariate Gaussian (Normal)distribution

Parameters $\mu ,\Sigma$

$$p(x;\mu ,\Sigma )=\frac{1}{(2\pi {)}^{\frac{n}{2}}|{\Sigma }^{\frac{1}{2}}|}{e}^{(-\frac{1}{2}(x-\mu {)}^T{\Sigma }^{-1}(x-\mu ))}$$

Parameter fitting:

Given training set { x ( 1 ) , x ( 2 ) , ⋯   , x ( m ) } \{ x ^ { ( 1 ) } , x ^ { ( 2 ) } , \cdots , x ^ { ( m ) } \} {x(1),x(2),⋯,x(m)}

$$u=\frac{1}{m}\sum _{i=1}^m{x}^{(i)}$$

$$\Sigma =\frac{1}{m}\sum _{i=1}^m(x^{(i)}-\mu )(x^{(i)}-\mu {)}^T$$

Anomaly detection with the multivariate Gaussian

1. Fit model $p(x)$ by setting

   $$u=\frac{1}{m}\sum _{i=1}^m{x}^{(i)}$$

   $$\Sigma =\frac{1}{m}\sum _{i=1}^m(x^{(i)}-\mu )(x^{(i)}-\mu {)}^T$$

2. Given a new example x,compute

   $$p(x;\mu ,\Sigma )=\frac{1}{(2\pi {)}^{\frac{n}{2}}|{\Sigma }^{\frac{1}{2}}|}{e}^{(-\frac{1}{2}(x-\mu {)}^T{\Sigma }^{-1}(x-\mu ))}$$
   Flag an anomaly if $p(x)<ϵ$

Relationship to original model

Original model:$$p(x)=p(x_1;{\mu }_1,{\sigma }_1^2)\times p(x_2;{\mu }_2,{\sigma }_2^2)\times \cdots \times p(x_n;{\mu }_n,{\sigma }_n^2)$$

Corresponds to multivariate Gaussian

$$p(x;\mu ,\Sigma )=\frac{1}{(2\pi {)}^{\frac{n}{2}}|\Sigma {|}^{\frac{1}{2}}}exp(-\frac{1}{2}(x-\mu {)}^T{\Sigma }^{-1}(x-\mu ))$$

where