Anomaly detection - Anomaly detection vs. supervised learning

最新推荐文章于 2023-12-13 15:59:08 发布

王彩旗 edwardwangcq.com

最新推荐文章于 2023-12-13 15:59:08 发布

阅读量114

点赞数

分类专栏：人工智能 # 机器学习

本文链接：https://blog.csdn.net/edward_wang1/article/details/112259854

版权

人工智能同时被 2 个专栏收录

142 篇文章 0 订阅

订阅专栏

机器学习

109 篇文章 0 订阅

订阅专栏

本文探讨了在有标记数据的情况下，何时选择异常检测算法和何时选择监督学习方法。当正面样例非常少或者未来异常可能与历史样本显著不同时，推荐使用异常检测。而当拥有大量正负样本且未来异常可能与训练集相似时，应采用监督学习，如逻辑回归或神经网络。文章列举了异常检测与监督学习在欺诈检测和制造业等场景中的应用。

摘要由CSDN通过智能技术生成

摘要: 本文是吴恩达 (Andrew Ng)老师《机器学习》课程，第十六章《异常检测》中第127课时《异常检测vs监督学习》的视频原文字幕。为本人在视频学习过程中记录下来并加以修正，使其更加简洁，方便阅读，以便日后查阅使用。现分享给大家。如有错误，欢迎大家批评指正，在此表示诚挚地感谢！同时希望对大家的学习能有所帮助.

————————————————

If we have labeled data we know which examples are anomalous and which examples are non-anomalous, when should we use supervised learning (logistic regression, neural network...) to try to learn directly from our labeled data to predict whether y=1 or y=0 ? And when should we use anomaly detection algorithm? Followings are the guidelines.

Choose anomaly detection when:
- If you have very small number of positive examples (, 0-20 maybe up to 50 is pretty typical).
  - It can be difficult for an algorithm to learn from the very small set of positive examples what the anomalies look like.
  - We'll save the positive examples just for cross validation and test set
  - We'll use the large number of negative/non-anomalous/normal examples to fit the the Gaussian parameters of the model $p(x)=p(x_{1}; \mu _{1},\sigma _{1}^{2}),...,p(x_{n}; \mu _{n},\sigma _{n}^{2})$
- There are many different types of anomalies. Future anomalies may look nothing like the ones you've seen so far
  - It would be more promising to just model the negative examples with kind a Gaussian model rather than trying to model the positive examples because tomorrow's anomaly may be nothing like the ones you've seen so far
Choose supervised learning when:
- If you have reasonably large number of both positive and negative examples.
  - There are enough positive examples for an algorithm to get a sense of what the positive examples look like.
- The future positive examples are likely to be similar to ones in the training set

Followings are typical applications of anomaly detection & supervised learning:

If you have a very major online retainler, and if you actually have had a lot of people try to commit fraud on your website, so you have a lot of examples with . Sometimes fraud detection could actually shift over to the supervised learning column.
For some manufacturing process, if you're manufacturing very large volumes and you've seen a lot of bad examples, maybe manufacturing could shift to the supervised learning column as well.

<end>