机器学习系列之coursera week 4 Neural Networks: Representation

目录

1. Motivations

1.1 Non-linear hypothesis (Neural Networks)

1.2 Neurons and the brain

2. Neural Networks

2.1 Model Representation I

2.2 Model Representation II

3. Application

3.1 Example and Intuition I

3.2 Example and Intuition II

3.3 Multiclass Classification

---

1. Motivations

1.1 Non-linear hypothesis (Neural Networks)

Non-linear classification

(引自coursera week 4 Non-linear hypothesis)

for LR:

x1 = size, x2 = #bedrooms, x3 = #floors, x4 = age....x100, n=100

只考虑二阶项，约有5000features， O(n^2)

这会造成特征太多， 进而过拟合。

以汽车识别为例，取50*50像素的图片，则有n=2500（灰度）， 二阶项约为2500^2/2， 特征太过于多，因此需要更复杂的算法。

1.2 Neurons and the brain

Neural Networks:

origins: Algorithms that try to mimic the brain

2. Neural Networks

2.1 Model Representation I

Neurons in the brain:

(引自coursera week 4 Model Representation I)

neuron model: Logistic unit:

(引自coursera week 4 Model Representation I)

通常会加入x0， 被称为bias unit, always = 1, 相当于阈值单元

θ又叫weights

Neuron networks:

(引自coursera week 4 Model Representation I)

ai^j = "activation of unit i in layer j"

Θ^j = matrix of weights controlling function mapping from layer j to layer j+1

if networks has Sj units in layer j, S(j+1) unit in layer j+1, then Θ^j will be of dimension S(j+1) * (Sj + 1)

2.2 Model Representation II

Forward propagation: Vectorized implemention

(引自coursera week 4 Model Representation II)

于是有：

Neural networks learning its own features

(引自coursera week 4 Model Representation II)

hidden layer  -----> output layer is like ligistic regression, the difference is NN use new feature calulated by hidden layer instead of x1, x2, x3.

 

3. Application

3.1 Example and Intuition I

Non-linear classification example: XOR/XNOR

x1, x2 are binary， y = x1 XNOR x2

(引自coursera week 4 Example and Intuition I)

(引自coursera week 4 Example and Intuition I)

simple example: AND

x1, x2 are binary, y = x1 AND x2

(引自coursera week 4 Example and Intuition I)

(引自coursera week 4 Example and Intuition I)

3.2 Example and Intuition II

接上一小节 y = x1 XNOR x2

合并AND, (NOT x1) AND (NOT x2), x1 OR x2即可解决， 具体如下：

(引自coursera week 4 Example and Intuition I)

3.3 Multiclass Classification

Multiple output units, One vs all:

want classifly pedestrian, car, moto, track.

training set 

y^(i) one of 

NN:

笔者在写练习时，在多分类这里出错，因此贴上matlab code：

%% p为预测结果， all_theta为每一类对应的参数，为num_labels * (n+1)
p = zeros(size(X, 1), 1);

X = [ones(m, 1) X];


h = 1.0 ./ (1.0 + exp(-X * all_theta'));
[~, p] = max(h, [], 2);