《深度学习2021》——第一节 机器学习基本概念

机器学习的目标

• Regression
• Classification
• Stuctured Learning

Machine Learning <=> Look for function

1. Function with unknown parameter (Model)
2. Define loss from train data

• input: w,b
• output: how good a set of value is

3. Optimization w*, b* = argmin L

Piecewise linear

From the perspective of deep learning, all curve can be approximated by constant + sum of a set of hard sigmoid

c sinmoid(b + wx)

• w: change slopes
• b: shigt
• c : height

When x is single variable

• y = b + sigma_i (c_i * sigmoid(b_i + w_i * x_i)

When x is multivariate

• y = b + sigma_i (c_i * sigmoid(b_i + sigma_j(w_ij * x_j))
  that is: y = b + c^T sigmoid(b + w^T *X)

深度学习的目标

Based on the above piecewise linear, the goal of deep learning can be described below:

• Define function with unknown parameters: y = b + c^T sigmoid( b + w^T * X)
• Define loss function: L(theta)
• Optimization: theta* = argmin L(theta)

Sigmoid can also be replaced by ReLu (rectified linear unit)
Tow ReLu can synthesize the hard sigmoid