Lecture 1 Introduction of Machine Deep Learning

Lecture 1: Introduction of Machine / Deep Learning

Chapter 1 Introduction

Machine Learning $\approx$ Looking for Function

Different types of Functions

Regression

The function outputs a scalar.

Classification

Given options (classes), the function outputs the correct one.

Structured Learning

create something with structure (image, document) 让机器学会输出一些有特定结构的东西（例如，图片、文档等）

这部分的机器学习任务占比更高。

How to find a function? A Case Study

Function with Unknown Parameters

Define Loss from Training Data

Optimization

$$w^{\ast },b^{\ast }=\arg \underset{w,b}{\min }L$$

Gradient Descent

单个参数做梯度下降，比如将 $b$ 作为常数处理

两个参数做梯度下降

Training Procedure of Machine Learning

Linear models are too simple, we need more sophisticated modes.

Sigmoid Function

Training Procedure of Machine Learning

Step 1 Function

未知参数都统称为 $\theta$

Step 2 Loss

• Loss is a function of parameters $L(\theta )$;
• Loss means how good a set of value is.

Step 3 Optimization
$${\theta }^{\ast }=\arg \underset{\theta }{\min }L,\theta =\left[\begin{array}{c}{\theta }_1\\ {\theta }_2\\ {\theta }_3\\ ⋮\end{array}\right].$$

ReLU (Rectified Linear Unit)

Backpropagation

Gradient Descent

Chain Rule

Forward and Backward Pass

Forward Pass

Backward Pass

Summary