Machine Learning Week One

Keep updating
This passage summarize the knowledge in web
https://www.coursera.org/learn/machine-learning/
Supervised learning
In supervised learning, we are given a data set and already know what our correct output should look like, having the idea that there is a relationship between the input and the output.
Supervised learning problems are categorized into “regression” and “classification” problems.
regression problem
map input variables to some continuous function
classification problem
map input variables into discrete categories

Unsupervised learning
Unsupervised learning allows us to approach problems with little or no idea what our results should look like. We can derive structure from data where we don’t necessarily know the effect of the variables.

notation
x(i) denotes the i th input
y(i) denotes the i th output
A pair (x(i),y(i)) denotes a training example
m denotes the number of training examples
X denote the space of input values
Y denotes the space of output values
h(x) denote a function(predictor) that maps x to y

our goal is find the proper

$$\theta_0 \space and \space \theta_1$$ that satisfies
$$\min_{(\theta_0,\theta_1)}\frac{1}{2m} \sum_{i=0}^m (h_\theta(x^{(i)}) - y^{(i)})^2$$

Gradient Descent Algorithm

$$\theta_j := \theta_j - \alpha\frac{\partial}{\partial\theta_j}J(\theta_0,\theta_1,...,\theta_j)$$
where $$\alpha$$ is the learning rate deciding the size of each step
and where $$\partial$$ means partial derivative
we should simultaneously update theta0 and theta1 at each iteration.

Typically h(x) can be in the form of

$$h_\theta(x_1,...,x_i) = \theta_1x_1+...+\theta_ix_i+\theta_0$$
Batch Gradient Descend
The method looks at every example in the entire training set on every step.