Neural network - part 3 - Cost Function

Neural network - part 3 - Cost Function

reference Andrew Ng “machine learning8”

In order to fit the paramater, we need to minimize the cost function.How can we get the cost function. It likes the cost function in logistic regression but has some difference. To have a deep learning witth the cost function in nn, we ought to memorize and know backward propagation.

• Cost Function
• Backpropagation Algorithm

---

Cost Function

1. Two types of classification
   There are two classification, one is binary classification, the other is multi-class classification. They have different output, just like below.
   
2. Cost function
   The cost function in nn has some similarity with the cost function in logistic regresstion. The cost fucntion below is the total layers cost function and $h_{\theta }(x)$ means the final output. $\theta$ means a big matrix. We will not put the bias unit’s parameter into cost function , which refers to ${\theta }_{i0}$.

Backpropagation Algorithm

In order to fit the parameters, we need to minimize the cost funtion $J(\theta )$ . Thus, we neet to have $J(\theta )$ and the partial derivative term $dd$.