机器学习（Machine learning）7——实际问题解决经验（practical problem solving experience）

一、数据集划分

1. Supposed have a dataset:
   
2. Model selection:
   
3. 然后依次求出：
   
4. 最后用测试集来衡量算法所选出的模型。

二、Diagnosing bias & variance

Bias(underfit):
Jtrain(theta) will be high
Jcv(theta) 约= Jtrain(theta)
Variance(overfit):
Jtrain(theta) will be low
Jcv(theta) >> Jtrain(theta)

三、Regularization and bias/variance

Polynomial regression with regularization:
Model:

1. 参数的选取:
   
2. 参数的影响:
   

四、high bias & variance

1. High bias
   
   If a learning algorithm is suffering from high bias, getting more training data will not help much.

2. High variance
   
   If a learning aigorithm is suffering from high variance, getting more training data is likely to help.

五、Neural networks and overfitting

1. “small” neural network
   fewer parameters; more prone to underfitting
   
   Computationally cheaper

2. “large” neural network
   more parameters; more prone to overfitting
   
   
   computationally more expensive.

It turns out to use a large neural network that the larger, the better(性能越好), if it’s overfitting,you can use regularization to address overfitting.
Using a large neural network is more effect than using a small neural network.

六、Debugging a learning algorithm

 1. get more trainings examples——fix high variance
 2. try smaller sets of features——fix high variance
 3. try getting additional features——fix high bias
 4. try adding polynomial features——fix high bias
 5. try decreasing lamuda ——fix high bias
 6. try increasing lamuda——fix high variance

七、其它

1. 当我们实现一个算法时，首先是要尽可能地去实现它，不要担心算法太简单或者效果太差。
2. 实现出之后，看看它所造成的错误，即通过误差分析来看看它出现了什么错误，这样就可以决定之后的优化算法。
3. 如果已经简单直接实现了一个算法，又有了一个数值评价指标。之后就可以帮助你实现新的想法，即是否能够提高算法的表现，这能让你更快的决定你的算法应该放弃什么，应该包含什么。