Stanford机器学习课程笔记1-Linear Regression与Logistic Regression

转载请注明出处:  http://xiahouzuoxin.github.io/notes/

• 课程计划
• Linear Regression与预测问题
  • Locally Weighted Linear Regression
• Logistic Regression与分类问题
  • 特征映射与过拟合(over-fitting)

Stanford机器学习课程的主页是： http://cs229.stanford.edu/

课程计划

主讲人Andrew Ng是机器学习界的大牛，创办最大的公开课网站coursera，前段时间还听说加入了百度。他讲的机器学习课程可谓每个学计算机的人必看。整个课程的大纲大致如下：

1. Introduction (1 class) Basic concepts.
2. Supervised learning. (6 classes) Supervised learning setup. LMS. Logistic regression. Perceptron. Exponential family. Generative learning algorithms. Gaussian discriminant analysis.
   Naive Bayes. Support vector machines. Model selection and feature selection. Ensemble methods: Bagging, boosting, ECOC.
3. Learning theory. (3 classes) Bias/variance tradeoff. Union and Chernoff/Hoeffding bounds. VC dimension. Worst case (online) learning. Advice on using learning algorithms.
4. Unsupervised learning. (5 classes) Clustering. K-means. EM. Mixture of Gaussians. Factor analysis. PCA. MDS. pPCA. Independent components analysis (ICA).
   
5. Reinforcement learning and control. (4 classes) MDPs. Bellman equations. Value iteration. Policy iteration. Linear quadratic regulation (LQR). LQG. Q-learning. Value function approximation. Policy search. Reinforce. POMDPs.

本笔记主要是关于Linear Regression和Logistic Regression部分的学习实践记录。

Linear Regression与预测问题

举了一个房价预测的例子，

Area(feet^2)	#bedrooms	Price(1000$)
2014	3	400
1600	3	330
2400	3	369
1416	2	232
3000	4	540
3670	4	620
4500	5	800

Assume：房价与“面积和卧室数量”是线性关系，用线性关系进行放假预测。因而给出线性模型， h_θ(x) = ∑