作业ex1题目及答案源码下载地址ex1题目及答案
一. 单变量线性回归
问题背景:假如你是餐馆老板,已知若干城市中人口和利润的数据(ex1data1.txt),用线性回归方法计算该去哪个城市发展。 ex1.m是官方提供的入口函数,通过该函数补全其他函数的代码。
- warmUpExercise.m:输出5*5的单位矩阵
A = eye(5);
Matlab中,eye(n)
返回n*n的单位矩阵,eye(m,n)
返回m*n的单位矩阵
- plotData.m:给定x、y两列向量,画出相应点
figure; % open a new figure window
plot(x, y, 'rx', 'MarkerSize', 10); % Plot the data
ylabel('Profit in $10,000s'); % Set the y axis label
xlabel('Population of City in 10,000s'); % Set the x axis label
![](https://i-blog.csdnimg.cn/blog_migrate/93524deb82569d8ad90f7c945b3d011e.jpeg)
- computeCost.m:计算代价
假设hypothesis函数为
h θ = θ T x = θ 0 + θ 1 x 1 h_\theta=\theta^Tx=\theta_0+\theta_1 x_1 hθ=θTx=θ0+θ1x1
一共m个数据,代价函数为
J ( θ ) = 1 2 m ∑ i = 1 m ( h θ ( x ( i ) ) − y ( i ) ) 2 J(\theta)=\frac{1}{2m}\sum_{i=1}^m(h_\theta(x^{(i)})-y^{(i)})^2 J(θ)=2m1i=1∑m(hθ(x(i))−y(i))2
J = sum((X * theta - y).^2) / (2*m);
- gradientDescent.m:计算梯度
为方便求导,将 J ( θ ) J(\theta) J(θ)写成如下形式
J ( θ ) = 1 2 m [ ( h θ ( x ( 1 ) ) − y ( 1 ) ) 2 + ( h θ ( x ( 2 ) ) − y ( 2 ) ) 2 + . . . ] J(\theta)=\frac{1}{2m}[(h_\theta(x^{(1)})-y^{(1)})^2+(h_\theta(x^{(2)})-y^{(2)})^2+...] J(θ)=2m1[(hθ(x(1))−y(1))2+(hθ(x(2))−y(2))2+...]
对 θ 1 \theta_1 θ1求偏导得
∂ ∂ θ 1 J ( θ ) = 1 2 m [ 2 ( h θ ( x ( 1 ) ) − y ( 1 ) ) x 1 ( 1 ) + 2 ( h θ ( x ( 2 ) ) − y ( 2 ) ) x 1 ( 2 ) + . . . ] = 1 m ∑ i = 1 m ( h θ ( x ( i ) ) − y ( i ) ) x 1 ( i ) \begin{aligned} \frac{\partial}{\partial{\theta_1} }J(\theta)=&\frac{1}{2m}[2(h_\theta(x^{(1)})-y^{(1)})x^{(1)}_1+2(h_\theta(x^{(2)})-y^{(2)})x^{(2)}_1+...]\\ =&\frac{1}{m}\sum_{i=1}^m(h_\theta(x^{(i)})-y^{(i)})x^{(i)}_1 \end{aligned} ∂θ1∂J(θ)==2m1[2(hθ(x(1))−y(1))x1(1)+2(hθ(x(2))−y(2))x1(2)+...]m1i=1∑m(hθ(x(i))−y(i))x1(i)
所以有
θ j : = θ j − α 1 m ∑ i = 1 m ( h θ ( x ( i ) ) − y ( i ) ) x j ( i ) , ( j = 0 , 1 ) \theta_j:=\theta_j-\alpha\frac{1}{m}\sum_{i=1}^m(h_\theta(x^{(i)})-y^{(i)})x^{(i)}_j,(j=0,1) θj:=θj−αm1i=1∑m(hθ(x(i))−y(i))xj(i),(j=0,1)
theta(1) = theta(1) - alpha / m * sum(X * theta_s - y);
theta(2) = theta(2) - alpha / m * sum((X * theta_s - y) .* X(:,2));
theta_s=theta;
迭代多次后便可得到理想的 θ \theta θ值。接下来的根据x预测y值,以及绘制三维图、等高线图题目中已给出,这里不再赘述。matlab中的画图函数surf和画等高线图函数contour可以自行百度。
![](https://i-blog.csdnimg.cn/blog_migrate/4118c34a115644235bf571c8eca02c3b.jpeg)
![](https://i-blog.csdnimg.cn/blog_migrate/417564aa58146ea9c228fdbfed52086c.jpeg)
二、多变量线性回归
问题背景:卖房子,已知一系列数据,根据房屋大小和卧室数量两个特征,预测房子价钱,数据在(ex1data2.txt)。
- featureNormalize.m:数据特征归一化,因为房子大小和卧室数量不在一个数量级,直接计算收敛速度慢,所以先采用0均值标准化,归一化公式如下,详细介绍见线性回归与特征归一化(feature scaling):
z = x − μ σ z=\frac{x-\mu}{\sigma} z=σx−μ
均值方差根据matlab容易求得,代码如下
mu = mean(X); % mean value
sigma = std(X); % standard deviation
X_norm = (X - repmat(mu,size(X,1),1)) ./ repmat(sigma,size(X,1),1);
- computeCostMulti.m:计算代价
假设hypothesis函数为
h θ = θ T x = θ 0 + θ 1 x 1 + θ 2 x 2 h_\theta=\theta^Tx=\theta_0+\theta_1 x_1+\theta_2 x_2 hθ=θTx=θ0+θ1x1+θ2x2
一共m个数据,代价函数为
J ( θ ) = 1 2 m ∑ i = 1 m ( h θ ( x ( i ) ) − y ( i ) ) 2 J(\theta)=\frac{1}{2m}\sum_{i=1}^m(h_\theta(x^{(i)})-y^{(i)})^2 J(θ)=2m1i=1∑m(hθ(x(i))−y(i))2
J = sum((X * theta - y).^2) / (2*m);
- gradientDescentMulti.m:计算梯度
J ( θ ) J(\theta) J(θ)对 θ j \theta_j θj求偏导得
∂ ∂ θ j J ( θ ) = 1 2 m [ 2 ( h θ ( x ( 1 ) ) − y ( 1 ) ) x j ( 1 ) + 2 ( h θ ( x ( 2 ) ) − y ( 2 ) ) x j ( 2 ) + . . . ] = 1 m ∑ i = 1 m ( h θ ( x ( i ) ) − y ( i ) ) x j ( i ) \begin{aligned} \frac{\partial}{\partial{\theta_j} }J(\theta)=&\frac{1}{2m}[2(h_\theta(x^{(1)})-y^{(1)})x^{(1)}_j+2(h_\theta(x^{(2)})-y^{(2)})x^{(2)}_j+...]\\ =&\frac{1}{m}\sum_{i=1}^m(h_\theta(x^{(i)})-y^{(i)})x^{(i)}_j \end{aligned} ∂θj∂J(θ)==2m1[2(hθ(x(1))−y(1))xj(1)+2(hθ(x(2))−y(2))xj(2)+...]m1i=1∑m(hθ(x(i))−y(i))xj(i)
所以有
θ j : = θ j − α 1 m ∑ i = 1 m ( h θ ( x ( i ) ) − y ( i ) ) x j ( i ) , ( j = 0 , 1 ) \theta_j:=\theta_j-\alpha\frac{1}{m}\sum_{i=1}^m(h_\theta(x^{(i)})-y^{(i)})x^{(i)}_j,(j=0,1) θj:=θj−αm1i=1∑m(hθ(x(i))−y(i))xj(i),(j=0,1)
令 θ = [ θ 0 , θ 1 , θ 2 ] T \theta=[\theta_0,\theta_1,\theta_2]^T θ=[θ0,θ1,θ2]T, X ∈ R m × 3 , Y ∈ R m X\in R^{m\times3},Y\in R^{m} X∈Rm×3,Y∈Rm有
θ = θ − α 1 m X T ( X θ − Y ) \theta=\theta-\alpha\frac{1}{m} X^T(X\theta-Y) θ=θ−αm1XT(Xθ−Y)
theta = theta - alpha / m * X' * (X * theta - y);
- 根据迭代n次后得到的 θ \theta θ,计算1650平方英尺、3个卧室的房子价钱
price = [1 (([1650 3]-mu) ./ sigma)] * theta ;
三、用正规方程(解析解)直接计算 θ \theta θ
根据
∇
θ
J
(
θ
)
=
0
\nabla_\theta J(\theta)=0
∇θJ(θ)=0求得解析解如下(具体过程见视频课程)
θ
=
(
X
T
X
)
−
1
X
T
Y
\theta=(X^T X)^{-1}X^T Y
θ=(XTX)−1XTY
代码为
theta = pinv( X' * X ) * X' * y;
不再需要使用归一化处理数据。
解析解和梯度下降都可以很好的进行预测,上述实验数据虽然最后得到的 θ \theta θ不同,但是预测结果一致。解析方法在数据量极大时存在时间效率问题。具体可见正规方程normal equation。