PyTorch学习笔记（二）PyTorch回归问题——求解二元一次方程组

PyTorch学习笔记（二）PyTorch回归问题——求解二元一次方程组

梯度下降算法（Gradient Descent）

1. 梯度下降算法：x=x-learningrate*梯度——慢慢逼近最优解，最后在x附近停留。目的：求函数最小值
2. 利用梯度下降算法，求解二元一次方程（Linear Equation ）：
   ▪𝑦=𝑤 ∗𝑥+𝑏
   1.567 = w * 1 + b
   3.043 = w * 2 + b
   ▪得到：w = 1.477 ；b = 0.089
3. 带有误差Noise的二元一次方程求解：
   ▪𝑦 =𝑤 ∗𝑥+𝑏+ 𝜖
   ▪𝜖∽ 𝑁(0, 1)
   即：求loss=（WX+b-Y)^2 极小值，其中：WX+b为预测值，Y为实际值。

Linear Regression线性回归

▪ Linear Regression：线性回归。预测值y是连续，且属于实数空间（负无穷，正无穷）的问题。
▪ Logistic Regression：逻辑回归，适合二分类。预测值y是连续，且属于（0，1）。一般在线性回归的基础上+激活函数sigmoid…。
▪ Classification：适合多分类。总的输出节点概率和为1，取概率最大的点为预测值。

回归问题实战

#计算Loss值：
def compute_error_for_line_given_points(b, w, points):#points为一系列x、y的组合
    totalError = 0
    for i in range(0, len(points)):
        x = points[i, 0]
        y = points[i, 1]
        # computer mean-squared-error
        totalError += (y - (w * x + b)) ** 2
    # average loss for each point
    return totalError / float(len(points))

#计算梯度gradient
#loss=（WX+b-Y)^2 
#w‘=w-lr*（loss对w求导）

def step_gradient(b_current, w_current, points, learningRate):
    b_gradient = 0
    w_gradient = 0
    N = float(len(points))
    for i in range(0, len(points)):
        x = points[i, 0]
        y = points[i, 1]
        # grad_b = 2(wx+b-y)——导数累加，因此需要求average
        b_gradient += (2/N) * ((w_current * x + b_current) - y)
        # grad_w = 2(wx+b-y)*x——导数累加，因此需要求average
        w_gradient += (2/N) * x * ((w_current * x + b_current) - y)
    # update w'
    new_b = b_current - (learningRate * b_gradient)
    new_w = w_current - (learningRate * w_gradient)
    return [new_b, new_w]

#循环计算梯度gradient
def gradient_descent_runner(points, starting_b, starting_w, learning_rate, num_iterations):
    b = starting_b
    w = starting_w
    # update for several times
    for i in range(num_iterations):
        b, w = step_gradient(b, w, np.array(points), learning_rate)
    return [b, w]

def run():
	#points获取
    points = np.genfromtxt("data.csv", delimiter=",")
    learning_rate = 0.0001
    initial_b = 0 # initial y-intercept guess
    initial_w = 0 # initial slope guess
    num_iterations = 1000
    print("Starting gradient descent at b = {0}, w = {1}, error = {2}"
          .format(initial_b, initial_w,
                  compute_error_for_line_given_points(initial_b, initial_w, points))
          )
    print("Running...")
    #一次跑100个
    [b, w] = gradient_descent_runner(points, initial_b, initial_w, learning_rate, num_iterations)
    print("After {0} iterations b = {1}, w = {2}, error = {3}".
          format(num_iterations, b, w,
                 compute_error_for_line_given_points(b, w, points))
          )

if __name__ == '__main__':
    run()