梯度下降法-python实现（个人学习+记录，望大佬指正）

梯度下降法：以三元函数f(x,y,z) = x^2 + y^2 + z^2 + 2x + 4y - 6z 为例，求其极值及极值点
python代码实现：

from sympy import *
import time
def gradient_descent():
    x = symbols('x')
    y = symbols('y')
    z = symbols('z')
    w = x**2 + y**2 + z**2 + 2*x + 4*y - 6*z
    x1 = 20
    y1 = 20
    z1 = 20
    alpha = 0.03
    dx = diff(w, x).subs({x: x1, y: y1, z: z1})
    dy = diff(w, y).subs({x: x1, y: y1, z: z1})
    dz = diff(w, z).subs({x: x1, y: y1, z: z1})
    last_result = (w.subs({x: x1, y: y1, z: z1}))                    # 先上升一步以进行后面的收敛判断
    x1 -= alpha * dx
    y1 -= alpha * dy
    z1 -= alpha * dz
    this_result = (w.subs({x: x1, y: y1, z: z1}))
    time_start = time.time()
    while abs(this_result - last_result) > 0.0001:   # 收敛条件：|f(x^k)-f(x^k+1)|<& (&取参数，代表下降已经不明显即收敛)
         last_result = (w.subs({x: x1, y: y1, z: z1}))
         dx = diff(w, x).subs({x: x1, y: y1, z: z1})
         dy = diff(w, y).subs({x: x1, y: y1, z: z1})
         dz = diff(w, z).subs({x: x1, y: y1, z: z1})
         x1 -= alpha * dx
         y1 -= alpha * dy
         z1 -= alpha * dz
         this_result = (w.subs({x: x1, y: y1, z: z1}))
    time_end = time.time()

    print("优化结果：x="+str(x1))
    print("\t\ty= " + str(y1))
    print("\t\tz= " + str(z1))
    print("\t\tmin f(x,y) = " + str(this_result))
    print('总时间：', time_end - time_start)
gradient_descent()

最后的输出：

优化结果：x=-0.988232354405425
		y= -1.98767199032949
		z= 3.00952618929085
		min f(x,y) = -13.9996631667427
总时间： 0.7430148124694824

实际根据多元函数极值求法求得的极值点：

注:A为三元函数的Hessian矩阵，A矩阵正定说明（-1，-2，3）为极小值点