坐标下降/上升算法(Coordinate Decsent/Ascent)

坐标下降/上升算法(Coordinate Decsent/Ascent)

1.坐标上升/下降算法的原理：

坐标上升法(Coordinate Ascent)每次通过更新函数中的一维，通过多次的迭代以达到优化函数的目的。

过程可视化出来后大概是这样的：

蓝色部分就是迭代的过程，一直到达中心的最优部分

2.自己的一些理解

首先我认为坐标上升/下降算法是同一种算法（这个问题我纠结了好久！！！）因为有些函数有最大值，有些有最小值。不同的函数应用这种算法的结果显然是不一样的。当然，这个算法的主要思想也就是先达到局部最优，由局部最优达到整体最优。本算法在支持向量机中得到了应用。

python实现：

# 坐标下降算法
import numpy as np
import matplotlib.pyplot as plt

# 建立步长为0.025，即每隔0.01取一个点
delta = 0.025
x = np.arange(-3.0, 3.0, delta)
y = np.arange(-3.0, 3.0, delta)
# x,y =
# [ -3.00000000e+00  -2.97500000e+00  -2.95000000e+00  -2.92500000e+00
#   -2.90000000e+00  -2.87500000e+00  -2.85000000e+00  -2.82500000e+00
#   -2.80000000e+00  -2.77500000e+00  -2.75000000e+00  -2.72500000e+00
#   -2.70000000e+00  -2.67500000e+00  -2.65000000e+00  -2.62500000e+00
#   -2.60000000e+00  -2.57500000e+00  -2.55000000e+00  -2.52500000e+00
#   -2.50000000e+00  -2.47500000e+00  -2.45000000e+00  -2.42500000e+00
#   -2.40000000e+00  -2.37500000e+00  -2.35000000e+00  -2.32500000e+00
#   -2.30000000e+00  -2.27500000e+00  -2.25000000e+00  -2.22500000e+00
#   -2.20000000e+00  -2.17500000e+00  -2.15000000e+00  -2.12500000e+00
#   -2.10000000e+00  -2.07500000e+00  -2.05000000e+00  -2.02500000e+00
#   -2.00000000e+00  -1.97500000e+00  -1.95000000e+00  -1.92500000e+00
#   -1.90000000e+00  -1.87500000e+00  -1.85000000e+00  -1.82500000e+00
#   -1.80000000e+00  -1.77500000e+00  -1.75000000e+00  -1.72500000e+00
#   -1.70000000e+00  -1.67500000e+00  -1.65000000e+00  -1.62500000e+00
#   -1.60000000e+00  -1.57500000e+00  -1.55000000e+00  -1.52500000e+00
#   -1.50000000e+00  -1.47500000e+00  -1.45000000e+00  -1.42500000e+00
#   -1.40000000e+00  -1.37500000e+00  -1.35000000e+00  -1.32500000e+00
#   -1.30000000e+00  -1.27500000e+00  -1.25000000e+00  -1.22500000e+00
#   -1.20000000e+00  -1.17500000e+00  -1.15000000e+00  -1.12500000e+00
#   -1.10000000e+00  -1.07500000e+00  -1.05000000e+00  -1.02500000e+00
#   -1.00000000e+00  -9.75000000e-01  -9.50000000e-01  -9.25000000e-01
#   -9.00000000e-01  -8.75000000e-01  -8.50000000e-01  -8.25000000e-01
#   -8.00000000e-01  -7.75000000e-01  -7.50000000e-01  -7.25000000e-01
#   -7.00000000e-01  -6.75000000e-01  -6.50000000e-01  -6.25000000e-01
#   -6.00000000e-01  -5.75000000e-01  -5.50000000e-01  -5.25000000e-01
#   -5.00000000e-01  -4.75000000e-01  -4.50000000e-01  -4.25000000e-01
#   -4.00000000e-01  -3.75000000e-01  -3.50000000e-01  -3.25000000e-01
#   -3.00000000e-01  -2.75000000e-01  -2.50000000e-01  -2.25000000e-01
#   -2.00000000e-01  -1.75000000e-01  -1.50000000e-01  -1.25000000e-01
#   -1.00000000e-01  -7.50000000e-02  -5.00000000e-02  -2.50000000e-02
#   -1.06581410e-14   2.50000000e-02   5.00000000e-02   7.50000000e-02
#    1.00000000e-01   1.25000000e-01   1.50000000e-01   1.75000000e-01
#    2.00000000e-01   2.25000000e-01   2.50000000e-01   2.75000000e-01
#    3.00000000e-01   3.25000000e-01   3.50000000e-01   3.75000000e-01
#    4.00000000e-01   4.25000000e-01   4.50000000e-01   4.75000000e-01
#    5.00000000e-01   5.25000000e-01   5.50000000e-01   5.75000000e-01
#    6.00000000e-01   6.25000000e-01   6.50000000e-01   6.75000000e-01
#    7.00000000e-01   7.25000000e-01   7.50000000e-01   7.75000000e-01
#    8.00000000e-01   8.25000000e-01   8.50000000e-01   8.75000000e-01
#    9.00000000e-01   9.25000000e-01   9.50000000e-01   9.75000000e-01
#    1.00000000e+00   1.02500000e+00   1.05000000e+00   1.07500000e+00
#    1.10000000e+00   1.12500000e+00   1.15000000e+00   1.17500000e+00
#    1.20000000e+00   1.22500000e+00   1.25000000e+00   1.27500000e+00
#    1.30000000e+00   1.32500000e+00   1.35000000e+00   1.37500000e+00
#    1.40000000e+00   1.42500000e+00   1.45000000e+00   1.47500000e+00
#    1.50000000e+00   1.52500000e+00   1.55000000e+00   1.57500000e+00
#    1.60000000e+00   1.62500000e+00   1.65000000e+00   1.67500000e+00
#    1.70000000e+00   1.72500000e+00   1.75000000e+00   1.77500000e+00
#    1.80000000e+00   1.82500000e+00   1.85000000e+00   1.87500000e+00
#    1.90000000e+00   1.92500000e+00   1.95000000e+00   1.97500000e+00
#    2.00000000e+00   2.02500000e+00   2.05000000e+00   2.07500000e+00
#    2.10000000e+00   2.12500000e+00   2.15000000e+00   2.17500000e+00
#    2.20000000e+00   2.22500000e+00   2.25000000e+00   2.27500000e+00
#    2.30000000e+00   2.32500000e+00   2.35000000e+00   2.37500000e+00
#    2.40000000e+00   2.42500000e+00   2.45000000e+00   2.47500000e+00
#    2.50000000e+00   2.52500000e+00   2.55000000e+00   2.57500000e+00
#    2.60000000e+00   2.62500000e+00   2.65000000e+00   2.67500000e+00
#    2.70000000e+00   2.72500000e+00   2.75000000e+00   2.77500000e+00
#    2.80000000e+00   2.82500000e+00   2.85000000e+00   2.87500000e+00
#    2.90000000e+00   2.92500000e+00   2.95000000e+00   2.97500000e+00]
# 将原始数据变为网格形式
X, Y = np.meshgrid(x, y)
Z1 = -(X ** 2)
Z2 = -(Y ** 2)
# 原始函数
Z = 1.0 * (Z1 + 3 * Z2 + 2 * X * Y) + 6.0
# z= [[-12.       -12.000625 -12.0025   ..., -47.105625 -47.4025   -47.700625]
# [-11.701875 -11.70125  -11.701875 ..., -46.51125  -46.806875 -47.10375 ]
# [-11.4075   -11.405625 -11.405    ..., -45.920625 -46.215    -46.510625]
# ...,
# [-46.216875 -45.92125  -45.626875 ..., -11.11125  -11.111875 -11.11375 ]
# [-46.8075   -46.510625 -46.215    ..., -11.405625 -11.405    -11.405625]
# [-47.401875 -47.10375  -46.806875 ..., -11.70375  -11.701875 -11.70125 ]]

# 打开画布
plt.figure()
CS = plt.contour(X, Y, Z)  # 画等高线
# 初始化权重
a = []
b = []
a.append(2.0)
b.append(2.0)

j = 1

for i in range(200):
    a_tmp = b[j - 1]
    a.append(a_tmp)
    b.append(b[j - 1])

    j = j + 1

    b_tmp = a[j - 1] / 3
    a.append(a[j - 1])
    b.append(b_tmp)

plt.plot(a, b)
max_x1 = a[-1]
max_x2 = b[-1]

print('当取最大值的时候，x1的取值为：', max_x1)
print('当取最大值的时候，x2的取值为：', max_x2)

print('max f:', -(max_x1 ** 2) - 3 * (max_x2 ** 2) + 2 * max_x1 * max_x2 + 6)

plt.title('Coordinate Ascent')
plt.xlabel('x')
plt.ylabel('y')
plt.show()

谢谢！！！