python实现ADMM

|lyc

已于 2024-03-20 00:31:12 修改

阅读量1.3k

点赞数 26

分类专栏：深度学习文章标签：深度学习

于 2024-03-20 00:28:34 首次发布

本文链接：https://blog.csdn.net/qq_57730943/article/details/136859386

版权

深度学习专栏收录该内容

2 篇文章 0 订阅

订阅专栏

对偶上升法

1.优化问题

$\underset{x_1,x_2}{\min} f(x_1,x_2)=2(x_1-1)^2+(x_2+2)^2$
$s\ldotp t\ldotp\begin{cases}x_1\ge 2\\x_2\ge 0 \end{cases}$

2.拉格朗日函数

$L(x_1,x_2,\lambda _1,\lambda _2)=2(x_1-1)^2+(x_2+2)^2+\lambda _1(2-x_1)+\lambda _2(-x_2)$

3.更新公式

${x_1}^{k+1} ={x_1}^k - t_x * (4 * (x_1 - 1) - \lambda _1)$
${x_2}^{k+1} ={x_2}^k - t_x * (2 * (x_2 + 2) - \lambda _2)$
${\lambda _1} ^ {k + 1} = {\lambda _1} ^ k + t_\lambda(2 - x_1)$
${\lambda _2} ^ {k + 1} = {\lambda _2} ^ k + t_\lambda(- x_2)$

import numpy as np
import matplotlib.pyplot as plt

x = np.array([[100], [-120]])
lambda_ = np.zeros_like(x)

x_v, f_v, G_v, L_v, lambda_v = [], [], [], [], []

t_x = 0.1
t_lambda = 0.1
max_iters = 10000
tol = 10 ** (-5)

for i in range(max_iters):
    x[0] = x[0] - t_x * (4 * (x[0] - 1) - lambda_[0])
    x[1] = x[1] - t_x * (2 * (x[1] + 2) - lambda_[1])
    if(x[0] < 2):
        x[0] = 2

    Grad = np.array([[2 - x[0].item()], [- x[1].item()]])
    lambda_ = np.maximum(lambda_ + t_lambda * Grad, 0)

    f = 2 * (x[0] - 1) ** 2 + (x[1] + 2) ** 2
    L = f + lambda_[0] * (2 - x[0]) + lambda_[1] * (-x[1])

    x_v.append(x.copy())
    G_v.append(Grad)
    f_v.append(f)
    L_v.append(L)


    if(abs(f - L) < tol):
        print(i + 1)
        break
print(x)
plt.figure()
plt.plot(range(2, len(L_v) + 1), np.array(f_v[1:]) - np.array(L_v[1:]), '-b*', linewidth = 2)
plt.show()

ADMM

1.原理

$f(x)+g(z)\\s\ldotp t\ldotp Ax+Bz=c$
其增广拉格朗日函数为:
$L_\rho(x,z,\lambda)=f(x)+g(z)+\lambda^T(Ax+Bz-c)+(\rho/2){\lVert Ax+Bz-c\rVert}_2^2$

Scaled Form更新方法

$x^{k + 1} = \underset{x}{\argmin}(f(x) + (\rho/2){\lVert Ax+Bz^k-c+u^k\rVert}_2^2)$
$z^{k + 1} = \argmin(g(z) + (\rho/2){\lVert Ax^{k+1}+Bz^k-c+u^k\rVert}_2^2)$
$u^{k + 1} = u^k+Ax^{k + 1} + Bz^{k + 1 - c}$

1.优化问题

$\underset{x,y}{\min}(x-1)^2+(y-2)^2$
$s\ldotp t\ldotp\begin{cases}0\le x \le 2\\1\le y\le 4\\2x+3y=5\end{cases}$

import numpy as np
import matplotlib.pyplot as plt
from scipy.optimize import minimize_scalar

def f_x(x, y, rho, u):
    return (x - 1) ** 2 + (rho / 2) * np.linalg.norm(2 * x + 3 * y - 5 + u) ** 2

def f_y(x, y, rho, u):
    return (y - 2) ** 2 + (rho / 2) * np.linalg.norm(2 * x + 3 * y - 5 + u) ** 2

x = 1
y = 1
rho = 1
u = 0
max_iters = 100

for i in range(max_iters):
    # print(x,y)
    resx = minimize_scalar(lambda x: f_x(x, y, rho, u), bounds = (0, 3), method = 'bounded')
    x = resx.x
    resy = minimize_scalar(lambda y: f_y(x, y, rho, u), bounds = (1, 4), method = 'bounded')
    y = resy.x

    u = u + 2 * x + 3 * y - 5

    rho = min(2000, 1.1 * rho)

print(x,y)