采用二次插值法进行不精确一维搜索

简介

上次介绍了不精确一维搜索的概念和准则，以及采用直接法进行不精确一维搜索的实例，本次介绍采用二次插值法来进行不精确一维搜索的方法。$\phi (\lambda )=f({\mathbf{x}}^k+\lambda {\mathbf{s}}^k)$，若已知点$({\lambda }_1,{\phi }_1),({\lambda }_2,{\phi }_2),$，则可由抛物线法进行一维搜索那节的内容可知，可采用抛物线方程$g(\lambda )=a{\lambda }^2+b\lambda +c$来近似曲线，可推导得到近似极小点的计算公式如下：$$\hat{\lambda }={\lambda }_1+\frac{1}{2}({\lambda }_2-{\lambda }_2)/[1+\frac{{\phi }_1-{\phi }_2}{({\lambda }_2-{\lambda }_2){\phi }_1^{{}^{\prime }}}]$$
具体实施步骤如下：

1. 给定$c_1\in (0,1),c_2\in (c_1,1),T>0$，令${\lambda }_1=0,{\lambda }_2=1,{\lambda }_3=+\infty ,i=0$;
2. 若${\lambda }_2$满足条件1、2（具体条件见不精确一维搜索），则令${\lambda }_k={\lambda }_2$,计算终止；否则转步骤3；
3. 计算${\phi }_1,{\phi }_2,{\phi }_1^{{}^{\prime }}=▽f({\mathbf{x}}^k+{\lambda }_1{\mathbf{s}}^k{)}^T{\mathbf{s}}^k$和$\hat{\lambda }$，令$i=i+1$;
4. 若${\lambda }_1<\hat{\lambda }<{\lambda }_2$，则令${\lambda }_3={\lambda }_2$，${\lambda }_2=\hat{\lambda }$，返回步骤2；若${\lambda }_2⩽\hat{\lambda }<{\lambda }_3$，则令${\lambda }_1={\lambda }_2,{\lambda }_2=\hat{\lambda }$，返回步骤2；否则返回步骤5；
5. 若$\hat{\lambda }⩽{\lambda }_1$，则令$\hat{\lambda }={\lambda }_1+T△\lambda$；若$\hat{\lambda }⩾{\lambda }_3$，则令$\hat{\lambda }={\lambda }_3-T△\lambda ,△\lambda ={\lambda }_3-{\lambda }_1$，令${\lambda }_2=\hat{\lambda }$，返回步骤2。

优化问题

求$f(x)=x^4-4x^3-6x^2-16x+4$的极小值点。设置参数$c_1=0.5,c_2=0.7,T=0.1,{\lambda }_1=0,{\lambda }_2=1,{\lambda }_3=+\infty ,x=11,s=-1$，计算程序如下：

def cal_fun_f(x):
    """
    函数值计算函数
    """
    return x**4-4*x**3-6*x**2-16*x+4

def derivative_cal_f1(x):
    """
    一阶导数计算函数
    """
    return 4*x**3-12*x**2-12*x-16

def derivative_cal_f2(x):
    """
    二阶导数计算函数
    """
    return 12*x**2-24*x-12

def wolfe_powell_rule1(f_x, f_x_, c1, lam, d_x, s):
    """
    判断准则1
    :param f_x:k点函数值
    :param f_x_:k+1点函数值
    :param c1:给定常数
    :param lam:步长
    :param d_x:k点一阶导数
    :param s:方向向量
    :return:
    """
    if (f_x-f_x_) >= (-c1*lam*d_x*s):
        return True
    else:
        return False

def wolfe_powell_rule2(d_x_, s, c2, d_x):
    """
    判断准则2
    :param d_x_: k+1点一阶导数
    :param s: k点方向导数
    :param c2: 给定常数
    :param d_x: k点一阶导数
    """
    if (d_x_*s) >= (c2*d_x*s):
        return True
    else:
        return False

def imprecise_parabolic_method(c1, c2, T, lam1, lam2, lam3, x, s):
    """
    采用二次插值法进行不精确一维搜索
    :param T: 大于0
    :param c1:给定常数
    :param c2:给定常数
    :param lam1:步长区间左端点
    :param lam2:步长
    :param lam3:步长区间右端点
    :param x:初始点
    :param s:方向向量
    """
    i = 0
    while True:
        temp_x = x
        x2 = x+lam2*s
        f_x = cal_fun_f(x)
        f_x2 = cal_fun_f(x2)
        d_x = derivative_cal_f1(x)
        d_x2 = derivative_cal_f1(x2)
        rule1_judge = wolfe_powell_rule1(f_x, f_x2, c1, lam2, d_x, s)
        rule2_judge = wolfe_powell_rule2(d_x2, s, c2, d_x)
        # print(f"第{i}迭代点，步长", x2, lam2)
        if rule1_judge and rule2_judge:
            return x2, lam2
        else:
            x1 = x + lam1 * s
            f_x1 = cal_fun_f(x1)
            d_x1 = derivative_cal_f1(x1)
            d_lam1 = d_x1*s
            lam = lam1 + 0.5*(lam2 - lam1)/(1 + (f_x1 - f_x2) / ((lam2 - lam1) * d_lam1))
            i += 1
            if lam < 0:
                return x, lam
            elif lam1 < lam < lam2:
                lam3 = lam2
                lam2 = lam
            elif lam2 <= lam < lam3:
                lam1 = lam2
                lam2 = lam
            elif lam <= lam1:
                delta_lam = lam3-lam1
                lam = lam1+T*delta_lam
                lam2 = lam
            else:
                delta_lam = lam3 - lam1
                lam = lam3-T*delta_lam
                lam2 = lam

def main_loop(c1, c2, T, lam1, lam2, lam3, x, s, gap):
    i = 0
    while True:
        x, lam = imprecise_parabolic_method(c1, c2, T, lam1, lam2, lam3, x, s)
        i += 1
        print(f"第{i}次迭代极小值点,迭代步长", x, lam)
        if abs(derivative_cal_f1(x)) < gap or lam<0:
            break

if __name__ == "__main__":
    main_loop(0.5, 0.7, 0.1, 0, 1, float("inf"), 11, -1, 1e-3)

计算结果如下：

第1次迭代极小值点,迭代步长 7.608378870673952 3.3916211293260483
第2次迭代极小值点,迭代步长 6.608378870673952 1
第3次迭代极小值点,迭代步长 5.608378870673952 1
第4次迭代极小值点,迭代步长 4.608378870673952 1
第5次迭代极小值点,迭代步长 4.048639983992185 0.559738886681767
第6次迭代极小值点,迭代步长 4.000335997451437 0.04830398654074813
第7次迭代极小值点,迭代步长 4.00000001612566 0.0003359813257768087

但是这个计算结果并不稳定，更改初始点和参数之后，收敛结果改变的比较明显，读者自行甄别。