python 系数带不等式约束的线性回归

问题描述

对于常见的线性回归问题，已有相当多工具包如sklearn的linear_regression可以实现。
现在对于经典线性回归
$y=b_1{x}_1+b_2{x}_1+b_3{x}_1$ 或 $y=Xb$
我们期望对系数$b_1,b_2,b_3$进行约束，给定其上下限即：
$lower<b_1,b_2,b_3<upper$

解决方案Scipy.optimize.lsq_linear

文档地址:https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.lsq_linear.html?highlight=lsq_linear#r74f8b7a68993-stir

官方示例

from scipy.sparse import rand
from scipy.optimize import lsq_linear
rng = np.random.default_rng()

m = 20000
n = 10000

A = rand(m, n, density=1e-4, random_state=rng)
b = rng.standard_normal(m)

lb = rng.standard_normal(n)
ub = lb + 1

res = lsq_linear(A, b, bounds=(lb, ub), lsmr_tol='auto', verbose=1)

理论文献参考

M. A. Branch, T. F. Coleman, and Y. Li, “A Subspace, Interior, and Conjugate Gradient Method for Large-Scale Bound-Constrained Minimization Problems,” SIAM Journal on Scientific Computing, Vol. 21, Number 1, pp 1-23, 1999.

BVLS
P. B. Start and R. L. Parker, “Bounded-Variable Least-Squares: an Algorithm and Applications”, Computational Statistics, 10, 129-141, 1995.