ℓ1 Constrained Least Squares
In sparse learning, ℓ1 constrained LS, also known as Lasso Regression, is a common learning method:
minθJLS(θ)s.t.∥θ∥1≤R
where
∥θ∥1=∑j=1b|θj|
Generally speaking, the solution of an ℓ1 constrained LS is located on the axis, that is to say, there are several parameters θj equal to zero (sparse).
Then how to solve it? Given the indifferentiable property of the absolute value at the origin, solving an ℓ1 constrained LS is not so easy as solving the ℓ2 constrained one. However, we can still apply Lagrange multiplier.
minθJ(θ),J(θ)=JLS(θ)+λ∥θ∥1