1. Definitions
∇f(x) is L- Lipschitz continuous, then we have
- (1) ∥∇f(x)−∇f(y)∥2≤L∥x−y∥2 (note that this does not assume convexity of f(x) )
- (2) L2xTx−f(x) is convex ( if dom(f) is convex )
- (3) ∇2f(x)≤L⋅I ( if f(x) is twice differentiable )
- (4) f(y)≤f(x)+∇f(x)⋅(y−x)+L2∥y−x∥22 ( if f(x) is convex)
(一) (1) to (2): From the equivalent definition of convexity, that
f(x) is convex iff (∇f(x)−∇f(y))T(x−y)≥0 and dom(f) is convex.
We just need to prove that
[(L⋅x−∇f(x))−(L⋅y−∇