[LA] Different convexity

1. Convex and strictly convex

Common used notations about convexity are convex and strictly convex. Their definitions are

Definition 1: [convex]: f(x) is said to be convex if one of the following holds x,y

f(λx+(1λ)y)λf(x)+(1λ)f(y)

Definition 2: [strictly convex]: f(x) is said to be strictly convex if one of the following holds x,y

f(λx+(1λ)y)<λf(x)+(1λ)f(y)

And there exist two equivalent definitions:

Theorem 3. [first order condition(1)]: If f(x) is first differentiable, then f(x) is convex iff x,y

f(y)f(x)+f(x)(yx)

This equivalence holds for strictly convex for > .

proof:
necessary: If f(x) is convex and let λ0

f(x)f(λx+(1λ)y)(1λ)f(y)λ=f(y)+f(y+λ(xy))f(y)λ=f(y)+f(y+λ(xy))f(y)λ(xy)(xy)=f(y)+f(y)(xy)

sufficient: If the first order condition is satisfied,
f(x)f(y)f(λx+(1λ)y)+f(λx+(1λ)y)(1λ)(xy)f(λx+(1λ)y)+f(λx+(1λ)y)λ(yx)

combining these two together, we get:

λf(x)+(1λ)f(y)f(λx+(1λ)y)

**Theorem 4. [first order condition(2)[monotone of f(x) ]]: f(x) is convex iff (f(x)f(y))(xy)0 .
proof: necessary:
If f(x) is convex, then x,y , we have

f(x)f(y)+f(y)(xy)f(y)f(x)+f(x)(yx)

adding these two equalities:
f(x)+f(y)f(y)+f(x)+(f(y)f(x))(xy)

i.e.
(f(x)f(y))(xy)0

sufficient:
Let g(t)=f(x+t(yx)) . Then g(x)=f(x+t(yx))(yx)

g(t)g(0)=f(x+t(yx))(yx)f(x)(yx)=1t(f(x+t(yx))f(x))t(yx)0

so g(t) is monotone increasing.

So

g(1)=g(0)+10g(t)dtg(0)f(y)f(x)+f(x)(yx)

Theorem 5. [second order condition]: If f(x) is second differentiable, then f(x) is convex iff x

2f(x)0

This equivalence holds for strictly convex for > .
proof:
For simply, we firstly prove one variable function situation:
If h(x):xR is convex iff its twice derivative h′′(x)0
sufficient:
From h′′(x)0 and taylor expansion, we have
h(y)h(x)+h(x)(yx)

and from last theorem, we know h(x) is convex.
necessary:
xzy , we have z=λx+(1λy) with λ=yzyx
h(z)=h(λx+(1λ)y)λh(x)+(1λ)h(y)=yzyxh(x)+zxyxh(y)

(yx)h(z)(yz)h(x)+(zx)h(y)

(yz)(h(z)h(x))(zx)(h(y)h(z))

h(z)h(x)zxh(y)h(z)yz

So for t1xzyt2 , we have

h(x)h(t1)xt1h(z)h(x)zxh(y)h(z)yzh(t2)h(y)t2y

letting t1x and t2y , we have
h(x)h(z)h(x)zxh(y)h(z)yzh(y)

So h(x) is increasing h′′(x)0 .

Now we prove for multivariable function. Let g(t)=f(x+t) be one variable function.
sufficient:
From convexity of f(x) ,

g(λt1+(1λ)t2)=f(x+λt1+(1λ))λf(x+t1)+(1λ)f(x+t2)=g(t1)+g(t2)

So g(t) is convex as a one variable function. Then
g′′(t)=t2f(x+t)0

So
2f(x)0

necessary:*
Let g(t)=f(x+t(yx)) , then
g′′(t)=(yx)t2f(x+t(yx))(yx)0

So g(t) is convex.

Then

f(λx+(1λ)y)=f(x+(1λ)(yx))=g(1λ)=g(λ0+(1λ)1)λg(0)+(1λ)g(1)=λg(x)+(1λ)f(y)

So f(x) is convex.

From the proof, we know that the convexity of a function on a convex set is one-dimensional fact.

Intuition:

  • convex says a function is convex a linear function
  • strictly convex says a function is convex > a linear function

2. Strong convex

Definition 3: [strong convex]: f(x) is said to be m-strong convex if f(x)m2x22 is convex.

Then from last section, we have that:
first order condition (1):

f(y)f(x)+f(x)(yx)+m2yx22

first order condition (2)[monotone of derivative]:
(f(x)f(y))(xy)>mxy22

seconf order condition :
2f(x)>mI

Intuition: strong convex says a function is convex a quadratic function.

Theorem: If a function is strong convex then the first derivative of it is Lipschitz continuous.
proof: Firstly, we claim that the subset S={x,f(x)f(x(0))} is closed. Since yS , we have

f(x(0))f(y)f(x)+f(x)(yx)+m2yx22yx222mf(x(0))

And the maximum eigenvalue of 2f(x) is continuous, so there exists a upper bound M for 2f(x), which says that f(x) is lipschitz continuous.

  • 1
    点赞
  • 3
    收藏
    觉得还不错? 一键收藏
  • 1
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值