Life's Joy & Comfortable

[笔记] Convex Optimization 2015.11.18

Let {fa:aA} be a collection of convex functinos from Rn to R, with same domain, then f(x)=supaAfa(x) is a convex function.

  • Proof1: Take x,ydomf, θ[0,1],

(The proposition is true for domf=aAdomfa, but false for domf=aAdomfa.)

  • Proof2:


  • Example: Let x[i] denote the i-th largest component of x=(x1,,xn)Rn,
    then maxsumr(x)=x[1]+x[2]++x[r] is convex.

  • Proof: maxsumr(x)xi1+xi2++xir
    for any {i1,,ir}{1,,n} with ijik for jk
    so it is convex.

  • Example: Let CRn, define Sc(x)=sup{yTx:yC}, then Sc is convex.

  • Example: Let f:SnR, f(X) is the largest eigenvalue of X.
    Claim: f is convex.

  • Proof: First claim that f(X)=sup{yTXy:y2=1}
  • Proof of claim:


  • Example: Let f:Rm×nR be defined by f(X)=X2 where X2=supy2=1Xy2 is the spectural norm of XRm×n.

  • Claim: f(X)=supu,v{uTXv:u2=1,v2=1}
    because Xv2=sup{uTXv:uRn,u2=1}
    more generally: Xa,b=sup{Xvb:va=1}=sup{uTXv:va=1,ub=1}

  • Composition: Have f(x)=h(g(x)), xRn, g(x)R, when is f convex?

  • Would-be-proof: Take x,ydomf, θ[0,1] then
    f(θx+(1θ)y)==h(g(θx+(1θ)y))h(θg(x)+(1θ)g(y))θh(g(x))+(1θ)h(g(y))θf(x)+(1θ)f(y)use domf is convexuse g is convex/concave, h is nondecreasing/nonincreasinguse h is convex
- - 1 2 3 4
Condition g convex concave convex concave
- h nondecreasing nonincreasing nonincreasing nondecreasing
- h convex convex concave concave
Result f convex convex concave concave

- Example: g(x)=x21,h(x)=x3/2,domh=R+
then dom(hg)=(,1][1,+] is not convex.

  • Example 3.13:
    If g is convex then eg(x) is convex. 1
    If g is concave and positive then log(g(x)) is concave. 4
    If g is concave and positive then 1g(x) is convex. 2
    If g is convex and nonnegative and p1, g(x)p is convex. 1
    g:RnRm, say g is K-convex, where K is a cone in Rm,
    if domg is convex and g(θx+(1θ)y)Kθg(x)+(1θ)g(t)
    xKyh(x)h(y) K-nondecreasing.

  • Example 3.14:
    h(z)=log(ki=1ezi), so log(ki=1egi(x)) will be convex if g1,,gk are convex.

  • Minimization: Let f:Rn×Rm be a convex function, then
    g(x)=infy:(x,y)domff(x,y) is convex.

  • Proof: Let x1,x2domg, θ[0,1].
    then for any ε>0, y1,y2, s.t.
    g(x1)f(x1,y1)ε, g(x2)f(x2,y2)εand


  • Example: Let CRn be a convex set, then
    g(x)=infyCxy is a convex function.

  • Proof: Use f(x,y)=xydomf=Rn×C,

Consider function g(w)=infxmi=1wi(aTixbi)2, “weighted least square”
concave function of w
Let g(w)=infx(Axb)TW(Axb)=infx(xTATWAx2bTWAx+bTWb)
assume ATWA0, then optimal x=(ATWA)1ATWb

optimal value====bTWA(ATWA)1ATWb2bTWA(ATWA)1ATWb+bTWbbTWA(ATWA)1ATWb+bTWbi=1mb2iwi(i=1mwibiaTi)(i=1mwiaiaTi)1(i=1mwibiai)i=1mb2iwii,jwiwjbibjaTi(i=1mwiaiaTi)1aj


文章标签: Convex
个人分类: 笔记
上一篇Metric learning 度量学习
下一篇Raspberry Pi B+ 实验
想对作者说点什么? 我来说一句

凸优化convex optimization 中文电子版

2017年01月23日 47.21MB 下载


2017年09月13日 5.62MB 下载