【运筹学】utility theory

decision criteria

decision making under uncertainty

• action, $a_i\in A$
• state, $s_j\in S$
• reward, $r_{ij}$

a discrete newsvendor example

• c = 20 , p = 25 , S = { 600 , 700 , 800 , 900 , 1000 } c=20,p=25,S=\{600,700,800,900,1000\} c=20,p=25,S={600,700,800,900,1000}

• $r_{ij}=$ reward of purchasing i with demand j
  $$r_{ij}=\{\begin{array}{rlr}& (p-c)i,& i\le j\\ & pj-ci,& i>j\end{array}$$

• dominated actions 支配
  $$\begin{gathered} a_{i}isdominatedby{a}_i^{\prime }, \\ if{r}_{ij}\le r_{i^{\prime }j},\forall s_j\in S,r_{i^{\prime }j}<r_{i^{\prime }{j}^{\prime }},forsome{s}_{j^{\prime }}\in S \end{gathered}$$
  对于action i，在所有state里，reward都小于等于action i’ 带来的reward；

  ​ 在部分state里，reward比action i’ 带来的reward 小

decision criteria

• maximin：最大化最少的reward
  a i = arg ⁡ max ⁡ a i ∈ A { min ⁡ s j ∈ S r i j } a_i=\arg\max_{a_i\in A}\{\min_{s_j\in S}r_{ij} \} ai​=argai​∈Amax​{sj​∈Smin​rij​}

• maximax：最大化最大的reward
  a i = arg ⁡ max ⁡ a i ∈ A { max ⁡ s j ∈ S r i j } a_i=\arg\max_{a_i\in A}\{\max_{s_j\in S}r_{ij}\} ai​=argai​∈Amax​{sj​∈Smax​rij​}

• expected value：最大化期望

a i = arg ⁡ max ⁡ a i ∈ A { ∑ s j ∈ S p j r i j } a_i=\arg\max_{a_i\in A}\{\sum_{s_j\in S}p_jr_{ij}\} ai​=argai​∈Amax​{sj​∈S∑​pj​rij​}

• minimax regret：最小化最大的后悔（最好的action和实际采取的action之间的差值）
  a i = arg ⁡ min ⁡ a i ∈ A { max ⁡ s j ∈ S R i j } R e g r e t R i j = r i ∗ ( j ) , j − r i j , w h e r e i ∗ ( j ) = arg ⁡ max ⁡ a i ∈ A r i j , ∀ s j ∈ S r i ∗ ( j ) , j = 对 于 j ， 最 好 的 a c t i o n 产 生 的 r e w a r d r i j = 实 际 采 取 的 a c t i o n 产 生 的 r e w a r d a_i=\arg\min_{a_i\in A}\{\max_{s_j\in S}R_{ij}\} \\ \begin{aligned} &Regret& R_{ij} = & r_{i^*(j),j}-r_{ij}, \\ & where && i^*(j)=\arg\max_{a_i\in A}r_{ij}, &\forall s_j\in S \end{aligned} \\ \\ r_{i^*(j),j}=对于j，最好的action产生的reward \\ r_{ij}=实际采取的action产生的reward ai​=argai​∈Amin​{sj​∈Smax​Rij​}​Regretwhere​Rij​=​ri∗(j),j​−rij​,i∗(j)=argai​∈Amax​rij​,​∀sj​∈S​ri∗(j),j​=对于j，最好的action产生的rewardrij​=实际采取的action产生的reward

Utility Theory

Lottery （L）

• $(p_1,r_1;p_2,r_2;...,p_n,r_n)$

• r： reward

• p： probability

• tree representation

• 表示方式

  • $L_1$$p$$L_2$： prefers $L_1$
  • $L_{1}i{L}_2$: equivalent lotteries, indifferent between $L_1$ and $L_2$
  • $L_{2}p{L}_1$: prefers $L_2$

Von Neumann-Morgenstern Utility Theory

• Utility of the reward $r_i$, $u(r_i)$ is the number $q_i$ such that $Li{L}^{\prime }$.

  回报$r_i$的效用，$u(r_i)$，使得$Li{L}^{\prime }$的数$q_i$

  • $L=(1,r_1)$

    $L^{\prime }=(q_1,mostfavorableoutcome;1-q_1,leastfavorableoutcome)$

  • u(least favorable outcome)=0

    u(most favorable outcome)=1

• Utility function, $u(r_i),\forall r_i$

• expected utility of the lottery L
  $$E(UforL)=\sum _{i=1}^n{p}_{i}u(r_i)$$

Axiom

1. Complete ordering axiom: define the most/least favorable outcome
   $$\begin{gathered} \forall r_1,r_2:r_1\succ r_2,r_1\sim r_2,r_1\prec r_2 \\ {r}_1\succ r_2,r_2\succ r_3,then{r}_1\succ r_3(transitivityofpreferences) \end{gathered}$$

2. Continuity axiom: evaluate the intermediate outcome values between the most/least favorable outcome
   $$\begin{gathered} L_1=(1,r_2),L_2=(c,r_1;1-c,r_3) \\ If{r}_1\succ r_2,r_2\succ r_3 \\ then\exists c(0<c<1),suchthat{L}_{1}i{L}_2 \end{gathered}$$

3. Independence axiom: “plug in” the equivalent lottery of the intermediate outcomes
   $$\begin{gathered} L_1=(c,r_1;1-c,r_3),L_2=(c,r_2;1-c,r_3) \\ {r}_1\sim r_2,任意r_3 \\ \forall c(0<c<1),L_{1}i{L}_2 \end{gathered}$$

4. Unequal probability axiom: rank the lotteries
   $$\begin{gathered} L_1=(p_1,r_1;1-p_1,r_2),L_2=(p_2,r_1;1-p_2,r_2) \\ {r}_1\succ r_2,p_1>p_2\to L_{1}p{L}_2 \end{gathered}$$

5. Compound lottery axiom: convert a compound lottery into a simple lottery

   If we consider all the possible rewards, a compound lottery $L$ has a probability $p_i$ of receiving a reward $r_i$ , then $Li{L}^{\prime }$, where $L^{\prime }=(p_1,r_1;p_2,r_2;...;p_n,r_n)$

6. lemma 1：线性

   给定 utility function $u(x)$, 定义 positive linear function $v(x)=au(x)+b,\forall a>0,b$,

   对于两个lotteries $L_1$ 和 $L_2$
   $$\begin{gathered} L_{1}p{L}_{2}usingu(x)\leftrightarrow L_{1}p{L}_{2}usingv(x) \\ {L}_{1}i{L}_{2}usingu(x)\leftrightarrow L_{1}i{L}_{2}usingv(x) \end{gathered}$$

definitions

• expected value of L’s outcomes, EV(L)
  $$EV(L)=\sum _{i=1}^n{p}_i{r}_i$$

• expected utility of L, E(U for L)
  $$E(UforL)=\sum _{i=1}^n{p}_{i}u(r_i)$$

• certainty equivalent of L, CE(L)

  is the number such that a decision maker is indifferent between L and receiving a certain payoff of CE(L)
  $$E(UforL)=u(CE(L))$$

• risk premium of L, RP(L)
  $$RP(L)=EV(L)-CE(L)$$

• risk attitudes

  risk-averse	$RP(L)>0$	u(x) is strictly concave 严格凹
  risk-neutral	$RP(L)=0$	u(x) is linear
  risk-seeking	$RP(L)<0$	u(x) is strictly convex 严格凸

• graphic illustration

  $L=(p,x_1;1-p,x_2),x_1<x_2$

  [外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-D9PkJiCE-1652706216498)(C:\Users\FJB\AppData\Roaming\Typora\typora-user-images\1652692715154.png)]

Exercise

• value of insurance (Winston 2004, p.p. 85-86)

  • cash: $10,000
  • Home: $90,000
  • accident probability: 0.1%

• 问题：如果房子被毁，需要支付多少保险？

• 假设：$u(x)=x^{1/2}$, x表示整体财富（现金加房子）

• solution
  KaTeX parse error: Can't use function '$' in math mode at position 28: …ned} L_1 & =(1，$̲100,000-y)，L_2=…

---

---

---

example

1.

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-wVakE5na-1652706216500)(C:\Users\FJB\AppData\Roaming\Typora\typora-user-images\1652604680187.png)]

---

• $5000

• $u(x)=\sqrt{x}/1,000$

  investment 1	investment 2
  +$295,000, 80%; +$95,000, 20%	+$595,000, 50% +$5,000, 50%

| [外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-t3lT6mIi-1652706216501)(C:\Users\FJB\AppData\Roaming\Typora\typora-user-images\1652694074064.png)] | [外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-PQDy2KJS-1652706216501)(C:\Users\FJB\AppData\Roaming\Typora\typora-user-images\1652694089566.png)] |
| $EV(L_1)=80%*$300,000+20% *$100,000= $260,000$ | $EV(L_2)=$305,000$ |
| $E(Ufor{L}_1)=80\%\ast 0.55+20\%\ast 0.32=0.504$ | $E(Ufor{L}_2)=0.437$ |
| $CE(L_1)=u^{{-1}(E(UforL_1))=(0.504*1,000)}2=$254,016$ | $CE(L_2)=$190,969$ |
| $RP(L_1)=EV(L_1)-CE(L_1)=$5,984>0$ | $RP(L_2)=$114,031$ |
| risk-averse | risk-averse |

We are risk-averse.

• $CE(L_1)>CE(L_2)$

  We prefer investment 1.

2.

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-RM8aPoCb-1652706216502)(C:\Users\FJB\AppData\Roaming\Typora\typora-user-images\1652604699537.png)]

• risk-neutral, so $RP(L)=0,EV(L)=CE(L),u(x)$ is linear.

  $P(thefirstheadisobtainedonthenthtossofthecoin)=\frac{1}{2^n}$

$$\begin{array}{rl}{p}_i& =\frac{1}{2^i},r_i=2^i\\ EV(L)& =\sum _{i=1}^n{p}_i{r}_i=\frac{1}{2}\ast 2+\frac{1}{2^2}\ast 2^2+...+\frac{1}{2^n}\ast 2^n=n\\ CE(L)& =EV(L)=n\\ n& \to \infty ,CE(L)\to \infty \\ \because & u(x)islinear\\ \therefore & u(x)\to \infty ,E(UforL)\to \infty \end{array}$$

​ This is unreasonable.

• $u(x)={\log }_2(x)$
  $$\begin{array}{rl}u(r_i)& ={\log }_2(r_i)={\log }_2(2^i)=i\\ E(UforL)& =\sum _{i=1}^n{p}_{i}u(r_i)=\sum _{i=1}^n\frac{1}{2^i}i=2-\frac{1}{2^{n-1}}-\frac{n}{2^n}\\ CE(L)& =u^{-1}(E(UforL))=2^{2-\frac{1}{2^{n-1}}-\frac{n}{2^n}}\end{array}$$
  ble.

• $u(x)={\log }_2(x)$
  $$\begin{array}{rl}u(r_i)& ={\log }_2(r_i)={\log }_2(2^i)=i\\ E(UforL)& =\sum _{i=1}^n{p}_{i}u(r_i)=\sum _{i=1}^n\frac{1}{2^i}i=2-\frac{1}{2^{n-1}}-\frac{n}{2^n}\\ CE(L)& =u^{-1}(E(UforL))=2^{2-\frac{1}{2^{n-1}}-\frac{n}{2^n}}\end{array}$$