A Probabilistic Perspective-chapter2-sol

2.1.1.
$$\begin{gathered} A=onechildisboy \\ |A|=3 \\ P=\frac{2}{3} \end{gathered}$$
2.1.2.
First child is boy.
$$P=\frac{1}{2}$$

2.3:
$$\begin{array}{r}var[x+y]=E[(x+y{)}^2]-E^2[x+y]\\ var[x+y]=E[x^2]+E[y^2]+2E[xy]-(E[x]+E[y]{)}^2\\ var[x+y]=var[x]+var[y]+2(E[xy]-E[x]E[y])=var[x]+var[y]+2conv(x,y)\end{array}$$
2.4
$$P(ill|positive)=0.99,P(ill)=1e-4$$
the answer is 0.99.
text example:
$$p(positive|ill)=0.8,p(ill)=0.004,p(positive|notill)=0.1$$
$$p(positive)=p(positive|ill)p(ill)+p(positive|notill)p(notill)=0.8\ast 0.004+0.1\ast (1-0.004)=0.1028$$
$$p(ill|positive)=\frac{p(positive|ill)p(ill)}{p(positive)}=\frac{0.8\ast 0.004}{0.1028}=0.031$$

2.5
A=prize behind first picked door
B=prize behind final picked door
$P(A)=1/3,P(A)=2/3$
$$P(B)=P(B|A)P(A)+P(B|A)P(A)=0\ast 1/3+1\ast 2/3=2/3$$

2.6
1.
$$P(H|e_1,e_2)=\frac{P(e_1,e_2|H)P(H)}{P(e_1,e_2)}$$
answer is ii.

$$P(e_1,e_2|H)=P(e_1|H)P(e_2|H)$$
i, ii is sufficient.
$$P(e_1,e_2)=\sum _{H}P(H)P(e_1|H)P(e_2|H)$$
iii is sufficient.

2.7
wikipedia
$x=U(0,1),y=U(0,1),z=xxory$

2.8
$$x\perp y|z\to p(x,y|z)=h(x,y)g(y,z)$$
It is trival that $h(x,z)=p(x|z),g(y,z)=p(y|z)$.
vice versa,
$$\begin{array}{rl}p(x|z)& =\sum _{y}p(x,y|z)\\ & =g(x,z)\sum _{y}h(y,z)\\ p(y|z)& =\sum _{y}p(x,y|z)\\ & =h(y,z)\sum _{x}g(x,z)\\ 1& =p(x,y|z)\\ & =\sum _{x,y}h(x,z)g(y,z)\\ & =\sum _{x}h(x,z)\sum _{y}g(y,z)\\ then,& \\ p(x|z)p(y|z)& =g(x,z)h(y,z)\sum _{x}h(x,z)\sum _{y}g(y,z)\\ & =g(x,z)h(y,z)\end{array}$$

2.9
(i) true
(ii)false

2.10
$$\begin{array}{rl}p(y)& =p(x)\frac{dy}{dx}\\ \frac{dy}{dx}& =-\frac{1}{x^2}\\ IG(x|a,b)=\frac{b^a}{\Gamma (a)}{x}^{-(a+1)}{e}^{-\frac{b}{x}}& \end{array}$$

2.11
Intergral $\theta$ first,
$$\begin{array}{rl}{Z}^2& ={\int }_0^{2\pi }d\theta {\int }_{-\infty }^{\infty }r\exp (-\frac{r^2}{2{\sigma }^2})dr\\ & =2\pi {\int }_0^{\infty }r\exp (-\frac{r^2}{2{\sigma }^2})dr\end{array}$$

KaTeX parse error: Expected 'EOF', got '\end' at position 168: …frac{\sigma^2} \̲e̲n̲d̲{align}
So, $Z^2=2\pi {\sigma }^2$,then $Z=\sigma \sqrt{2\pi }$

2.12
$$\begin{array}{rl}I(X,Y)=& \sum _{x,y}p(x,y)\log \frac{p(x,y)}{p(x)p(y)}\\ =& \sum _{x,y}p(x,y)\log \frac{p(x|y)}{p(x)}\\ =& \sum _{x,y}p(x,y)(\log px|y-\log p(x)\\ =& -H(x|y)-\sum _x\log p(x)(\sum _{y}p(x,y))\\ =& -H(x|y)+H(x)\end{array}$$

2.13
$$\begin{array}{rl}I(X,Y)=& H(x)-H(x|y)\\ =& H(x)+H(y)-H(x,y)\\ =& \log 2\pi e{\sigma }^2+\frac{1}{2}\log (2\pi e{)}^2{\sigma }^4(1-{\rho }^2)\end{array}$$
For $\rho =0$,
$$I(x,y)=\log 2\pi e{\sigma }^2+\frac{1}{2}\log (2\pi e{)}^2{\sigma }^4=2\log 2\pi e{\sigma }^2=H(x)+H(y)$$
When $Cov(x,y)=0$, mutual information is simply sum of single information of two variables, which knowing $x$ does not give any information about $y$ and vice versa.
For $\rho =\pm 1$,
$$I(x,y)=\infty$$
All information conveyed by $x$ is shared with $y$: knowing $x$ determines the value of $y$ and vice versa.

2.14
(i)
obvious.
(ii)
It is easy to prove non negativity of entropy $H(x)>0$.
$I(x,y)\ge 0\to r\ge 0$
$I(x,y)\ge 0$ is obvious due to its formula.
(iii)
$I(x,y)=0$, x,y are independent
(iiii)
$I(x,y)=1$, x is fully dependent of y.

2.15
$$\begin{array}{rl}\theta =& \underset{\theta }{\mathrm{argmin}}KL(P_{emp}||q(\theta ))\\ =& \underset{\theta }{\mathrm{argmin}}E(P_{emp}\log \frac{P_{emp}}{q(\theta )})\\ =& \underset{\theta }{\mathrm{argmin}}E(P_{emp}(\log P_{emp}-\log q(\theta )))\\ =& H_{emp}-\underset{\theta }{\mathrm{argmax}}E(P_{emp}\log q(\theta ))\\ =& \underset{\theta }{\mathrm{argmax}}E(P_{emp}\log q(\theta ))\\ =& \underset{\theta }{\mathrm{argmax}}\sum _{x\in Dataset}\log q(x;\theta )\end{array}$$

2.16
pdf of beta distribution:
$$\frac{x^{\alpha -1}(1-x{)}^{\beta -1}}{B(\alpha ,\beta )}$$
mode:
$$\begin{array}{rl}\frac{d}{dx}\frac{x^{\alpha -1}(1-x{)}^{\beta -1}}{B(\alpha ,\beta )}& =0\\ x& =\frac{\alpha -1}{\alpha +\beta -2}\end{array}$$
$$\begin{array}{rl}E(x^N)& =\frac{1}{B(\alpha ,\beta )}\int x^{\alpha +N-1}(1-x{)}^{\beta -1}\\ & =\frac{B(\alpha +N,\beta )}{B(\alpha ,\beta )}\end{array}$$
mean:
$$\begin{array}{rl}E(x)& =\frac{1}{B(\alpha ,\beta )}\int x^{\alpha +N-1}(1-x{)}^{\beta -1}\\ & =\frac{a}{a+b}\end{array}$$
var:
$$E(x^2)-E^2(x)=\frac{ab}{(a+b{)}^2(a+b+a)}-\frac{a^2}{(a+b{)}^2}$$

2.17
The leftest point’s coordinate $f(x,y)=min(x,y)$,
$$\begin{array}{rl}p(f(x,y)=m)=& p(x=m,y>=m)+p(x>=m,y=m)\\ =& 2(1-m)\\ E(m)=& {\int }_0^{1}2m(1-m)dm\\ & ={\int }_0^{1}2m-2m^{2}dm\\ & ={m^2-\frac{2}{3}{m}^3|}_0^1\\ & =\frac{1}{3}\end{array}$$
The problem can also be solved in 3-d coordinates. The body is a cone with height 1 and bottom area 1.