Chapter 3 (General Random Variables): Continuous Random Variables and PDFs (连续随机变量、概率密度函数)

Continuous Random Variables and PDFs

• A random variable $X$ is called continuous if there is a nonnegative function $f_X$, called the probability density function of $X$, or PDF for short, such that
  $$P(X\in B)={\int }_B{f}_X(x)dx$$for every subset $B$ of the realline. Note that to qualify as a PDF, a function $f_X$ must be nonnegative, and must also have the normalization property
  $${\int }_{-\infty }^{\infty }{f}_X(x)dx=P(-\infty <X<\infty )=1$$
• In particular, the probability that the value of $X$ falls within an interval is
  $$P(a\le X\le B)={\int }_a^b{f}_X(x)dx$$For any single value $a$, we have
  $$P(X=a)={\int }_a^a{f}_X(x)dx=0$$For this reason, including or excluding the endpoints of an interval has no effect on its probability:
  $$P(a\le X\le B)=P(a<X<B)=P(a\le X<B)=P(a<X\le B)$$
• To interpret the PDF, note that for an interval $[x,x+\delta ]$ with a very small length $\delta$, we have
  $$P([x,x+\delta ])={\int }_x^{x+\delta }{f}_X(t)dt\approx f_X(x)\cdot \delta$$so we can view $f_X(x)$ as the “probability mass per unit length” near $x$. It is important to realize that even though a PDF is used to calculate event probabilities, $f_X(x)$ is not the probability of any particular event. In particular, it is not restricted to be less than or equal to one.

---

Example 3.2. Piecewise Constant PDF. (逐段常数的 PDF)

Alvin’s driving time to work is between 15 and 20 minutes if the day is sunny, and between 20 and 25 minutes if the day is rainy, with all times being equally likely in each case. Assume that a day is sunny with probability $2/3$ and rainy with probability $1/3$. What is the PDF of the driving time, viewed as a random variable $X$?

SOLUTION

• PDF:
  
  where $c_1$ and $c_2$ are some constants. We can determine these constants by using the given probabilities of a sunny and of a rainy day:
  $$\begin{gathered} \frac{2}{3}=P(sunnyday)={\int }_{15}^{20}{f}_X(x)dx=5c_1,c_1=\frac{2}{15} \\ \frac{1}{3}=P(rainyday)={\int }_{20}^{25}{f}_X(x)dx=5c_2,c_2=\frac{1}{15} \end{gathered}$$

---

Example 3.3. A PDF Can Take Arbitrarily Large Values.

• Consider a random variable $X$ with PDF
  
• Even though $f_X(x)$ becomes infinitely large as $x$ approaches zero, this is still a valid PDF, because
  $${\int }_{-\infty }^{\infty }{f}_X(x)dx={\int }_0^1\frac{1}{2\sqrt{x}}dx=1$$

Expectation

• The expected value or expectation or mean of a continuous random variable $X$ is defined by
  $$E[X]={\int }_{-\infty }^{\infty }x{f}_X(x)dx$$
• If $X$ is a continuous random variable with given PDF. any real-valued function $Y=g(X)$ of $X$ is also a random variable.
  $$E[g(X)]={\int }_{-\infty }^{\infty }g(x)f_X(x)dx$$
• The variance of $X$ is defined by
  $$\begin{array}{rl}var(X)& =E[(X-E[X]{)}^2]={\int }_{-\infty }^{\infty }(x-E[X]{)}^2{f}_X(x)dx\\ & =E[X^2]-(E[X]{)}^2\end{array}$$
• If $Y=aX+b$, where $a$ and $b$ are given scalars, then
  $$\begin{gathered} E[Y]=aE[X]+b \\ var(Y)=a^{2}var(X) \end{gathered}$$

One has to deal with the possibility that the integral ${\int }_{-\infty }^{\infty }x{f}_X(x)dx$ is infinite or undefined. More concretely. we will say that the expectation is well-defined if ${\int }_{-\infty }^{\infty }|\mathbf{x}|{\mathbf{f}}_{\mathbf{X}}(\mathbf{x})\mathbf{d}\mathbf{x}<\infty$. In that case, it is known that the integral ${\int }_{-\infty }^{\infty }x{f}_X(x)dx$ takes a finite and unambiguous value. Throughout this book. in the absence of an indication to the contrary, we implicitly assume that the expected value of any random variable of interest is well-defined.

---

Example 3.4. Mean and Variance of the Uniform Random Variable. (均匀随机变量)

• We can consider a random variable $X$ that takes values in an interval $[a,b]$, and again assume that any two subintervals of the same length have the same probability. We refer to this type of random variable as uniform or uniformly distributed. Its PDF has the form
  
  $$\begin{gathered} E[X]={\int }_{-\infty }^{\infty }x{f}_X(x)dx={\int }_a^b\frac{x}{b-a}dx=\frac{a+b}{2} \\ E[X^2]={\int }_a^b\frac{x^2}{b-a}dx=\frac{a^2+ab+b^2}{3} \\ var(X)=E[X^2]-(E[X]{)}^2=\frac{(b-a{)}^2}{12} \end{gathered}$$

---

Problem 3

Show that the expected value of a discrete or continuous random variable $X$ satisfies
$$E[X]={\int }_0^{\infty }P(X>x)dx-{\int }_0^{\infty }P(X<-x)dx$$

SOLUTION

• Suppose that $X$ is continuous. We then have
  $$\begin{array}{rl}{\int }_0^{\infty }P(X>x)dx& ={\int }_0^{\infty }({\int }_x^{\infty }{f}_X(y)dy)dx\\ & ={\int }_0^{\infty }({\int }_0^y{f}_X(y)dx)dy\\ & ={\int }_0^{\infty }{f}_X(y)({\int }_0^{y}dx)dy\\ & ={\int }_0^{\infty }y{f}_X(y)dy\end{array}$$, where for the second equality we have reversed the order of integration by writing the set { ( x . y ) ∣ 0 ≤ x < ∞ , x ≤ y < ∞ } \{(x.y) | 0\leq x <\infty, x\leq y <\infty\} {(x.y)∣0≤x<∞,x≤y<∞} as { ( x . y ) ∣ 0 ≤ x ≤ y , 0 ≤ y < ∞ } \{(x.y) |0\leq x\leq y, 0\leq y <\infty\} {(x.y)∣0≤x≤y,0≤y<∞}. Similarly. we can show that
  $${\int }_0^{\infty }P(X<-x)dx=-{\int }_{-\infty }^{0}y{f}_X(y)dy$$Combining the two relations above, we obtain the desired result.
• If $X$ is discrete, we have
  $$\begin{array}{rl}P(X>x)& ={\int }_0^{\infty }(\sum _{y>x}{p}_X(y))dx\\ & =\sum _{y>0}({\int }_0^y{p}_X(y)dx)\\ & =\sum _{y>0}{p}_X(y)({\int }_0^{y}dx)\\ & =\sum _{y>0}{p}_X(y)y\end{array}$$and the rest of the argument is similar to the continuous case.

---

Problem 4.
Establish the validity of the expected value rule
$$E[g(X)]={\int }_{-\infty }^{\infty }g(x)f_X(x)dx$$where $X$ is a continuous random variable with PDF $f_X$.

SOLUTION

• Let us express the function $g$ as the difference of two nonnegative functions,
  $$g(x)=g^{+}(x)-g^{-}(x)$$where g + ( x ) = m a x { g ( x ) , 0 } g^+(x)= max\{g(x ),0\} g+(x)=max{g(x),0}, and g − ( x ) = m a x { − g ( x ) , 0 } g^-(x) = max\{-g( x ),0\} g−(x)=max{−g(x),0}. We will use the result
  $$E[g(X)]={\int }_0^{\infty }P(g(X)>x)dx-{\int }_0^{\infty }P(g(X)<-x)dx$$from the preceding problem. The first term in the right-hand side is equal to
  ∫ 0 ∞ ∫ { x ∣ g ( x ) > t } f X ( x ) d x d t = ∫ − ∞ ∞ ∫ { t ∣ 0 ≤ t < g ( x ) } f X ( x ) d t d x = ∫ − ∞ ∞ f X ( x ) g + ( x ) d x \int_0^\infty \int_{\{x|g(x)>t\}}f_X(x) dx dt = \int_{-\infty}^\infty\int_{\{t|0\leq t<g(x)\}}f_X(x)dtdx=\int_{-\infty}^\infty f_X(x)g^+(x)dx ∫0∞​∫{x∣g(x)>t}​fX​(x)dxdt=∫−∞∞​∫{t∣0≤t<g(x)}​fX​(x)dtdx=∫−∞∞​fX​(x)g+(x)dxBy a symmetrical argument, the second term in the right-hand side is given by
  $${\int }_{-\infty }^{\infty }{f}_X(x)g^{-}(x)dx$$
• Combining the above equalities, we obtain
  $$E[g(X)]={\int }_{-\infty }^{\infty }{f}_X(x)g^{+}(x)dx-{\int }_{-\infty }^{\infty }{f}_X(x)g^{-}(x)dx={\int }_{-\infty }^{\infty }{f}_X(x)g(x)dx$$

Exponential Random Variable (指数随机变量)

where $\lambda$ is a positive parameter characterizing the PDF.

• This is a legitimate PDF because
  $${\int }_{-\infty }^{\infty }{f}_X(x)dx={\int }_0^{\infty }\lambda e^{-\lambda x}dx=1$$
• Note that the probability that $X$ exceeds a certain value decreases exponentially. Indeed, for any $a\ge 0$, we have
  $$P(X\ge a)={\int }_a^{\infty }\lambda e^{-\lambda x}dx=e^{-\lambda a}$$($P(a\le X\le b)=P(X\ge a)-P(X\ge b)=e^{-\lambda a}-e^{-\lambda b}$)
  
• The mean and the variance can be calculated to be
  $$E[X]=\frac{1}{\lambda },var(X)=\frac{1}{{\lambda }^2}$$$$\begin{array}{rl}E[X]& ={\int }_0^{\infty }x\lambda e^{-\lambda x}dx\\ & =(-x{e}^{-\lambda x}){|}_0^{\infty }+{\int }_0^{\infty }{e}^{-\lambda x}dx\\ & =0-\frac{e^{-\lambda x}}{\lambda }{|}_0^{\infty }\\ & =\frac{1}{\lambda }\\ E[X^2]& ={\int }_0^{\infty }{x}^2\lambda e^{-\lambda x}dx\\ & =(-x^2{e}^{-\lambda x}){|}_0^{\infty }+{\int }_0^{\infty }2x{e}^{-\lambda x}dx\\ & =0+\frac{2}{\lambda }E[X]\\ & =\frac{2}{{\lambda }^2}\\ var(X)& =E[X^2]-(E[X]{)}^2=\frac{1}{{\lambda }^2}\end{array}$$

---

• An exponential random variable can, for example, be a good model for the amount of time until an incident of interest takes place. We will see that it is closely connected to the geometric random variable, which also relates to the (discrete) time that will elapse until an incident of interest takes place.

References

• $Introduction$ $to$ $Probability$