CSE 312 – Section 8

Review of Main Concepts

• Multivariate: Discrete to Continuous:

  	Discrete	Continuous
  Joint PMF/PDF	\( p_{X,Y}(x,y) = \Pr (X = x,Y=y)\)	\( f_{X,Y}(x,y) \neq \Pr (X = x,Y=y)\)
  Joint range/support
  \(\Omega _{X,Y}\)	\(\{(x,y)\in \Omega _X\times \Omega _Y: p_{X,Y}(x,y)>0\}\)	\(\{(x,y)\in \Omega _X\times \Omega _Y: f_{X,Y}(x,y)>0\}\)
  Joint CDF	\(F_{X,Y}\left ( x,y \right ) = \sum _{t \leq x,s\le y}{p_{X,Y}(t,s)}\)	\(F_{X,Y}\left ( x,y \right ) = \int _{- \infty }^{x}{\int _{-\infty }^y{f_{X,Y}\left ( t,s \right )dsdt}}\)
  Normalization	\(\sum _{x,y}^{}{p_{X,Y}(x,y)} = 1\)	\(\int _{- \infty }^{\infty }{\int _{-\infty }^\infty {f_{X,Y}\left ( x,y \right )dxdy}} = 1\)
  Marginal PMF/PDF	\(p_X(x)=\sum _y{p_{X,Y}(x,y)}\)	\(f_X(x)=\int _{-\infty }^\infty {f_{X,Y}(x,y)dy}\)
  Expectation	\(\expect {g(X,Y)} = \sum _{x,y}{g(x,y)p_{X,Y}(x,y)}\)	\(\expect {g(X,Y)}= \int _{- \infty }^{\infty }{\int _{-\infty }^{\infty }{g(x,y)f_{X,Y}( x,y)dxdy}}\)
  Independence	\(\forall x,y, p_{X,Y}(x,y)=p_X(x)p_Y(y)\)	\(\forall x,y, f_{X,Y}(x,y)=f_X(x)f_Y(y)\)
  must have	\(\Omega _{X,Y}=\Omega _X\times \Omega _Y\)	\(\Omega _{X,Y}=\Omega _X\times \Omega _Y\)

• Law of Total Probability (r.v. version): If \(X\) is a discrete random variable, then \[\Pr (A)=\sum _{x\in \Omega _X}{\Pr (A|X=x)p_X(x)}\quad \quad \text {discrete $X$}\]

• Continuous Law of Total Probability: \[\Pr (A)=\int _{x\in \Omega _X}{\Pr (A|X=x)f_X(x)dx}\]

  There will be problems covering the following concepts (some of which have not yet been discussed in lecture) on the Section 9 worksheet:

• Conditional expectation: The expected value of random variable \(X\) given that event \(A\) has occurred, written \(\expect {X|A}\), is defined as \[\expect {X|A} = \sum _{x \in \Omega _X} x \cdot \Pr (X=x | A).\]
• Discrete Law of Total Expectation (event version): Let \(A_1 \ldots , A_n\) be a partition of the sample space. Then \[\expect {X}=\sum _{i=1}^n \expect {X|A_i} \Pr (A_i).\]
• Discrete Law of Total Expectation (r.v. version): Let \(X\) and \(Y\) be two random variables. Then \[\expect {X}=\sum _{y \in \Omega _Y} \expect {X|Y=y} \cdot \Pr (Y=y).\]
• Continuous Law of Total Expectation: \[\expect {X}=\int _{y \in \Omega _Y}{\expect {X|Y=y}f_Y(y)dy}\]
• Expected value of \(X\) conditioned on r.v. \(Y\): Suppose that \(Y\) is a random variable that takes values \(y_1, \ldots , y_k\). Then \(\expect {X|Y}\) is the following random variable \[\expect {X|Y} = \begin {cases} \expect {X|Y=y_1} & \text {with probability }\Pr (Y=y_1)\\ \expect {X|Y=y_2} & \text {with probability }\Pr (Y=y_2)\\ \ldots & \\ \expect {X|Y=y_k} & \text {with probability }\Pr (Y=y_k)\\ \end {cases}\]
• Law of total expectation (rewritten): Given the above definition, we can write \[\expect {X} = \expect {\expect {X | Y}} = \sum _{i=1}^k \expect {X|Y=y_i}\cdot \Pr (Y=y_i).\]
• Covariance: We may not get to this in class, but there is a problem on the pset about it. To find out more, check out section 5.4 in the Tsun book. And now the definition: For any two random variables \(X,Y\) the covariance is defined as \[\Covariance {X,Y}=\expect {(X-\expect {X})(Y-\expect {Y})}.\] It can also be shown that \[\Covariance {X,Y}=\expect {XY}- \expect {X}\expect {Y}.\]

• Conditional distributions: We are not explicitly covering this topic in class, but it is highly recommended that you study it. Much of the above can be more appropriately rewritten in terms of conditional distributions. See Tsun, Section 5.3.

  	Discrete	Continuous
  Conditional PMF/PDF	\(p_{X|Y}(x|y)=\frac {p_{X,Y}(x,y)}{p_Y(y)}\)	\(f_{X|Y}(x|y)=\frac {f_{X,Y}(x,y)}{f_Y(y)}\)
  Conditional Expectation	\(\expect {X|Y=y}=\sum _{x}{xp_{X|Y}(x|y)}\)	\(\expect {X|Y=y}=\int _{-\infty }^\infty {xf_{X|Y}(x|y)dx}\)

Plan for Section

• Content Review (Problem 1)
• Joint PMF’s (Problem 2) - do it fast
• Continuous joint density - Problem 6
• 3 points on a line - Problem 7
• Min and max of i.i.d. random variables - Problem 8 if time permits

We recommend that students look at the final problem for examples of how to set up the ranges of integration. Might be helpful on the homework.

1  Content Review

a)

Select one: Given two discrete random variables \(X\) and \(Y\), the joint CDF is

• \(F_{X,Y}(x,y) = \sum _{t<x} p_{X,Y}(t,y)\)
• \(F_{X,Y}(x,y) = \sum _{s<y} p_{X,Y}(x,s)\)
• \(F_{X,Y}(x,y) = \sum _{t \le x}\sum _{s \le y} p_{X,Y}(t,s)\)
• \(F_{X,Y}(x,y) = p_{X,Y}(x,y)\)

b)

Marginal PDF. Let \(X\) and \(Y\) be continuous random variables with joint PDF \(f_{X,Y}(x,y)\). Which of the following correctly expresses the marginal PDF \(f_X(x)\)?

• \(\int _{-\infty }^\infty f_{X,Y}(x,y) \,dx\)
• \(\int _{-\infty }^\infty f_{X,Y}(x,y) \,dy\)
• \(\frac {f_{X,Y}(x,y)}{f_Y(y)}\)
• \(\int _{-\infty }^x \int _{-\infty }^y f_{X,Y}(t,s) \,ds\,dt\)

c)

Independence and Support. True or False: If the joint support \(\Omega _{X,Y}\) of the random variables \((X,Y)\) is a circle defined by \(x^2 + y^2 \le 1\), and \(\Omega _X=\Omega _Y = [0,1]\) then \(X\) and \(Y\) are independent.

• True
• False

d)

Continuous Law of Total Probability. Let \(A\) be an event and \(X\) be a continuous random variable with PDF \(f_X(x)\). Which of the following is the correct expression for the Continuous Law of Total Probability?

• \(\Pr (A) = \int _{-\infty }^\infty \Pr (A \mid X=x) \,dx\)
• \(\Pr (A) = \int _{-\infty }^\infty \Pr (A \cap X=x) f_X(x) \,dx\)
• \(\Pr (A) = \int _{-\infty }^\infty \Pr (X=x \mid A) \Pr (A) \,dx\)
• \(\Pr (A) = \int _{-\infty }^\infty \Pr (A \mid X=x) f_X(x) \,dx\)

2  Joint PMF’s

Suppose \(X\) and \(Y\) have the following joint PMF:

a)

Identify the range of \(X\) (\(\Omega _X\)), the range of \(Y\) (\(\Omega _Y\)), and their joint range (\(\Omega _{X,Y}\)).

b)

Find the marginal PMF for \(X\), \(p_X(x)\) for \(x\in \Omega _X\).

c)

Find the marginal PMF for \(Y\), \(p_Y(y)\) for \(y\in \Omega _Y\).

d)

Are \(X\) and \(Y\) independent? Why or why not?

e)

Find \(\expect {X^3Y}\).

3  Trinomial Distribution

A generalization of the Binomial model is when there is a sequence of \(n\) independent trials, but with three outcomes, where \(\Pr (\text {outcome }i)=p_i\) for \(i=1,2,3\) and of course \(p_1+p_2+p_3=1\). Let \(X_i\) be the number of times outcome \(i\) occurred for \(i=1,2,3\), where \(X_1+X_2+X_3=n\). Find the joint PMF \(p_{X_1,X_2,X_3}(x_1,x_2,x_3)\) and specify its value for all \(x_1,x_2,x_3\in \mathbb {R}\).

4  Do You “Urn” to Learn More About Probability?

Suppose that 3 balls are chosen without replacement from an urn consisting of 5 white and 8 red balls. Let \(X_i = 1\) if the \(i\)-th ball selected is white and let it be equal to 0 otherwise. Give the joint probability mass function of

a)

\(X_1, X_2\)

b)

\(X_1, X_2, X_3\)

5  Successes

Consider a sequence of independent Bernoulli trials, each of which is a success with probability \(p\). Let \(X_1\) be the number of failures preceding the first success, and let \(X_2\) be the number of failures between the first 2 successes. Find the joint pmf of \(X_1\) and \(X_2\). Write an expression for \(E[\sqrt {X_1 X_2}]\). You can leave your answer in the form of a sum.

6  Continuous joint density

The joint density of \(X\) and \(Y\) is given by \[f_{X,Y} (x,y) = \begin {cases} xe^{-(x+y)} & x > 0,y > 0\\ 0 & \text {otherwise}. \end {cases}\] and the joint density of \(W\) and \(V\) is given by \[f_{W,V} (w,v) = \begin {cases} 2 & 0 < w < v, 0 < v < 1\\ 0 & \text {otherwise}. \end {cases}\] Are \(X\) and \(Y\) independent? Are \(W\) and \(V\) independent?

7  3 points on a line

Three values \(X_1,X_2,X_3\) are selected uniformly at random, each between \(0\) and \(1\) (continuous independent uniform distributions). What is the probability that \(X_2\) is greater than \(X_1\) but less than \(X_3\)?

8  Min and max of i.i.d. random variables

Let \(X_1, X_2, \ldots , X_n\) be i.i.d. random variables each with CDF \(F_X(x)\) and pdf \(f_X(x)\). Let \(Y= \min (X_1, \ldots , X_n)\) and let \(Z = \max (X_1, \ldots , X_n)\). Show how to write the CDF and pdf of \(Y\) and \(Z\) in terms of the functions \(F_X(\cdot )\) and \(f_X(\cdot )\).

9  Law of Total Probability

a)

Suppose we flip a coin with probability \(U\) of heads, where \(U\) is equally likely to be one of \(\Omega _U=\{0,\frac {1}{n},\frac {2}{n},...,1\}\) (notice this set has size \(n+1\)). Let \(H\) be the event that the coin comes up heads. What is \(\Pr (H)\)?

b)

Now suppose \(U\sim \textsf {Uniform(0,1)}\) has the continuous uniform distribution over the interval \([0,1]\). Use the continuous law of total probability to handle this case.

c)

Suppose that \(X_1\) and \(X_2\) are independent continuous random variables. Find an expression for \(\Pr (X_{1} < {2X}_{2})\) using the law of total probability, in terms of \(F_{X_{1}},F_{X_{2}},f_{X_{1}},{f_{X}}_{2}\). (Your answer will be in the form of a single integral, and requires no calculations – do not evaluate it).

d)

Suppose \(X_{1}\sim \mathcal {N}(\mu _{1},\sigma _{1}^{2})\) and \(X_{2}\sim \mathcal {N}(\mu _{2},\sigma _{2}^{2})\). Find \(s\), where \(\Phi \left ( s \right ) = \Pr (X_{1} < 2X_{2})\) using the fact that linear combinations of independent normal random variables are still normal.

e)

Suppose \(Z = X+Y\), where \(X\) and \(Y\) are independent. \(Z\) is called the convolution of the two random variables. If \(X,Y,Z\) are discrete, using the law of total probability, we can write \[p_{Z}\left ( z \right ) = \Pr ( X+Y = z) = \sum _{x}^{}{\Pr (X = x \cap Y = z - x)} = \sum _{x}^{}{p_{X}\left ( x \right )p_{Y}(z - x)}\] Write an analogous expression for \(F_Z(z)\) in the case that \(X,Y,Z\) are continuous where, again, \(X\) and \(Y\) are independent.

10  Jointly distributed random variables involving 3 variables

a)

Validating a Joint Density. Let \(X, Y\), and \(Z\) be continuous random variables. To verify that \(f_{X,Y,Z}(x,y,z) = 6\) for the region \(0 \le x \le y \le z \le 1\) (and \(0\) otherwise) is a valid joint probability density function, which of the following equations must hold true?

• \(\int _{0}^{1} \int _{0}^{1} \int _{0}^{1} 6 \,dx \,dy \,dz = 1\)
• \(\int _{0}^{1} \int _{0}^{z} \int _{0}^{y} 6 \,dx \,dy \,dz = 1\)
• \(\int _{0}^{1} \int _{0}^{x} \int _{0}^{y} 6 \,dz \,dy \,dx = 1\)
• \(\int _{0}^{1} \int _{x}^{1} \int _{y}^{1} 6 \,dx \,dy \,dz = 1\)

b)

Integrating out a Variable. Let \(X, Y\), and \(Z\) be continuous random variables with joint PDF \(f_{X,Y,Z}(x,y,z) = 6\) for the region \(0 \le x \le y \le z \le 1\), and \(0\) otherwise. Which of the following correctly expresses the joint marginal PDF \(f_{X,Y}(x,y)\) for the valid region?

• \(\int _{0}^{1} 6 \,dz\)
• \(\int _{x}^{y} 6 \,dz\)
• \(\int _{0}^{y} 6 \,dz\)
• \(\int _{y}^{1} 6 \,dz\)

c)

The 3D Simplex. Let \(X, Y\), and \(Z\) be independent random variables, each uniformly distributed over \((0,1)\). Which of the following integrals correctly computes the probability that \(X + Y + Z \le 1\)?

• \(\int _{0}^{1} \int _{0}^{1} \int _{0}^{1} 1 \,dz \,dy \,dx\)
• \(\int _{0}^{1} \int _{0}^{1-x} \int _{0}^{1-x-y} 1 \,dz \,dy \,dx\)
• \(\int _{0}^{1} \int _{0}^{x} \int _{0}^{y} 1 \,dz \,dy \,dx\)
• \(\int _{0}^{1} \int _{0}^{1-x} \int _{0}^{1} 1 \,dz \,dy \,dx\)

d)

Bounding with Max. Let \(X, Y\), and \(Z\) be independent random variables, each uniformly distributed over \((0,1)\). Which of the following integrals correctly computes \(\Pr (X \ge \max (Y, Z))\)?

• \(\int _{0}^{1} \int _{x}^{1} \int _{x}^{1} 1 \,dz \,dy \,dx\)
• \(\int _{0}^{1} \int _{0}^{1} \int _{0}^{\max (y,z)} 1 \,dx \,dy \,dz\)
• \(\int _{0}^{1} \int _{0}^{x} \int _{0}^{x} 1 \,dz \,dy \,dx\)
• \(\int _{0}^{1} \int _{0}^{1} \int _{\min (y,z)}^{1} 1 \,dx \,dy \,dz\)

e)

Conditional PDF. (Not covered in class.) For two continuous random variables \(X\) and \(Y\), which of the following defines the conditional PDF \(f_{X|Y}(x|y)\)?

• \(\frac {f_{X,Y}(x,y)}{f_X(x)}\)
• \(\frac {f_{X,Y}(x,y)}{f_Y(y)}\)
• \(f_X(x)f_Y(y)\)
• \(\int _{-\infty }^\infty f_{X,Y}(x,y) \,dy\)