CSE 312 – Section 5

Review of Main Concepts

• Independence: Random variable \(X\) and event \(E\) are independent iff \[\forall x ,\quad \Pr ( X = x \cap E) = \Pr ( X = x)\Pr (E)\]

  Random variables \(X\) and \(Y\) are independent iff \[\forall x \forall y,\quad \Pr ( X = x \cap Y = y) = \Pr ( X = x)\Pr (Y = y)\] In this case, we have \(\Exp [XY]= \Exp [X]\Exp [Y]\) (the converse is not necessarily true).

• i.i.d. (independent and identically distributed): Random variables \(X_{1},\ldots ,X_{n}\) are i.i.d. (or iid) iff they are mutually independent and have the same probability mass function.
• Independence of functions of a r.v.: If \(X\) and \(Y\) are independent and \(g(\cdot ),h(\cdot )\) are functions mapping real numbers to real numbers, then \(g(X)\) and \(h(Y)\) are independent. (See if you can prove this!) Thus, for example: if \(X\) and \(Y\) are independent, then so are \(X^2\) and \(Y^2\).
• Variance of Independent Variables: If \(X\) and \(Y\) are independent random variables, \(\Var \left ( X + Y \right ) = \Var \left ( X \right ) + \Var (Y)\). Linearity of variance does not hold in general. It depends on independence, whereas linearity of expectation always holds. Note that this combined with the above shows that \(\forall a,b,c\ \mathbb {\in R}\) and if \(X\) is independent of \(Y\), \(\Var (aX + bY + c) =a^{2}\Var (X) + b^{2}\Var (Y)\).

• Review: Zoo of Discrete Random Variables

  a)

  Uniform: \(X\sim \textsf {Uniform}(a,b)\) (\(\textsf {Unif}(a,b)\) for short), for integers \(a \leq b\), iff \(X\) has the following probability mass function: \[p_{X}\left ( k \right ) = \frac {1}{b - a + 1},\ \ k = a, a+1, \ldots ,b\] \(\Exp [X]= \frac {a + b}{2}\) and \(\Var (X) = \frac {\left ( b - a \right )\left ( b - a + 2 \right )}{12}\). This represents a random experiment in which each outcome from \(\lbrack a,b\rbrack \) is equally likely. For example, a single roll of a fair die is \(\textsf {Uniform}(1,6)\).

  b)

  Bernoulli (or indicator): \(X\sim \textsf {Bernoulli}(p)\) (\(\textsf {Ber}(p)\) for short) iff \(X\) has the following probability mass function: \[p_{X}\left ( k \right ) = \left \{ \begin {matrix} p,\ \ & k = 1 \\ 1 - p,\ \ & k = 0 \\ \end {matrix} \right .\ \] \(\Exp [ X]= p\) and \(\Var (X) = p(1 - p)\). An example of a Bernoulli r.v. is one flip of a coin with \(\Pr \left ( \text {head} \right ) = p\).

  c)

  Binomial: \(X\sim \textsf {Binomial}(n,p)\) (\(\textsf {Bin}(n,p)\) for short) iff \(X\) is the sum of \(n\) iid \(\textsf {Bernoulli}(p)\) random variables. \(X\) has probability mass function \[p_{X}\left ( k \right ) = \binom {n}{k} p^{k}\left ( 1 - p \right )^{n - k},\ \ k = 0,1,\ldots ,n\] \(\Exp [X]= np\) and \(\Var (X) = np(1 - p)\). An example of a Binomial r.v. is the number of heads in \(n\) independent flips of a coin with \(\Pr \left ( \text {head} \right ) = p\). Note that \(\textsf {Bin}(1,p) \equiv \textsf {Ber}(p)\). As \(n \rightarrow \infty \ \text {and}\ p \rightarrow 0,\text {with}\ np\ = \lambda \), then \(\textsf {Bin}\left ( n,p \right ) \rightarrow \textsf {Poi}(\lambda )\). If \(X_{1},\ldots ,X_{n}\) are independent Binomial r.v.’s, where \(X_{i}\sim \textsf {Bin}(N_{i},p)\), then \(X = X_{1} + \ldots + X_{n}\sim \textsf {Bin}(N_{1} + \ldots + N_{n},p)\).

  d)

  Geometric: \(X\sim \textsf {Geometric}(p)\) (\(\textsf {Geo}(p)\) for short) iff \(X\) has the following probability mass function: \[p_{X}\left ( k \right ) = \left ( 1 - p \right )^{k - 1}p,\ \ k = 1,2,\ldots \] \(\Exp [X] = \frac {1}{p}\) and \(\Var (X) = \frac {1 - p}{p^{2}}\). An example of a Geometric r.v. is the number of independent coin flips up to and including the first head, where \(\Pr \left ( \text {head} \right ) = p\).

  e)

  Poisson: \(X\sim \textsf {Poisson}(\lambda )\) (\(\textsf {Poi}(\lambda )\) for short) iff \(X\) has the following probability mass function: \[p_{X}\left ( k \right ) = e^{- \lambda }\frac {\lambda ^{k}}{k!},\ \ k = 0,1,\ldots \] \(\Exp [X] = \lambda \) and \(\Var (X) = \lambda \). An example of a Poisson r.v. is the number of people born during a particular minute, where \(\lambda \) is the average birth rate per minute. If \(X_{1},\ldots ,X_{n}\) are independent Poisson r.v.’s, where \(X_{i}\sim \textsf {Poi}(\lambda _{i})\), then \(X = X_{1} + \ldots + X_{n}\sim \textsf {Poi}(\lambda _{1} + \ldots + \lambda _{n})\).

Announcements & Plan for Section

Announcements

• PSet 3 grades were released and can be viewed on Gradescope. Regrade requests will close on 5/2. We highly recommend taking a look through feedback on the psets, the common errors on Ed and the solutions that were posted (link on Ed).
• PSet 4 was due earlier this week.
• PSet 5 is released — start early.
• This week’s focus: Random Variable Zoo + applications, independence and more practice with expectation/variance.
• A practice midterm will be posted today or tomorrow. Please attempt the problems before next week’s section where we will review the solutions.

Plan for Section

• Content Review (Problem 1)
  Quick review of concepts such as variance properties, linearity of expectation and independence.
• Problem 2 (Pond Fishing)
  Build intuition for the different zoo variables and what each of them represent.
• Problem 4 (Variance of a Product)
  Gives students practice with the definition of variance, and understanding when linearity of variance holds.
• Problem 5 (True or False?)
  Exam-style true or false questions to strengthen the definitions of zoo variables.
• Problem 3 (Best Coach Ever!! - if time)
  Gives students intuition for when to apply which zoo variable, practices geometric and binomial variables.

1  Content Review Questions

a)

True or false: \(\Var (X + Y) = \Var (X) + \Var (Y)\) for all random variables \(X\) and \(Y\)

b)

What is \(\Var (3X + 4)\)?

• \(3\Var (X) + 4\)
• \(3\Var (X)\)
• \(9\Var (X)\)
• \(\Var (X)\)

c)

True or false: \(\Exp [X + Y] = \Exp [X] + \Exp [Y]\) for all random variables \(X\) and \(Y\).

d)

What is \(\Exp [3X + 4]\)?

• \(3\Exp [X] + 4\)
• \(3\Exp [X]\)
• \(9\Exp [X]\)
• \(\Exp [X]\)

2  Pond Fishing

Suppose I am fishing in a pond with \(B\) blue fish, \(R\) red fish, and \(G\) green fish, where \(B + R + G = N\). Each fish is equally likely to be caught. For each of the following scenarios, identify the most appropriate distribution (with parameter(s)):

a)

How many of the next 10 fish I catch are blue, if I catch and release them

• \[\textsf {Bin}\left ( 10,\frac {B}{N} \right )\]
• \[\textsf {Ber}\left ( \frac {B}{N} \right )\]
• \[\textsf {Bin}\left ( 1,\frac {B}{N} \right )\]

b)

How many fish I had to catch until my first green fish, if I catch and release them

• \[\textsf {Ber}\left ( \frac {G}{N} \right )\]
• \[\textsf {Bin}\left ( 1,\frac {G}{N} \right )\]
• \[\textsf {Geo}\left ( \frac {G}{N} \right )\]

c)

How many red fish I catch in the next five minutes, if I catch on average \(r\) red fish per minute

• \[\textsf {Poi}(5R)\]
• \[\textsf {Bin}\left ( 5,\frac {R}{N} \right )\]
• \[\textsf {Poi}(5r)\]

d)

Whether or not my next fish is blue

• \[\textsf {Poi}(5B)\]
• \[\textsf {Bin}\left ( 1,\frac {R}{N} \right )\]
• \[\textsf {Ber}\left ( \frac {B}{N} \right )\]

3  Best Coach Ever!!

You are a hardworking boxer. Your coach tells you that the probability of your winning a boxing match is \(0.2\) independently of every other match.

a)

How many matches do you expect to fight until you win 10 times and what kind of random variable is this?

b)

You only get to play 12 matches every year. To win a spot in the Annual Boxing Championship, a boxer needs to win at least 10 matches in a year. What is the probability that you will go to the Championship this year and what kind of random variable is the number of matches you win out of the 12?

c)

Let \(p\) be your answer to part (b). How many times can you expect to go to the Championship in your 20 year career?

4  Variance of a Product

Let \(X, Y, Z\) be independent random variables with means \(\mu _{X}, \mu _{Y}, \mu _{Z}\) and variances \(\sigma _{X}^{2}, \sigma _{Y}^{2}, \sigma _{Z}^{2}\), respectively. Find \(\Var (XY - Z)\).

5  True or False?

Identify the following statements as true or false (true means always true). Justify your answer.

a)

For any random variable \(X\), we have \(\Exp [X^2]\ge \Exp [X]^2\).

b)

Let \(X,Y\) be random variables. Then, \(X\) and \(Y\) are independent if and only if \(\Exp [XY]=\Exp [X]\Exp [Y]\).

c)

Let \(X\sim \textsf {Binomial}(n,p)\) and \(Y\sim \textsf {Binomial}(m,p)\) be independent. Then, \(X+Y\sim \textsf {Binomial}(n+m,p)\).

d)

Let \(X_1,...,X_{n+1}\) be independent \(\textsf {Bernoulli}(p)\) random variables. Then, \(\Exp [\sum _{i=1}^n{X_iX_{i+1}}]=np^2\).

e)

Let \(X_1,...,X_{n+1}\) be independent \(\textsf {Bernoulli}(p)\) random variables. Then, \(Y=\sum _{i=1}^n{X_iX_{i+1}}\sim \textsf {Binomial}(n,p^2)\).

f)

If \(X\sim \textsf {Bernoulli}(p)\), then \(nX\sim \textsf {Binomial}(n,p)\).

g)

If \(X\sim \textsf {Binomial}(n,p)\), then \(\frac {X}{n}\sim \textsf {Bernoulli}(p)\).

h)

For any two independent random variables \(X,Y\), we have \(\Var (X-Y)=\Var (X)-\Var (Y)\).

6  Fun with Poissons

Let \(X \sim \textsf {Poisson}(\lambda _1)\) and \(Y\sim \textsf {Poisson}(\lambda _2)\), and \(X\) and \(Y\) are independent.

a)

Show that \(X+Y \sim \textsf {Poisson}(\lambda _1 + \lambda _2)\). (We did this problem in class.)

b)

Show that \(\Pr (X = k \mid X+Y = n) = \Pr (W = k)\) where \(W \sim \textsf {Bin}(n, \frac {\lambda _1}{\lambda _1 + \lambda _2})\)

7  Memorylessness

We say that a random variable \(X\) is memoryless if \(\Pr (X > k + i \mid X > k) = \Pr (X > i)\) for all non-negative integers \(k\) and \( i\). The idea is that \(X\) does not remember its history. Let \(X\sim \textsf {Geo}(p)\). Show that \(X\) is memoryless.

8  Poisson Practice

Seattle averages 3 days with snowfall per year. Suppose the number of days with snowfall follows a Poisson distribution.

a)

What is the probability of getting exactly 5 days of snow in a year?

b)

According to the Poisson model, what is the probability of getting 367 days of snow?

9  How many 6’s?

Suppose that a fair 8-sided die is rolled repeatedly, with each roll independent of the others. Let \(Z\) be the number of rolls until (and including) the first time either a 2 or a 3 is rolled, and let \(W\) be the number of 6’s rolled until the first 2 or 3 is rolled. So, for example if the sequence of die values until the first 2 or 3 is 6,5,4,8,7,6,7,1,2, then \(Z\) is 9 and \(W\) is 2.

Define \[p(j) := \begin {cases} \Pr (W = j \mid Z = i)& j \in \{0, 1, \ldots , i-1\}\\ 0 & \text {otherwise} \end {cases}\] Show that \(p(j)\) is the probability mass function of a binomially distributed random variable and determine its parameters \(n\) and \(p\). Prove that your answer is correct mathematically.

10  Practice with LOTUS

Suppose that \(X\) is a Binomial random variable with parameters \(n\) and \(p\). Use LOTUS to show that \[\Exp \left [\frac {1}{X+1}\right ] = \frac {1- (1-p)^{n+1}}{(n+1)p}.\] Hint: Use the result of Problem 2 from Section 2, namely that \(\frac {1}{k+1} \binom {n}{k} = \frac {1}{n+1}\binom {n+1}{k+1} \) and then manipulate the expression until you can use the Binomial Theorem.

11  Parallel Server Failures

A computing system relies on \(m\) independent nodes. The lifetime (in days) of each node is modeled by a Geometric distribution with parameter \(p\) (meaning each node has a probability \(p\) of failing on any given day). The system completely shuts down only when all \(m\) nodes have failed.

a)

Let \(D\) be the day the first node fails. Find the probability mass function of \(D\). Start by computing the probability that \(D > d\).

b)

Find the exact probability that the entire system shuts down on day \(k\).

12  Grading time

Suppose that homeworks are graded by TA1 with probability 0.3, by TA2 with probability 0.5 and by TA3 with probability 0.2.

a)

TA1 takes an amount of time (in hours) to grade that is Poisson with parameter \(\lambda \).

b)

TA2 takes an amount of time (in hours) to grade that is Binomial with parameters \(n\) and \(p\).

c)

TA3 takes an amount of time (in hours) to grade that is Geometric with parameter \(p\).

What is the probability that TA1 did the grading given that the number of hours it took was \(h\)?