CSE390D Notes for Monday, 11/18/24
<![CDATA[
Bayes' Theorem:
extremely important
basis of lots of modern CS (machine learning, etc)
e.g., FareCast, spam filters
greatly misunderstood
built on conditional probability rule:
p(E | F) = p(E intersect F) / p(F)
for example, suppose we have studied a population from the perspective
of some event F:
F = has disease
we divide up the world into F and F' (Venn diagram)
then we experiment with some event E:
E = tests positive
in this case, we'd like E = F, but won't be true
false positives: E and F'
suppose we determine p(E | F') (rate of false positives)
what is the complement? p(E' | F') = true negatives
false negatives: E' and F
suppose we determine p(E' | F) (rate of false negatives)
what is the complement? p(E | F) (rate of true positives)
suppose you have E and want to know F
you're trying to reason backwards
Bayes' theorem:
p(F | E) = p(E | F) * p(F) / (p(E | F) * p(F) + p(E | F') * p(F'))
----------------------------------------------------------------------
Example: Suppose that a certain disease has a prevalance of 1 in 10
thousand and there is a test that is 99% effective for those
with the disease and 99.5% effective for those without the
disease. If you test positive, what is the probability that
you have the disease?
elements of Bayes' theorem:
p(E | F) rate of true positives 0.99
p(F) rate of disease 0.00001
p(E | F') rate of false positives 0.005
p(F') rate of non-disease 0.99999
p(F | E) = 0.99 * 0.00001 / (0.99 * 0.00001 + 0.005 * 0.99999)
~= .001976 (around 0.2%)
----------------------------------------------------------------------
Derivation of the formula:
notice 3 things:
p(F | E) = p(F intersect E) / p(E)
= p(E intersect F) / p(E)
p(E intersect F) = p(F | E) * p(E)
p(E | F) = p(E intersect F) / p(F)
p(E intersect F) = p(E | F) * p(F)
p(E | F') = p(E intersect F') / p(F')
p(E intersect F') = p(E | F') * p(F')
the first two are equal, so:
p(F | E) * p(E) = p(E | F) * p(F)
p(F | E) = p(E | F) * p(F) / p(E)
what about p(E)?
p(E) = p(E intersect S) = p(E intersect (F U F'))
= p(E intersect F) + p(E intersect F')
= p(E | F) * p(F) + p(E | F') * p(F')
plugging p(E) back in:
p(F | E) = p(E | F) * p(F) / (p(E | F) * p(F) + p(E | F') * p(F'))
which is Bayes' theorem
----------------------------------------------------------------------
Expected Value of a Random Variable
E(X) = sum(for s in S, p(s) * X(s))
e.g., roll a die once, X = roll
E(X) = 1/6 * (1 + 2 + 3 + 4 + 5 + 6) = 21/6 = 7/2 = 3.5
how about rolling die twice, X = sum?
E(x) = 1/36 (the 36 sums)
or...
p(X) = sum(r in X(S), p(X = r) * r)
E(sum of pair of dice) = p(sum=2)*2 + p(sum=3)*3 + ... + p(sum=12)*12
or...
linearity of expectation
E(X1 + x2 + ... + Xn) = E(X1) + E(X2) + ... + E(Xn)
E(sum of 2) = E(1st die) + E(2nd die) = 3.5 + 3.5 = 7
]]>
Stuart Reges
Last modified: Mon Nov 18 15:33:08 PST 2024