Notes on entropy

• 0 <= H(X) <= Ho(X)

  • With equality iff all the ak’s are equiprobable
  • Can show that 0log(1/0) = 0

• Consider the case where k = 2

  • X = {a1 a2}
  • P(a1) = ?; P(a2) = 1– ?

Previous slide

Next slide

Back to first slide

View graphic version