Entropy Examples

• {a, b, c} with a 1/8, b 1/4, c 5/8.

  • H = 1/8 *3 + 1/4 *2 + 5/8* .678 = 1.3 bits/symbol

• {a, b, c} with a 1/3, b 1/3, c 1/3. (worst case)

  • H = -3* (1/3)*log2(1/3) = 1.6 bits/symbol

• {a, b, c} with a 1, b 0, c 0 (best case)

  • H = -1*log2(1) = 0

• Note that the standard coding of 3 symbols takes 2 bits.

Previous slide

Next slide

Back to first slide

View graphic version