Example 2
x =1 2 3 4 5 4 5 6 7 8 7 8
- P(1) = P(2) = P(3) = P(6) = 1/12 (from frequencies)
- P(4) = P(5) = P(7) = P(8) = 2/12 (from frequencies)
H = -((4/12) log2(1/12) + (8/12) log2(2/12))= 2.92
Lower bound of 12 x 2.92 = 35.02 bits
Standard for an 8 symbol alphabet is 3 bits per symbol or 36 bits.
No compression algorithm will give us much.