Extending the Alphabet
Assuming independence P(ab) = P(a)P(b), so we can lump symbols together.
Example: P(0) = 1/100, P(1) = 99/100
- P(00) = 1/10000, P(01) = P(10) = 99/10000, P(11) = 9801/10000.
HC = 1.03 bits/symbol (2 bit symbol)
Still not that close to H = .08 bits/bit