Extending the Alphabet

• Assuming independence P(ab) = P(a)P(b), so we can lump symbols together.

• Example: P(0) = 1/100, P(1) = 99/100

  • P(00) = 1/10000, P(01) = P(10) = 99/10000, P(11) = 9801/10000.

01

11

10

00

1

1

1

0

0

0

HC = 1.03 bits/symbol (2 bit symbol)

= .515 bits/bit

Still not that close to H = .08 bits/bit

Previous slide

Next slide

Back to first slide

View graphic version