Example 1
x =1 1 1 1 1 0 1 1 1 1 0 1
- P(0) = 2/12 (from frequencies)
- P(1) = 10/12 (from frequencies)
H = -((2/12) log2(2/12) + (10/12) log2(10/12))= .65
Lower bound of 12 x .65 = 7.8 bits
Standard for a two symbol alphabet is 1 bits per symbol or 12 bits.
There is a potential gain in some algorithm.