The Channel-Coding Theorem (con’t)
In words:
- If the entropy of our symbol stream is equal to or less than the channel capacity, then there exists a coding technique that enables transmission over the channel with arbitrarily small error
- Can transmit information at a rate H(X) <= C
Shannon’s theorem tells us the asymptotically maximum rate
- It does not tell us the code that we must use to obtain this rate
- Achieving a high rate may require a prohibitively long code