First-order Information

• Suppose we are given symbols {a1, a2, ... , am}.

• P(ai) = probability of symbol ai occurring in the absence of any other information.

  • P(a1) + P(a2) + ... + P(am) = 1

• inf(ai) = -log2 P(ai) bits is the information in bits of ai.

Previous slide

Next slide

Back to first slide

View graphic version