Entropy definition of information

• Is there a subset of X such that Hd(X) << Ho(X)?

  • For sufficiently small values of d
  • i.e. can we discard the unlikely elements of X, so as to extract the useful information and significantly reduce the number of bits

• Want to specify the information content of X with a number

  • Hd(X) is a function

• The average self-information or entropy of an ensemble X

• • E ? expected value

Previous slide

Next slide

Back to first slide

View graphic version