Average mutual information

• Definition: I(X;Y) = H(X) – H(X½Y)

  • I(X;Y) = I(Y;X)

• Properties

  • I(X;Y) >= 0
    • With equality iff x and y are statistically independent
  • I(X;Y) <= H(X)
    • With equality iff x is a function of y
    • x cannot convey more information about y than its own entropy
  • I(X;Y) <= H(Y)

Previous slide

Next slide

Back to first slide

View graphic version