Intuition: Information Gain

• Suppose N is between 1 and 20

• How many binary questions to determine N?

• What is information gain of being told N?

• What is information gain of being told N is prime?

  • [7+, 13-]

• What is information gain of being told N is odd?

  • [10+, 10-]

• Which is better first question?

Previous slide

Next slide

Back to first slide

View graphic version