Review: Learning
Learning as Search
- Search in the space of hypotheses
- Hill climbing in space of decision trees
- Complete search in conjunctive hypothesis representation
Notion of Bias
- Restricted set of hypotheses
- Small H means can jump to conclusion
Tradeoff: Expressiveness / Tractability
- Big H => harder to learn
- PAC Definition
Ensembles of classifiers:
- Bagging, Boosting, Cross validated committees