Basics (1 lecture)

  • What is learning?
  • Point estimation and MLE
  • Gaussians

Wed., September 25:

[Top]

Linear Regression, Overfitting, Regularization, Sparsity (4 Lectures)

  • Linear regression [Applet]
    http://mste.illinois.edu/users/exner/java.f/leastsquares/
  • Bias-Variance tradeoff
  • Overfitting
  • Regularization
  • Cross-validation
  • LASSO

Mon., September 30:

  • Lecture: Bayesian learning, linear regression, bias-variance tradeoff. [Slides] [Annotated Slides]
  • Readings: Murphy 7.1-7.3, 6.4, 7.5.1

Tue., October :

Wed., October 2:

Mon., October 7:

Tue., October 8:

[Top]

Classification, Logistic Regression (1 Lecture)

  • Logistic regression [Applet]
    http://www.cs.technion.ac.il/~rani/LocBoost/
  • Gradient descent, stochastic gradient descent

Wed., October 9:

[Top]

Non-linear Models (3 Lectures)

  • Decision trees [Applet]
    http://webdocs.cs.ualberta.ca/~aixplore/learning/DecisionTrees/Applet/DecisionTreeApplet.html
  • Boosting [Adaboost Applet]
    http://cseweb.ucsd.edu/~yfreund/adaboost/
  • Instance-based learning
    • K-nearest neighbors
    • Kernels

Mon., October 14:

Tue., October 15:

Wed., October 16:

Mon., October 21:

Tue., October 22:

  • Recitation: Boosting/evaluation . LOW 101 5:30pm [Slides]

[Top]

Online Learning and Margin-based Approaches (2 Lectures)

Wed., October 23:

Mon., October 28:

Wed., October 30:

  • IN CLASS - Midterm

[Top]

Unsupervised Learning (3 Lectures)

  • K-means [Applet: K-means]
    http://home.deib.polimi.it/matteucc/Clustering/tutorial_html/AppletKM.html
  • Expectation Maximization (EM) for Mixture of Gaussians
  • Dimensionality reduction (PCA, SVD) [Applet: PCA]

Mon., November 4:

Tue., November 5:

  • Recitation: Perceptron, SVM. LOW 101 5:30pm. [Slides]

Wed., November 6:

Mon., November 11:

  • NO CLASS - Veterans Day

Tue., November 12:

Wed., November 13:

[Top]

Structured Models (2 Lectures)

Mon., November 18:

  • Lecture: CANCELLED.

Wed., November 20:

Mon., November 25:

Tue., November 26:

  • Recitation: Naive Bayes and Bayesian Networks. LOW 101 5:30pm [Slides]

[Top]

Learning Theory (2 Lectures)

  • Sample complexity
  • PAC learning
  • VC-dimension

Wed., November 27:

[Top]

Learning to Make Decisions (2 Lectures)

  • Markov decision processes
  • Reinforcement learning

Mon., December 2:

Wed., December 4:

[Top]