Basics (1 lecture)

  • What is learning?
  • Point estimation and MLE

Mon., April 1:

[Top]

Linear Regression, Overfitting, Regularization, Sparsity (5 Lectures)

  • Gaussians
  • Linear regression [Applet]
    http://mste.illinois.edu/users/exner/java.f/leastsquares/
  • Bias-Variance tradeoff
  • Overfitting
  • Regularization
  • LASSO

Wed., April 3:

Thu., April 4:

Fri., April 5:

Mon., April 8:

Wed., April 10:

Fri., April 12:

  • Lecture: Variable selection, sparsity, LASSO. [Annotated Slides]
  • Readings: Murphy 13.1, 13.3-13.4.1

[Top]

Classification, Logistic Regression (2 Lectures)

  • Logistic regression [Applet]
    http://www.cs.technion.ac.il/~rani/LocBoost/
  • Gradient descent, stochastic gradient descent

Mon., April 15:

Wed., April 17:

[Top]

Non-linear Models (3 Lectures)

  • Decision trees [Applet]
    http://webdocs.cs.ualberta.ca/~aixplore/learning/DecisionTrees/Applet/DecisionTreeApplet.html
  • Overfitting, Cross-validation
  • Boosting [Adaboost Applet]
    http://cseweb.ucsd.edu/~yfreund/adaboost/
  • Instance-based learning
    • K-nearest neighbors
    • Kernels

Fri., April 19:

  • Lecture: Stochastic gradient descent (continued from previous module). Decision trees. [Slides] [Annotated Slides]
  • Readings: Murphy 6.2

Mon., April 22:

Wed., April 24:

Fri., April 26:

[Top]

Online Learning and Margin-based Approaches (3 Lectures)

Mon., April 29:

Wed., May 1:

Fri., May 3:

Mon., May 6:

Wed., May 8:

  • IN CLASS - Midterm

[Top]

Learning Theory (1 Lectures)

  • Sample complexity
  • PAC learning
  • VC-dimension

Fri., May 10:

[Top]

Unsupervised Learning (4 Lectures)

  • K-means [Applet: K-means]
    http://home.deib.polimi.it/matteucc/Clustering/tutorial_html/AppletKM.html
  • Expectation Maximization (EM) for Mixture of Gaussians
  • Dimensionality reduction (PCA, SVD) [Applet: PCA]

Mon., May 13:

Wed., May 15:

Fri., May 17:

Mon., May 20:

[Top]

Structured Models (4 Lectures)

  • Bayes optimal classifier
  • Naive Bayes [Applet]
    http://www.cs.technion.ac.il/~rani/LocBoost/
  • HMMs
    • Forwards-backwards, Viterbi
    • Supervised learning
  • Graphical Models

Wed., May 22:

Fri., May 24:

Mon., May 27:

  • NO CLASS - Memorial Day

Wed., May 29:

Fri., May 31:

[Top]

Learning to Make Decisions (2 Lectures)

  • Markov decision processes
  • Reinforcement learning

Mon., June 3:

Wed., June 5:

Fri., June 7:

[Top]