Basics (0.5 lecture)

  • What is learning?
  • Point estimation and MLE
  • Gaussians

Mon., January 6

[Top]

Linear Regression, Overfitting, Regularization, Sparsity (1.5 Lectures)

  • Linear regression [Applet], bias-variance tradeoff
    http://mste.illinois.edu/users/exner/java.f/leastsquares/
  • Overfitting, regularization, ridge regression, cross-validation
  • Variable selection, sparsity, LASSO

Mon., January 6:

Mon., January 13:

  • Lecture: Regularization, ridge regression, cross-validation, variable selection, sparsity, LASSO. [Slides] [Annotated Slides]
  • Readings: Murphy 6.5.3, Murphy 13.1, 13.3-13.4.1

[Top]

Classification, Online Learning, Margin-based Approaches, Logistic Regression (2 Lectures)

  • Online learning, perceptron, hinge loss.
  • Kernels.
  • SVMs
  • Logistic regression [Applet]
    http://www.cs.technion.ac.il/~rani/LocBoost/
  • Gradient descent, stochastic gradient descent

Mon., January 20:

  • NO CLASS - MLK Day

Tue., January 21

Mon., January 27:

  • Lecture: Logistic regression, gradient descent, SVMs, Boosting. [Slides] [Annotated Slides]
  • Readings: Murphy 8.1 - 8.3, 8.5.0, 8.5.2, 8.5.4

[Top]

Non-linear Models (1 Lecture)

  • Boosting [Adaboost Applet]
    http://cseweb.ucsd.edu/~yfreund/adaboost/
  • Decision trees [Applet]
    http://webdocs.cs.ualberta.ca/~aixplore/learning/DecisionTrees/Applet/DecisionTreeApplet.html
  • Instance-based learning: Nearest Neighbors.

Mon., February 3:

[Top]

Recommender Systems (1 lecture)

  • Making recommendations
  • Item-based methods
  • Collaborative filtering, matrix factorization
  • Cold-start problem, feature-based methods

[Top]

Unsupervised Learning (1 Lectures)

  • Clustering, K-means [Applet: K-means]
    http://home.deib.polimi.it/matteucc/Clustering/tutorial_html/AppletKM.html
  • Mixtures of Gaussians, Expectation Maximization (EM).
  • Dimensionality reduction (PCA, SVD) [Applet: PCA]

Mon., February 17:

  • NO CLASS - President's Day

Tue., February 18:

  • Make up lecture in EEB 037, 630-920PM.
  • Lecture: Clustering, k-means, Gaussians, mixtures of Gaussians, EM.
  • [Slides] [Annotated Slides]
  • Readings: Murphy 11.2, 11.4, 12.1.0, 12.2

[Top]

[Top]

Neural networks, deep learning (1 Lecture)

  • Neural network
  • Deep architectures
  • Convolutional models
  • Dropouts

Mon. March 3:

[Top]

Learning to Make Decisions (1 Lecture)

  • Learning theory continued, Markov decision processes
  • Reinforcement learning

Mon. March 10:

  • Lecture: Markov decision processes, reinforcement learning [Slides]

[Top]