Basics (0.5 lecture)
 What is learning?
 Point estimation and MLE
 Gaussians
Mon., January 6
 Lecture: What is learning? Point estimation and MLE, Gaussians.
 Additional Reference: Andrew Moore's basic probability tutorial
 Readings: Murphy 1.1  1.4, 2.4.1
[Top]
Linear Regression, Overfitting, Regularization, Sparsity (1.5 Lectures)
 Linear regression
[Applet], biasvariance tradeoff
http://mste.illinois.edu/users/exner/java.f/leastsquares/  Overfitting, regularization, ridge regression, crossvalidation
 Variable selection, sparsity, LASSO
Mon., January 6:
 Lecture: Bayesian learning, linear regression, biasvariance tradeoff. [Jan 6th Slides] [Annotated Slides]
 Readings: Murphy 7.17.3, 6.4, 7.5.1
Mon., January 13:
 Lecture: Regularization, ridge regression, crossvalidation, variable selection, sparsity, LASSO. [Slides] [Annotated Slides]
 Readings: Murphy 6.5.3, Murphy 13.1, 13.313.4.1
[Top]
Classification, Online Learning, Marginbased Approaches, Logistic Regression (2 Lectures)
 Online learning, perceptron, hinge loss.
 Kernels.
 SVMs

[LibSVM Applet]
http://www.csie.ntu.edu.tw/~cjlin/libsvm/ 
[Another SVM Applet]
http://svm.dcs.rhbnc.ac.uk/pagesnew/GPat.shtml

[LibSVM Applet]
 Logistic regression
[Applet]
http://www.cs.technion.ac.il/~rani/LocBoost/  Gradient descent, stochastic gradient descent
Mon., January 20:
 NO CLASS  MLK Day
Tue., January 21
 Make up lecture in EEB 037, 630920PM.
 Lecture: Classification, online learning, perceptron, kernels, naive Bayes. [Slides] [Annotated Slides]
 Readings: Murphy 1.2.1
 Sections 1, 2, 3.1, 4 of Freund, Yoav, and Robert E. Schapire. "Large margin classification using the perceptron algorithm." Machine learning 37.3 (1999): 277296.
 Murphy: 3.5
Mon., January 27:
 Lecture: Logistic regression, gradient descent, SVMs, Boosting. [Slides] [Annotated Slides]
 Readings: Murphy 8.1  8.3, 8.5.0, 8.5.2, 8.5.4
[Top]
Nonlinear Models (1 Lecture)
 Boosting
[Adaboost Applet]
http://cseweb.ucsd.edu/~yfreund/adaboost/  Decision trees
[Applet]
http://webdocs.cs.ualberta.ca/~aixplore/learning/DecisionTrees/Applet/DecisionTreeApplet.html  Instancebased learning: Nearest Neighbors.
Mon., February 3:
 Lecture: Boosting continued, decision trees, instancebased learning. [Decision Trees, Instancebased Slides] [Annotated Slides]
 Readings: Murphy 16.4, 16.2, 1.4.2, 14,7,4, 14.7.5
[Top]
Recommender Systems (1 lecture)
 Making recommendations
 Itembased methods
 Collaborative filtering, matrix factorization
 Coldstart problem, featurebased methods
Mon., February 10:
 Lecture: Recommender systems, itembased, matrix factorization, nonnegative matrix factorization, coldstart. [Slides] [Annotated Slides, merged together from various sources, since laptop crashed]
 Readings: Koren, Yehuda, Robert Bell and Chris Volinsky. "Matrix Factorization Techniques for Recommender Systems." Computer Volume: 42, Issue: 8 (2009): 3037
[Top]
Unsupervised Learning (1 Lectures)
 Clustering, Kmeans
[Applet: Kmeans]
http://home.deib.polimi.it/matteucc/Clustering/tutorial_html/AppletKM.html  Mixtures of Gaussians, Expectation Maximization (EM).
 Dimensionality reduction (PCA, SVD) [Applet: PCA]
Mon., February 17:
 NO CLASS  President's Day
Tue., February 18:
 Make up lecture in EEB 037, 630920PM.
 Lecture: Clustering, kmeans, Gaussians, mixtures of Gaussians, EM. [Slides] [Annotated Slides]
 Readings: Murphy 11.2, 11.4, 12.1.0, 12.2
[Top]
Parallel Learning (1 Lectures)
 MapReduce
 Graphparallel problems, GraphLab.
Mon. February 24:
[Top]
Neural networks, deep learning (1 Lecture)
 Neural network
 Deep architectures
 Convolutional models
 Dropouts
Mon. March 3:
 Lecture: PCA, neural networks, convolutional neural networks for computer vision. [Neural Nets Slides] [Annotated Slides]
[Top]
Learning to Make Decisions (1 Lecture)
 Learning theory continued, Markov decision processes
 Reinforcement learning
Mon. March 10:
 Lecture: Markov decision processes, reinforcement learning [Slides]
[Top]