Basics (1 lecture)
 What is learning?
 Point estimation and MLE
 Gaussians
Wed., September 25:
 Lecture: What is learning? Point estimation and MLE, Gaussians. [Slides] [Annotated Slides]
 Additional Reference: Andrew Moore's basic probability tutorial
 Readings: Murphy 1.1  1.4, 2.4.1
[Top]
Linear Regression, Overfitting, Regularization, Sparsity (4 Lectures)
 Linear regression
[Applet]
http://mste.illinois.edu/users/exner/java.f/leastsquares/  BiasVariance tradeoff
 Overfitting
 Regularization
 Crossvalidation
 LASSO
Mon., September 30:
 Lecture: Bayesian learning, linear regression, biasvariance tradeoff. [Slides] [Annotated Slides]
 Readings: Murphy 7.17.3, 6.4, 7.5.1
Tue., October :
 Optional: Python tutorial. LOW 101 5:30pm [Code and dataset]
Wed., October 2:
 Lecture: Overfitting, regularization, ridge regression, crossvalidation. [Slides] [Annotated Slides]
 Readings: Murphy 6.5.3
Mon., October 7:
 Lecture: Variable selection, sparsity, LASSO. [Slides] [Annotated Slides]
 Readings: Murphy 13.1, 13.313.4.1
Tue., October 8:
 Recitation: BiasVariance TradeOff, L1 vs. L2. LOW 101 5:30pm [iPython Notebook] [static html]
[Top]
Classification, Logistic Regression (1 Lecture)
 Logistic regression
[Applet]
http://www.cs.technion.ac.il/~rani/LocBoost/  Gradient descent, stochastic gradient descent
Wed., October 9:
 Lecture: Classification, logistic regression, gradient descent. [Slides] [Annotated Slides]
 Readings: Murphy 8.1  8.3, 8.5.2
[Top]
Nonlinear Models (3 Lectures)
 Decision trees
[Applet]
http://webdocs.cs.ualberta.ca/~aixplore/learning/DecisionTrees/Applet/DecisionTreeApplet.html  Boosting
[Adaboost Applet]
http://cseweb.ucsd.edu/~yfreund/adaboost/  Instancebased learning
 Knearest neighbors
 Kernels
Mon., October 14:
 Lecture: Stochastic gradient descent, boosting. [Boosting Slides] [Annotated Slides]
 Readings: Murphy 16.4
Wed., October 16:
 Lecture: Boosting continued, decision trees. [Decision Trees Slides] [Annotated Slides]
 Readings: Murphy 16.2
Mon., October 21:
 Lecture: Decision trees continued, instancebased learning, nearest neighbors. [Instancebased Learning Slides] [Annotated Slides]
Tue., October 22:
 Recitation: Boosting/evaluation . LOW 101 5:30pm [Slides]
[Top]
Online Learning and Marginbased Approaches (2 Lectures)
 Perceptron
 Kernel trick
 SVMs

[LibSVM Applet]
http://www.csie.ntu.edu.tw/~cjlin/libsvm/ 
[Another SVM Applet]
http://svm.dcs.rhbnc.ac.uk/pagesnew/GPat.shtml

[LibSVM Applet]
Wed., October 23:
 Lecture: Online learning, perceptron, hinge loss. [Slides] [Annotated Slides]
 Readings: Murphy 8.5.0, 8.5.4, 14.4
Mon., October 28:
 Lecture: Kernels, SVMs. [Slides] [Annotated Slides]
 Readings: Murphy 14.5
Wed., October 30:
 IN CLASS  Midterm
[Top]
Unsupervised Learning (3 Lectures)
 Kmeans
[Applet: Kmeans]
http://home.deib.polimi.it/matteucc/Clustering/tutorial_html/AppletKM.html  Expectation Maximization (EM) for Mixture of Gaussians
 Dimensionality reduction (PCA, SVD) [Applet: PCA]
Mon., November 4:
 Lecture: Clustering, Kmeans. [Slides] [Annotated Slides]
 Readings: Murphy 11.4.2.5
Tue., November 5:
 Recitation: Perceptron, SVM. LOW 101 5:30pm. [Slides]
Wed., November 6:
 Lecture: Mixtures of Gaussians, EM. [Slides] [Annotated Slides]
 Readings: Murphy 11.2, 11.4  11.4.2.3
Mon., November 11:
 NO CLASS  Veterans Day
Wed., November 13:
 Lecture: EM Continued, dimensionality reduction, PCA, SVD. [Slides] [Annotated Slides]
 Readings: Murphy 12.1.0, 12.2
[Top]
Structured Models (2 Lectures)
 Naive Bayes
[Applet]
http://www.cs.technion.ac.il/~rani/LocBoost/  Graphical Models

[Applet: Java Bayes]
http://sites.poli.usp.br/pmr/ltd/Software/javabayes/Home/ 
[Another Bayes net Applet]
http://www.aispace.org/bayes/version5.1.6/bayes.jnlp  Representation
 Parameter and Structure Learning
Mon., November 18:
 Lecture: CANCELLED.
Wed., November 20:
 Lecture: PCA continued, Naive Bayes. [Slides] [Johan's Additional Notes]
 Readings: Murphy 3.5, 10.1  10.2.2
Mon., November 25:
 Lecture: Bayesian Networks, Bayes net structure learning. [Slides] [Annotated Slides]
 Readings: Murphy 10.3
Tue., November 26:
 Recitation: Naive Bayes and Bayesian Networks. LOW 101 5:30pm [Slides]
[Top]
Learning Theory (2 Lectures)
 Sample complexity
 PAC learning
 VCdimension
Wed., November 27:
 Lecture: Bayes nets structure learning, learning Theory. [Learning Theory Slides] [Annotated Slides]
 Readings: Murphy 6.5.4
[Top]
Learning to Make Decisions (2 Lectures)
 Markov decision processes
 Reinforcement learning
Mon., December 2:
 Lecture: Learning theory continued, Markov decision processes. [MDPs Slides] [Annotated Slides]
Wed., December 4:
 Lecture: MDPs, Reinforcement learning. [Slides] [Annotated Slides]
[Top]