Basics (1 lecture)
- What is learning?
- Point estimation and MLE
Mon., April 1:
- Lecture: What is learning? Point estimation and MLE. [Slides] [Annotated Slides]
- Additional Reference: Andrew Moore's basic probability tutorial
- Readings: Murphy 1.1 - 1.4
[Top]
Linear Regression, Overfitting, Regularization, Sparsity (5 Lectures)
- Gaussians
- Linear regression
[Applet]
http://mste.illinois.edu/users/exner/java.f/leastsquares/ - Bias-Variance tradeoff
- Overfitting
- Regularization
- LASSO
Wed., April 3:
- Lecture: MLE, Gaussians. [Slides] [Annotated Slides]
- Readings: Murphy 2.4.1
Thu., April 4:
- Optional: R tutorial. EEB 125 6:00pm [R Tutorial]
Fri., April 5:
- Lecture: Linear regression, bias-variance tradeoff. [Slides] [Annotated Slides]
- Readings: Murphy 7.1-7.3, 6.4
Mon., April 8:
- Lecture: Overfitting, regularization. [Slides] [Annotated Slides]
- Readings: Murphy 7.5.1
Wed., April 10:
- Lecture: Regularization, ridge regression, cross-validation. [Slides] [Annotated Slides]
- Readings: Murphy 6.5.3
Fri., April 12:
- Lecture: Variable selection, sparsity, LASSO. [Annotated Slides]
- Readings: Murphy 13.1, 13.3-13.4.1
[Top]
Classification, Logistic Regression (2 Lectures)
- Logistic regression
[Applet]
http://www.cs.technion.ac.il/~rani/LocBoost/ - Gradient descent, stochastic gradient descent
Mon., April 15:
- Lecture: LASSO Big Picture, classification, logistic regression. [Slides] [Annotated Slides]
- Readings: Murphy 8.1 - 8.3
Wed., April 17:
- Lecture: Logistic regression, gradient descent. [Slides] [Annotated Slides]
- Readings: Murphy 8.5.2
[Top]
Non-linear Models (3 Lectures)
- Decision trees
[Applet]
http://webdocs.cs.ualberta.ca/~aixplore/learning/DecisionTrees/Applet/DecisionTreeApplet.html - Overfitting, Cross-validation
- Boosting
[Adaboost Applet]
http://cseweb.ucsd.edu/~yfreund/adaboost/ - Instance-based learning
- K-nearest neighbors
- Kernels
Fri., April 19:
- Lecture: Stochastic gradient descent (continued from previous module). Decision trees. [Slides] [Annotated Slides]
- Readings: Murphy 6.2
Mon., April 22:
- Lecture: Decision trees continued, Boosting. [Boosting Slides] [Annotated Slides]
- Readings: Murphy 6.4
Wed., April 24:
- Lecture: Boosting continued. [Annotated Slides]
Fri., April 26:
- Lecture: Boosting continued. Instance-based learning, nearest neighbors. [Slides] [Annotated Slides]
[Top]
Online Learning and Margin-based Approaches (3 Lectures)
- Perceptron
- Kernel trick
- SVMs
-
[LibSVM Applet]
http://www.csie.ntu.edu.tw/~cjlin/libsvm/ -
[Another SVM Applet]
http://svm.dcs.rhbnc.ac.uk/pagesnew/GPat.shtml
-
[LibSVM Applet]
Mon., April 29:
- Lecture: Online learning, perceptron. [Slides] [Annotated Slides]
- Readings: Murphy 8.5.0, 8.5.4
Wed., May 1:
- Lecture: What's the Perceptron optimizing? Hinge loss. Kernel trick. [Slides] [Annotated Slides]
- Readings: Murphy 14.4
Fri., May 3:
- Lecture: Kernels, SVMs. [Slides] [Annotated Slides]
- Readings: Murphy 14.5
Mon., May 6:
- Lecture: SVMs, review. [Slides] [Annotated Slides]
Wed., May 8:
- IN CLASS - Midterm
[Top]
Learning Theory (1 Lectures)
- Sample complexity
- PAC learning
- VC-dimension
Fri., May 10:
- Lecture: Learning Theory. [Slides] [Annotated Slides]
- Readings: Murphy 6.5.4
[Top]
Unsupervised Learning (4 Lectures)
- K-means
[Applet: K-means]
http://home.deib.polimi.it/matteucc/Clustering/tutorial_html/AppletKM.html - Expectation Maximization (EM) for Mixture of Gaussians
- Dimensionality reduction (PCA, SVD) [Applet: PCA]
Mon., May 13:
- Lecture: Learning theory continued, clustering, K-means. [Slides] [Annotated Slides]
- Readings: Murphy 11.4.2.5
Wed., May 15:
- Lecture: Mixtures of Gaussians. [Slides] [Annotated Slides]
- Readings: Murphy 11.2
Fri., May 17:
- Lecture: Mixtures of Gaussians continued, EM. [Slides] [Annotated Slides]
- Readings: Murphy 11.4 - 11.4.2.3
Mon., May 20:
- Lecture: EM continued, dimensionality reduction, PCA, SVD. [Slides] [Annotated Slides]
- Readings: Murphy 12.1.0, 12.2
[Top]
Structured Models (4 Lectures)
- Bayes optimal classifier
- Naive Bayes
[Applet]
http://www.cs.technion.ac.il/~rani/LocBoost/ - HMMs
- Forwards-backwards, Viterbi
- Supervised learning
- Graphical Models
-
[Applet: Java Bayes]
http://sites.poli.usp.br/pmr/ltd/Software/javabayes/Home/ -
[Another Bayes net Applet]
http://www.aispace.org/bayes/version5.1.6/bayes.jnlp - Representation
- Parameter and Structure Learning
Wed., May 22:
- Lecture: PCA continued, Bayes optimal classifier, Naive Bayes. [Slides] [Annotated Slides]
- Readings: Murphy 3.5, 17.3-17.4
Fri., May 24:
- Lecture: Naive Bayes, Bayesian networks. [Slides] [Annotated Slides]
- Readings: Murphy 10.1 - 10.2.2
Mon., May 27:
- NO CLASS - Memorial Day
Wed., May 29:
- Lecture: Bayes net representation. [Annotated Slides]
- Readings: Murphy 10.4
Fri., May 31:
- Lecture: Bayes net structure learning. [Slides] [Annotated Slides]
- Readings: Murphy 10.3
[Top]
Learning to Make Decisions (2 Lectures)
- Markov decision processes
- Reinforcement learning
Mon., June 3:
- Lecture: Markov decision processes. [Slides] [Annotated Slides]
Wed., June 5:
- Lecture: Reinforcement learning. [Slides] [Annotated Slides]
Fri., June 7:
- Lecture: Reinforcement learning, Rmax. [Annotated Slides]
[Top]