Basics (0.5 lecture)
- What is learning?
- Point estimation and MLE
- Gaussians
Mon., January 6
- Lecture: What is learning? Point estimation and MLE, Gaussians.
- Additional Reference: Andrew Moore's basic probability tutorial
- Readings: Murphy 1.1 - 1.4, 2.4.1
[Top]
Linear Regression, Overfitting, Regularization, Sparsity (1.5 Lectures)
- Linear regression
[Applet], bias-variance tradeoff
http://mste.illinois.edu/users/exner/java.f/leastsquares/ - Overfitting, regularization, ridge regression, cross-validation
- Variable selection, sparsity, LASSO
Mon., January 6:
- Lecture: Bayesian learning, linear regression, bias-variance tradeoff. [Jan 6th Slides] [Annotated Slides]
- Readings: Murphy 7.1-7.3, 6.4, 7.5.1
Mon., January 13:
- Lecture: Regularization, ridge regression, cross-validation, variable selection, sparsity, LASSO. [Slides] [Annotated Slides]
- Readings: Murphy 6.5.3, Murphy 13.1, 13.3-13.4.1
[Top]
Classification, Online Learning, Margin-based Approaches, Logistic Regression (2 Lectures)
- Online learning, perceptron, hinge loss.
- Kernels.
- SVMs
-
[LibSVM Applet]
http://www.csie.ntu.edu.tw/~cjlin/libsvm/ -
[Another SVM Applet]
http://svm.dcs.rhbnc.ac.uk/pagesnew/GPat.shtml
-
[LibSVM Applet]
- Logistic regression
[Applet]
http://www.cs.technion.ac.il/~rani/LocBoost/ - Gradient descent, stochastic gradient descent
Mon., January 20:
- NO CLASS - MLK Day
Tue., January 21
- Make up lecture in EEB 037, 630-920PM.
- Lecture: Classification, online learning, perceptron, kernels, naive Bayes. [Slides] [Annotated Slides]
- Readings: Murphy 1.2.1
- Sections 1, 2, 3.1, 4 of Freund, Yoav, and Robert E. Schapire. "Large margin classification using the perceptron algorithm." Machine learning 37.3 (1999): 277-296.
- Murphy: 3.5
Mon., January 27:
- Lecture: Logistic regression, gradient descent, SVMs, Boosting. [Slides] [Annotated Slides]
- Readings: Murphy 8.1 - 8.3, 8.5.0, 8.5.2, 8.5.4
[Top]
Non-linear Models (1 Lecture)
- Boosting
[Adaboost Applet]
http://cseweb.ucsd.edu/~yfreund/adaboost/ - Decision trees
[Applet]
http://webdocs.cs.ualberta.ca/~aixplore/learning/DecisionTrees/Applet/DecisionTreeApplet.html - Instance-based learning: Nearest Neighbors.
Mon., February 3:
- Lecture: Boosting continued, decision trees, instance-based learning. [Decision Trees, Instance-based Slides] [Annotated Slides]
- Readings: Murphy 16.4, 16.2, 1.4.2, 14,7,4, 14.7.5
[Top]
Recommender Systems (1 lecture)
- Making recommendations
- Item-based methods
- Collaborative filtering, matrix factorization
- Cold-start problem, feature-based methods
Mon., February 10:
- Lecture: Recommender systems, item-based, matrix factorization, non-negative matrix factorization, cold-start. [Slides] [Annotated Slides, merged together from various sources, since laptop crashed]
- Readings: Koren, Yehuda, Robert Bell and Chris Volinsky. "Matrix Factorization Techniques for Recommender Systems." Computer Volume: 42, Issue: 8 (2009): 30-37
[Top]
Unsupervised Learning (1 Lectures)
- Clustering, K-means
[Applet: K-means]
http://home.deib.polimi.it/matteucc/Clustering/tutorial_html/AppletKM.html - Mixtures of Gaussians, Expectation Maximization (EM).
- Dimensionality reduction (PCA, SVD) [Applet: PCA]
Mon., February 17:
- NO CLASS - President's Day
Tue., February 18:
- Make up lecture in EEB 037, 630-920PM.
- Lecture: Clustering, k-means, Gaussians, mixtures of Gaussians, EM. [Slides] [Annotated Slides]
- Readings: Murphy 11.2, 11.4, 12.1.0, 12.2
[Top]
Parallel Learning (1 Lectures)
- Map-Reduce
- Graph-parallel problems, GraphLab.
Mon. February 24:
[Top]
Neural networks, deep learning (1 Lecture)
- Neural network
- Deep architectures
- Convolutional models
- Dropouts
Mon. March 3:
- Lecture: PCA, neural networks, convolutional neural networks for computer vision. [Neural Nets Slides] [Annotated Slides]
[Top]
Learning to Make Decisions (1 Lecture)
- Learning theory continued, Markov decision processes
- Reinforcement learning
Mon. March 10:
- Lecture: Markov decision processes, reinforcement learning [Slides]
[Top]