| Date | Content | Reading | Slides |
|---|---|---|---|
| Intro, Maximum Likelihood, Linear Regression, Overfitting, Regularization | |||
| 3/29 M | Welcome/overview (Simon and Sewoong) Maximum likelihood for Bernoulli (Sewoong) |
Murphy 1.1 - 1.3, 2.2-2.6, 4.1, 6.4 Optional: HTF 1, Wasserman 9.1.-9.8 | Combined week 1 Slides (Sewoong), Lecture 1 (Simon/Sewoong), Annotated Lecture 1 (Simon/Sewoong), |
| 3/31 W | Maximum likelihood for Gaussian, Linear least squares (Sewoong) | Murphy 6.4, 7.1-7.3
Optional: HTF 3.1-3.2 |
Lecture 2 (Sewoong), Annotated Lecture 2 (Sewoong) |
| 4/2 F | Linear least squares (Sewoong) | Murphy 6.4, 7.1-7.3
Optional: HTF 3.1-3.2 |
Lecture 3 (Sewoong), Annotated Lecture 3 (Sewoong) |
| 4/5 M | Generalized linear regression, Bias-Variance tradeoff (Simon) |
Murphy 6.4, 7.5.1, Fortmann-Roe essay Optional: HTF 7.1-7.4 |
Combined week 2 slides (Simon), Lecture 4 (Simon), Annotated Lecture 4 (Simon) |
| 4/7 W | Bias-Variance tradeoff (Simon) |
Murphy 6.4, 7.5.1 Optional: HTF 7.1-7.4 |
Lecture 5 (Simon), Annotated Lecture 5 (Simon) |
| 4/9 F | Bias-Variance tradeoff and Overfitting (Simon) |
Murphy 6.4-6.5 Optional: HTF 7.1-7.4 |
Lecture 6 (Simon), Annotated Lecture 6 (Simon) |
| 4/12 M | Ridge regression (Sewoong) |
Murphy 6.4-6.5 Optional: HTF 7.10-7.12, 3.4 |
Combined week 3 Slides (Sewoong), demo_tradeoff.ipynb, demo_tradeoff.html, Lecture 7 (Sewoong), Annotated Lecture 7 (Sewoong) |
| 4/14 W | k-fold cross validation (Sewoong) |
Murphy 6.4-6.5, Optional: HTF 3 |
Lecture 8 (Sewoong), Annotated Lecture 8 (Sewoong), |
| 4/16 F | Lasso (Sewoong) |
Murphy 13.1, 13.3 - 13.4.1 Optional: HTF 3 |
demo_lasso.html (Sewoong), demo_lasso.ipynb (Sewoong), Lecture 9 (Sewoong), Annotated Lecture 9 (Sewoong) |
| Classification, Optimization | |||
| 4/19 M | Classification, Logistic Regression (Simon) |
Murphy 8.1-8.3, 8.5
Optional: HTF 4.1-4.2, 4.4 |
Combined lecture 10 and 11 slides (Simon) Lecture 10 (Simon), Annotated Lecture 10 (Simon) |
| 4/21 W | Logistic Regression (Simon) |
Murphy 8.1-8.3, 8.5
Optional: HTF 4.1-4.2, 4.4 |
Lecture 11 (Simon), Annotated Lecture 11 (Simon) |
| 4/23 F | Convexity (Sewoong) |
Murphy 8.1-8.3, 8.5 Roughgarden-Valiant notes Optional: Gradient descent algorithms by Ruder Optional: Zen of gradient descent by Moritz Hardt Optional: HTF 4.1-4.2, 4.4 |
Combined lectures 12,13,14 (Sewoong), Lecture 12 (Sewoong), Annotated Lecture 12 (Sewoong) |
| 4/26 M | Gradient Descent (Sewoong) |
Daume 4, 11, Murphy 8.5.0 Murphy 14.5 Optional: HTF 4.5, 12-12.2; Optional: HTF 4.1-4.3.1, 18.7; EH 2-2.2, 10-10.4, 11-11.2 |
Lecture 13 (Sewoong), Annotated Lecture 13 (Sewoong) |
| 4/28 W | Coordiante descent and Stochastic gradient descent (Sewoong) |
Daume 4, 11, Murphy 8.5.0, 8.5.4
Murphy 14.5
Optional: HTF 4.5, 12-12.2;
Optional: HTF 4.1-4.3.1, 18.7; EH 2-2.2, 10-10.4, 11-11.2 |
Lecture 14 (Sewoong), Annotated Lecture 14 (Sewoong) |
| 4/30 F | Support Vector Machine (SVM) (Simon) |
Daume 4, 11, Murphy 8.5.0, 8.5.4
Murphy 14.5
Optional: HTF 4.5, 12-12.2;
Optional: HTF 4.1-4.3.1, 18.7; EH 2-2.2, 10-10.4, 11-11.2 |
Lecture 15 (Sewoong), Annotated Lecture 15 (Sewoong), Lecture 15 (Simon), Annotated Lecture 15 (Simon), Combined lectures 15,16,17 (Simon) |
| 5/3 M | Support Vector Machines (Simon) |
Daume 4, 11, Murphy 8.5.0, 8.5.4
Murphy 14.5
Optional: HTF 4.5, 12-12.2;
Optional: HTF 4.1-4.3.1, 18.7; EH 2-2.2, 10-10.4, 11-11.2 |
Lecture 16 (Simon), Annotated Lecture 16 (Simon) |
| Non-linear models | |||
| 5/5 W | kernels (Simon) | Lecture 17 (Simon), Annotated Lecture 17 (Simon) | |
| 5/7 F | kernels (Simon) | Lecture 18 (Simon), Annotated Lecture 18 (Simon) | |
| 5/10 M | Neural Network (Simon) |
Murphy 28
Optional: HTF 11 Dive into Deep Learning, Zhang et al https://playground.tensorflow.org/ |
Week 7 combined (Simon) Lecture 19 (Simon), Annotated Lecture 19 (Simon) |
| 5/12 W | Neural Network Training (Simon) |
Murphy 28
Optional: HTF 11 Dive into Deep Learning, Zhang et al https://playground.tensorflow.org/ |
Lecture 20 (Simon), Annotated Lecture 20 (Simon) |
| 5/14 F | Neural Network Architecture (Simon) |
Murphy 28
Optional: HTF 11 Dive into Deep Learning, Zhang et al https://playground.tensorflow.org/ |
Lecture 21 (Simon), Annotated Lecture 21 (Simon) |
| 5/17 M | Nearest Neighbor, Bootstrap (Sewoong) | HTF 2-2.5, HTF 6-6.3, EH 10-10.4, EH 11-11.2 | Week 8 combined (Sewoong), Lecture 22 (Sewoong), Annotated Lecture 22 (Sewoong), |
| Unsupervised Learning | |||
| 5/19 W | Principal Componenet Analysis (Sewoong) |
HTF 2.3-2.5, 7.11, 8.2 Murphy 6.1-6.3, 12.1.0, 12.2 Daume, chapter 15 Optional: HTF 5.9, 12.3; 14.5 |
Lecture 23 (Sewoong), Annotated Lecture 23 (Sewoong) |
| 5/21 F | PCA and matrix completion (Sewoong) |
PCA note 1
and notes 2, Matrix completion paper analyzing gradient descent paper , Influential paper on deanonymizing netflix challenge using IMDB paper |
Lecture 24 (Sewoong), Annotated Lecture 24 (Sewoong) |
| 5/24 M | k-means and Gaussian mixtures (Sewoong) | Dive into Deep Learning, Zhang et al
HTF 14.3.6-14.3.7 Murphy 11.2-11.4, k-means ++ paper |
Lecture 25 (Sewoong), Annotated Lecture 25 (Sewoong) |
| 5/26 W | Feature Extraction (Simon) | Dive into Deep Learning, Zhang et al | Combined Lectures 26, 27(Simon), Lectures 26 (Simon), Annotated Lecture 26(Simon) |
| 5/28 F | Feature Extraction (Simon) | Dive into Deep Learning, Zhang et al | Lectures 27 (Simon), Annotated Lecture 27(Simon) |
| 5/31 M | Memorial day (no lecture) | ||
| 6/2 W | Trees, Random forests, boosting (Simon) |
Murphy 16.2, 16.4 |
Lecture 28 (Simon), Annotated Lecture 28 (Simon) |
| 6/4 F | Robust machine learning (Sewoong) | Lecture 29 (Sewoong), Annotated Lecture 29 (Sewoong) |