| Date | Content | Reading | Slides |
|---|---|---|---|
| Basics | |||
| 3/30 | Welcome/overview (Kevin and Jamie) | Murphy 1.1 - 1.3, 2.2-2.6 Optional: HTF 1 | Slides |
| Maximum Likelihood, Linear Regression, Overfitting, Regularization | |||
| 4/1 |
Maximum likelihood for Bernoulli, Gaussian (Kevin) |
Murphy 4.1, 6.4 Optional: Wasserman 9.1.-9.8 |
Slides (Kevin)
Annotated Slides (Kevin) |
| 4/6 | Linear least squares (Jamie) | Murphy 6.4, 7.1-7.3 Optional: HTF 3.1-3.2 |
Slides (Jamie)
Annotated Slides (Jamie) |
| 4/8 | Bias-Variance |
Murphy 6.4, 7.5.1, Fortmann-Roe essay Optional: HTF 7.1-7.4 |
Slides (Jamie), Annotated (Jamie) |
| 4/13 | Bias-Variance, Overfitting |
Murphy 6.4-6.5 Optional: HTF 7.1-7.4 |
Slides (Kevin), Annotated (Kevin) |
| 4/15 | Bias-Variance, Overfitting, Ridge regression |
Murphy 6.4-6.5, 7.5 Optional: HTF 7.1-7.4, 7.10-7.12, 3.4 |
Slides (Kevin), Annotated (Kevin) |
| 4/20 | k-fold cross validation, Lasso, Logistic Regression |
Murphy 6.4-6.5, 8.1-8.3, 8.5 13.1, 13.3 - 13.4.1 Optional: HTF 3 |
Lasso Slides (Kevin), Annotated Lasso Slides (Kevin), Logistic Regression Slides |
| Classification, Optimization | |||
| 4/22 | Logistic Regression |
Murphy 8.1-8.3, 8.5 Roughgarden-Valiant notes Optional: Gradient descent algorithms by Ruder Optional: Zen of gradient descent by Moritz Hardt Optional: HTF 4.1-4.2, 4.4 |
Annotated Slides |
| 4/27 | Logistic Regression, Optimization basics |
Murphy 8.1-8.3, 8.5 Roughgarden-Valiant notes Optional: Gradient descent algorithms by Ruder Optional: Zen of gradient descent by Moritz Hardt Optional: HTF 4.1-4.2, 4.4 |
Slides, Annotated Slides |
| 4/29 | SGD and computation |
Daume 4, 11, Murphy 8.5.0 Murphy 14.5 Optional: HTF 4.5, 12-12.2; Optional: HTF 4.1-4.3.1, 18.7; EH 2-2.2, 10-10.4, 11-11.2 |
Slides (Kevin), Annotated Slides (Kevin) |
| Online Learning and Margin-based Approaches | 5/4 | SVMs and Kernels (Jamie) |
Daume 4, 11, Murphy 8.5.0, 8.5.4
Murphy 14.5
Optional: HTF 4.5, 12-12.2;
Optional: HTF 4.1-4.3.1, 18.7; EH 2-2.2, 10-10.4, 11-11.2 |
Slides, Annotated Slides |
| 5/6 | Neural Networks |
Murphy 28
Optional: HTF 11 Dive into Deep Learning, Zhang et al https://playground.tensorflow.org/ |
Slides (Jamie), Slides (Kevin), Annotated Slides (Kevin) |
| Unsupervised Learning | |||
| 5/11 | Nearest neighbors, Bootstrap (Kevin), K-means and PCA (Jamie) |
HTF 2.3-2.5, 7.11, 8.2 Murphy 6.1-6.3, 12.1.0, 12.2 PCA-1 and PCA-2 notes by Roughgarden and Valiant Daume, chapter 15 Optional: HTF 5.9, 12.3; 14.5 |
Slides (Kevin), Slides (Jamie) |
| 5/13 | K-means and PCA | Slides Annotated Slides | |
| 5/18 | PCA, SVD, and EM | Daume, chapter 16
Bishop, Sections 9.2-9.4
Optional: HTF 8.5
|
Slides (Jamie), Annotated Slides (Jamie) |
| 5/20 | Structured Neural Networks (Kevin) |
Dive into Deep Learning, Zhang et al Optional: Image convolutional networks blog post Coates and Ng (2012) random patches |
Slides (Kevin) |
| 5/27 | Hyperparameter tuning (Kevin) More unsupervised learning (Kevin) Basic Text processing (Kevin) |
Ch. 8-9 of Dive into Deep Learning, Zhang et al | Slides (Kevin), Annotated Slides (Kevin) | 6/1 | Privacy and Fairness | Fairness and machine learning book | Slides, Annotated Slides |
| Other stuff | |||
| 6/3 | Decision trees, Random forests (Kevin) Boosting (Kevin) | Murphy 16.2 Murphy 16.4 Optional: HTF 9.2, 15-15.3, 10-10.11 | Slides (Kevin), Annotated Slides (Kevin) |