Readings are from Murphy unless otherwise noted.
| Date | Content | Reading | Resources |
|---|---|---|---|
| The basics: regression and classification | |||
| M 3/30 | Welcome/Overview; MLE |
Probability review: Murphy 2.1-2.4, 2.6, 2.8, 3.1-3.2 Statistics review: Murphy 4.2 |
Slides |
| W 4/1 | Maximum Likelihood Estimation (MLE) | Statistics review, maximum likelihood: Murphy 4.2 | Slides |
| F 4/3 | Linear regression |
Linear algebra review: Murphy 7.1-7.3 Matrix calculus review: Murphy 7.8 Maximum likelihood estimation: Murphy 4.2 Linear regression: Murphy 11-11.2 |
Slides, Linear regression colab, diabetes.txt |
| M 4/6 | Linear regression with basis functions | Linear regression: Murphy 11-11.2 | Slides |
| W 4/8 | Training and test dataset split, cross validation | Cross validation: Murphy 4.5, 5.4 | Slides, Polynomial regression colab |
| F 4/10 | Bias variance trade-off | Bias variance trade-off: Murphy 4.7.6 | Slides, Bias variance colab |
| M 4/13 | Regularization | Ridge regression: Murphy 11.3-11.4 | Slides |
| W 4/15 | Regularization, sparsity, variable selection | Ridge regression: Murphy 11.3-11.4 | Slides |
| F 4/17 | LASSO | Lasso regression: Murphy 11.4 | Same as above, Ridge and LASSO colab, house_train_kaggle.csv |
| M 4/20 | Gradient descent, convexity | Gradient descent: Murphy 8-8.2.1 | Gradient descent slides, Convexity slides |
| W 4/22 | Gradient descent theoretical analysis | Gradient descent: Murphy 8-8.2.1 | Slides |
| F 4/24 | Stochastic gradient descent, prediction pitfalls | Stochastic gradient descent: Murphy 8.4-8.4.4 | Slides |
| M 4/27 | Classification, logistic regression | Logistic regression: Murphy 10-10.2.4, 10.3-10.3.3 | Slides |
| W 4/29 | Logistic regression, multi-class classification | Logistic regression: Murphy 10-10.2.4, 10.3-10.3.3 | Same as above |
| F 5/1 | Midterm | See exam page | |
| Non-parametric and non-linear methods | |||
| M 5/4 | Nearest neighbors | Nearest neighbors: Murphy 16.1 | Slides, Nearest neighbours demo |
| W 5/6 | Trees and bootstrap |
Trees: Murphy 18 Bootstrap: Efron and Hastie 10.2, 11-11.2 |
Trees slides, Bootstrap slides |
| F 5/8 | Cancelled | ||
| M 5/11 | Random forests and boosting; Kernels |
Random Forests: Murphy 18 Kernels: Bishop 6-6.2, Murphy 17, 17.1, 17.3.4, 17.3.9 |
Random forests slides, Kernels slides |
| Neural networks | |||
| W 5/13 | Cancelled | ||
| F 5/15 | Neural network basics; Back propagation | Neural Networks: Murphy 13-13.4.3 | Slides, Neural networks demo |
| M 5/18 | Convolutional neural networks | Convolutional neural networks: Chapter 7, 8 of Dive into Deep Learning, Zhang et al. | Slides |
| W 5/20 | Recurrent neural networks | Recurrent neural networks: Chapter 9, 10 of Dive into Deep Learning, Zhang et al. | Slides |
| F 5/22 | LSTM, language modeling | Recurrent neural networks: Chapter 9, 10 of Dive into Deep Learning, Zhang et al. | Slides |
| M 5/25 | No Class Memorial Day | ||
| W 5/27 | Attention and transformers | Attention mechanism: Chapter 11 of Dive into Deep Learning, Zhang et al. | Slides |
| Unsupervised learning | |||
| F 5/29 | K-means | K-means, GMM: Murphy 21.3-21.5 | Slides, K-means demo |
| M 6/1 | Gaussian Mixture Model (GMM) | K-means, GMM: Murphy 21.3-21.5 | Slides, GMM demo |
| W 6/3 | PCA | PCA, Autoencoders: Murphy 20.1, 20.3, 20.4.6, 22.1 | Slides |
| F 6/5 | (Optional) SVD, matrix completion | PCA, Autoencoders: Murphy 20.1, 20.3, 20.4.6, 22.1 | (Optional) Last year's slides, Last year's lecture recording: part 1, part 2 |
| TBD | Final Exam | See exam page | |