| Date | Content | Reading | Slides |
|---|---|---|---|
| Introduction to ML, Maximum Likelihood, Linear methods, Overfitting, Regularization, Optimization | |||
| 3/27 | Welcome/Overview, MLE |
Probability review: Murphy 2.1-2.4, 2.6, 2.8, 3.1-3.2 Statistics review, maximum likelihood: Murphy 4.2 |
slides, annotated slides |
| 3/29 | Maximum likelihood estimation |
Statistics review, maximum likelihood: Murphy 4.2 |
slides, annotated slides |
| 3/31 | Linear regression |
Linear algebra review: Murphy 7.1-7.3 Matrix calculus review: Murphy 7.8 Maximum likelihood regression: Murphy 4.2 Linear regression: Murphy 11-11.2 |
slides, annotated slides Linear regression demo .ipynb, .html, diabetes.txt Polynomial regression demo .ipynb, .html |
| 4/3 | Linear regression with basis functions |
Bias Variance trade-off: Murphy 4.7.6 |
slides, annotated slides |
| 4/5 | Bias Variance trade-off |
Bias variance trade-off: Murphy 4.7.6 |
slides for lecture 5-6
, annotated slides |
| 4/7 | Bias variance trade-off |
Bias variance trade-off: Murphy 4.7.6 |
slides for lecture 5-6
, annotated slides |
| 4/10 | Bias-variance demo, Cross validation |
Cross validation: Murphy 4.5, 5.4 |
slides, annotated slides Bias Variance tradeoff demo .ipynb, .html |
| 4/12 | Regularization | Ridge regression: Murphy 11.3-11.4 | slides, annotated slides |
| 4/14 | Sparsity, variable selection, LASSO | Lasso regression: Murphy 11.4 | slides, annotated slides |
| 4/17 | Gradient descent |
Gradient descent: Murphy 8-8.2.1 |
slides Gradient descent demo .ipynb, .html |
| 4/19 | Prediction pitfalls, stochastic gradient descent |
Gradient descent: Murphy 8-8.2.1 Stochastic gradient descent: Murphy 8.4-8.4.4 |
slides, annotated slides |
| 4/21 | Convexity |
Gradient descent: Murphy 8-8.2.1 Stochastic gradient descent: Murphy 8.4-8.4.4 |
slides, annotated slides |
| 4/24 | Classification, logistic regression |
Logistic regression: Murphy 10-10.2.4, 10.3-10.3.3 |
slides, annotated slides |
| 4/26 | Logistic regression, margin, support vector machine |
Logistic regression: Murphy 10-10.2.4, 10.3-10.3.3 Support vector machine: Murphy 17 - 17.3.3 |
logsitic regression slides , annotated logistic regression slides, SVM slides, annotated SVM slides |
| 4/28 | Support vector machine |
Support vector machine: Murphy 17.3 - 17.3.3 |
slides, annotated slides |
| 5/1 | Midterm | ||
| Non-linear methods | |||
| 5/3 | Kernels |
Kernels: Bishop 6-6.2, Murphy 17, 17.1, 17.3.4, 17.3.9 |
slides, annotated slides |
| 5/5 | Kernels | Kernels: Bishop 6-6.2, Murphy 17, 17.1, 17.3.4, 17.3.9 | slides, annotated slides |
| 5/8 | Bootstrap, neural network basics |
Bootstrap: Efron and Hastie 10.2, 11-11.2 Neural Networks : Murphy 13-13.4.3 |
slides on Bootstrap, annotated slides on Bootstrap slides on neural networks, annotated slides on neural networks |
5/10 | Back propagation |
Neural networks : Murphy 13-13.4.3 |
slides, annotated slides | 5/12 | Non-parametric methods, Nearest neighbors | Nearest neighbors: Murphy 16.1 | slides, annotated slides | 5/15 | More non-parametric methods, Tree-based | Trees, Random Forrests: Murphy 18 | slides, annotated slides | 5/17 | More non-parametric methods | Gradient Boosting Trees: Murphy 18 | slides, annotated slides |
| Unsupervised Learning | 5/19 | k-means, GMM | K-means, GMM: Murphy 21.3-21.5 | slides, annotated slides |
| 5/22 | PCA |
PCA, Autoencoders: Murphy 20.1, 20.3, 20.4.6, 22.1 |
slides, annotated slides |
| 5/24 | PCA, autoencoders |
PCA, Autoencoders: Murphy 20.1, 20.3, 20.4.6, 22.1 |
slides , annotated slides |
| Advanced Neural Networks | |||
| 5/26 | Matrix completion, convolutional neural networks |
Convolutional neural networks: Chapter 7,8 of Dive into Deep Learning, Zhang et al. |
slides , annotated slides |
| 5/29 | Memorial Day | ||
| 5/31 | Recurrent neural networks |
Recurrent neural networks: Chapter 9,10 of Dive into Deep Learning, Zhang et al. |
slides , annotated slides |
| 6/2 | Attention mechanism |
Attention mechanism: Chapter 11 of Dive into Deep Learning, Zhang et al. |
slides , annotated slides |