CSE/STAT 416 - Intro to Machine Learning

Regression

• 0.Overview

• 1.Regression

• 2.Validation

• 3.Regularization

• 4.Non-quadratic regularizers

• 5.Non-quadratic loss

Lectures:

• Lecture 1. Tues, April 2: Course overview, Regression.
  

[0.Overview] [1.Regression annotated] [ML pipeline demo, and html]

Optional Readings: [ESL, Sections 1, 2.3.1] [Overview slides from 2018Sp] [Regression slides from 2018Sp]

• Lecture 2. Thurs, April 4: Regression, Validation.
  

[2.Validation annotated] [polynomial fit demo, and html]

Optional Readings: [ESL, Sections 2.3.1] [Assessing performance slides from 2018Sp]

• Section 1. Thurs, April 4: Gradient descent.
  

[Gradient descent slides] [Gradient descent annotated slides] [section slides by Joshua]

Optional Readings: [ESL, Sections 4.5.1]

• Lecture 3. Tues, April 9: Validation, Regularization.
  

[3.Regularization annotated] [Ridge regression demo, and html]

Optional Readings:[ESL, Sections 3.1-3.2, 7.1-7.2] [ESL, Sections 7.4, 3.4.1] [Ridge regression slide rom 2018Sp]

• Lecture 4. Thurs, April 11: Regularization, Non-quadratic regularizers.
  

[4.Non-quadratic regularizers annotated] [Lasso regression demo and html]

Optional Readings: [ESL, Sections 2.9, 5.5.2, 7.2] [ESL, Sections 3.4.2, 7.10] [Lasso regression slides from 2018Sp]

• Section 2. Thurs, April 11: Assignment 1 tips, Bias variance tradeoff.
  You can view the notebooks by viewing the HTML version here. To try out the demo yourself, download the ipynb and data files.
  [Bias-Variance demo (html | ipynb)]
  [Pandas demo ipynb] [Philadelphia_Crime_Rate data]

Optional Readings: [ESL, Section 7.3]

• Lecture 5. Tues, April 16: Non-quadratic losses.
  

[5.Non-quadratic loss slides] [5.Non-quadratic loss annotated slides] [non-quadratic loss demo] and [html]

• Section 3. Thurs, April 18: Visualizing Ridge and LASSO.
  [why lasso gives sparse solutions slides] [lasso slides from 2018Sp] [Ridge video] [Lasso video]

Optional Readings: [ESL, Sections 3.4.1-3.4.3, 3.6]

Classification

• 6.Classification

• 7.Over-confident predictions

• 8.Decision trees

• 9.Boosting

• 10. Precision and Recall

Lectures:

• Lecture 6. Thurs, April 18:
  

[6.Classification slides], [6.Classification annotated slides] [Classification demo] and [html]

Optional Readings: [ESL, Sections 1, 2.3.1, 4.1-4.2]] [Classification slides from 2018Sp]

• Lecture 7. Tues, April 23:
  

[7.Overconfident predictions slides]

Optional Readings: [ESL, Sections 4.4.1-4.4.4, 9.1.2, 7.5-7.6] [Logistic regression slides from 2018Sp]

• Lecture 8. Thurs, April 25:
  

[8.Decision tree slides]

Optional Readings: [ESL, Sections 9.2.1-9.2.3] [Decision tree slides form 2018Sp]

• Section 4. Thurs, April 25: MLE and dealing with categorial features
  

[Classification review slides]

Optional Readings: [ESL, Sections 2.6, 8.2, 2.2, 9.2.4]

• Lecture 9. Tues, April 30:
  

[9.Boosting slides]

Optional Readings: [ESL, Sections 9.2.4, 10.1-10.10] [Deriving AdaBoost formula] [Explaining AdaBoost (Schapire 2013)] [Boosting slides 1 from 2018Sp] [Boosting slides 2 from 2018Sp]

• Lecture 10. Thurs, May 2:
  

[10.Precision and Recall slides] [10.Precision and Recall annotated slides]

• Section 5. Thurs, May 2: Tree ensembles.
  

[Tree ensemble slides]

Optional Readings: [ESL, Sections 11.9.1, 15.1-15.4]

Clustering and Similarity

• 11. Nearest Neighbor Search

• 12. Nearest Neighbor methods

• 13. Clustering

• 14. Dimensionality reduction

Lectures:

• Lecture 11. Tues, May 7:
  

[11.Nearest Neighbor Search slides] [11.Nearest Neighbor Search annotated slides]

Optional Readings: [Supplementary slides on advanced LSH and KD-trees]

• Lecture 12. Thurs, May 9:
  

[12.Nearest Neighbor Methods slides]

Optional Readings:

Optional Readings: [ESL, Sections 2.3.2, 2.5, 2.8.2, 6.1-6.3, 13.3]

• Section 6. Thurs, May 9: Missing data.
  

[Slides]

Optional Readings: [Sections 8.5.2, 9.6]

• Lecture 13. Tues, May 14:
  

[13.Clustering slides]

• Lecture 14. Thurs, May 16:
  

[Video recording of lecture 14 on Hierarchical Clustering (800MB)] [smaller size version (420MB)]

[13.clustering annotated]

• Section 7. Thurs, May 16: Clustering
  

[Spcetral clustering slides]

Recommending Products

• Recommender systems, performance metrics

Lectures:

• Lecture 15. Tues, May 21:
  

[14.Dimensionality Reduction slides] [Principal Component Analysis demo] and [html]

• Lecture 16. Thurs, May 23:
  

CANCELLED.

• Section 8. Thurs, May 23: Matrices, PCA, and coordinate descent.
  

GroupBy [html] and [ipynb]

• Lecture 17. Tues, May 28:
  

[15.Recommendation Systems slides]

Deep Learning: Searching for Images

• 16. Deep learning

• 17. Generative Adversarial Networks

Lectures:

• Lecture 18. Thurs, May 30:
  

[16. Deep learning slides]

Optional Readings: [General neural networks][Convolutional neural networks][Computing any function]

• Section 9. Thurs, May 30: Deep Learning
  

• Lecture 19. Tues, June 4:
  

[17. Generative Adversarial Networks slides]

• Lecture 20. Thurs, June 6:
  

• Section 10. Thurs, June 6: Marathon office hours
  

• Final Exam. Monday, June 10, 10:30

[Summary slides] [Practice final] [Solution to practice final]