About the Course

When talking to some students, one of the questions I get the most is how to get started in research as an undergrad. This course is designed to be the first step towards in-depth understanding and rigorous analyses in both theoretical and empirical machine learning.

This course will cover advanced machine learning, from VC dimension to Generative AI. It will be divided into two parts: empirical and theoretical. In the first half of the course, we will cover the components and development of advanced GEnerative AI systems. Next, we will cover topics such as VC dimension, Rademacher complexity, ERM, generalization bounds, and optimization basics.

Prerequisites: Students entering the class should be comfortable with programming and should have a pre-existing working knowledge of linear algebra (MATH 308), vector calculus (MATH 126), probability and statistics (CSE 312/STAT390), and algorithms. Knowledge of machine learning at the level of CSE446 is highly recommended.

Past offering of this course: Ludwig Schmidt 2023, Sewoong Oh 2025 Spring, Jamie Morgenstern 2025 Autumn (focused only on theory)

Useful resources: Understanding Machine Learning by Shai Shalev-Shwartz and Shai Ben-David -- free pdf

Staff: Add pictures and names...

Lectures (To be updated)

Lecture time and place: Tuesdays, Thursdays 11:30 -- 12:50pm, MGH 241

• Lecture 1 (3/31): Introduction
• Lecture 2 (4/2): Tokenization: SuperBPE and Data Mixture Inference (lecture notes)
• Lecture 3 (4/7): Language model basics and architecture (lecture notes)
• Lecture 4 (4/9): Transformers (lecture notes)
• Lecture 5 (4/14): Speculative decoding and in-context learning (lecture notes)
• Lecture 6 (4/16): Chain-of-thought prompting (lecture notes)
• Lecture 7 (4/21): Parameter efficeint fine-tuning (lecture notes)
• Lecture 8 (4/23): Alignemnt (lecture notes)
• Lecture 9 (4/28): Mixture-of-Experts, test-time compute (lecture notes)
• Lecture 10 (4/30): TBD
• Lecture 11 (5/5): PAC learning and ERM (lecture notes)
• Understanding Machine Learning, Chapters 2 and 3
• Kevin Jamieson's amazing class in interactive learning covers similar proofs but with slides
• Lecture 12 (5/7): Uniform convergence, agnostically PCA learnable, Hoeffding's inequality (lecture notes)
• Understanding Machine Learning, Chapter 4
• Lecture 13 (5/12): Concentration of measure (lecture notes)
• Understanding Machine Learning, Appendix B
• Lecture 14 (5/14): No-free-lunch theorem, and bias-complexity trade-off, VC dimensions (lecture notes)
• Understanding Machine Learning, Chapters 5, 6
• Lecture 15 (5/19): VC dimensions (lecture notes)
• Understanding Machine Learning, Chapter 6
• Lecture 16 (5/21): VC dimension of linear predictors and perceptron algorithm (lecture notes)
• Understanding Machine Learning, Chapter 9
• Lecture 17 (5/26): Convexity, regularization, and stability analysis (lecture notes)
• Understanding Machine Learning, Chapter 12, 13
• Lecture 18 (5/28): Rademacher complexity and generalization bounds (lecture notes)
• Understanding Machine Learning, Chapter 26
• Lecture 19 (6/2): TBD
• Lecture 20 (6/4): Final project presentation
• Final replaced by Final project presentation(6/10, Wednesday) 4:30-6:20 pm, MGH 241

Office Hours

• Sewoong's Office Hours: TBD
• TA Office Hours: TBD

Assignments (To be updated)

We expect all assignments to be typeset (i.e., no photos or scans of written work) and submitted to this Link to Gradescope. Empirical Homework can be typeset using any editor like Microsoft Word or Latex, and Theory Homework should be Latexed.

• Due on April 30th Thrusday at 11:59pm: Empirical Homework (PDF, Starter Code). To be done in the same group as the project group. Submissions should be done by one person per group.
• Theory Homework (PDF, latex source)
• Due on May 14th Thursday at 11:59PM: Theory HW1 (Question 1 and Question 2)
• Due on May 21st Thursday at 11:59PM: Theory HW2 (Question 3 and Question 4)
• Due on May 28th Thursday at 11:59PM: Theory HW3 (Question 5)

Projects (To be updated)

The project will be about a replication of research, original empirical research, or a summarization of a line of theoretical work (and potential extension). There are three milestones for the project: (1) a proposal what you will work on, (2) version 1 which checks if you are on track to finish the project in time, (3) the final version which includes the full report.

• Resources:
  • Team Finding Sheet
  • Project instruction
  • Project template
• Deadlines:
  • Proposal: Tuesday, April 21st at 11:59 PM, (submit on gradescope)
  • Version 1: Tuesday, May 12th at 11:59 PM, (submit on gradescope)
  • Final version: Final project presentation is TBD, (submit the reoirt on gradescope). The final report is due on Friday, June 5th at 11:59 PM.
• Grading for the project is distributed as such: 10% for the proposal, 25% for version 1, and 65% for the final version. The project is 50% of total course project grade.

Grading

For students enrolled in CSE 493S, your grade will be determined by:

• 25% Empirical homework
• 8% Theory homework 1
• 8% Theory homework 2
• 9% Theory homework 3
• 50% final project

For students enrolled in CSE 599, your grade will be determined by:

• 20% Empirical homework
• 7% Theory homework 1
• 7% Theory homework 2
• 7% Theory homework 3
• 50% final project
• 9% scribing the theory notes

Where to get help

• EdStem discussion board:
  • Public/Anonymous Posts
    • Questions like, "Is there a typo in the homework?", "What does this notation mean?", "Is this an accurate description of how this works?".
    • Questions that are not of a personal nature should be posted to the discussion board.
  • Private Posts
    • Questions involving your own code should be posted privately to the EdStem discussion board, not office hours.
    • Personal concerns (like "I was in the hospital", "Laptop was stolen").
• Course staff email cse493s-staff@cs.washington.edu
  • Personal concerns (like "I was in the hospital", "Laptop was stolen") you aren't comfortable sharing with the entire staff. We highly recommend posting privately to EdStem if you are comfortable.
  • Please direct all course-related inquiries to cse493s-staff@cs.washington.edu or EdStem. Please do not email the instructors or TAs individually.
• Submit anonymous feedback here.