CSE 541: Interactive Learning

About the course

The traditional approach to machine learning uses a training set of labeled examples to learn a prediction rule that will predict the labels of future examples. Collecting such training sets can be expensive and time-consuming. This course explores methods that use already-collected data to guide future measurements, in a closed loop, to best serve the task at hand. We focus on two paradigms:

• Pure exploration: algorithms that identify or learn a good-enough model using as few measurements as possible (e.g., classification, drug discovery, science).
• Regret minimization: algorithms that balance taking measurements to learn a model against exploiting that model to obtain high-reward outcomes (e.g., content recommendation, medical treatment design, ad serving).

The literature on interactive (machine) learning has exploded in recent years and can be overwhelming. This course classifies interactive learning problems by characteristics such as the hypothesis space, the available actions, the measurement model, and the available side information. We focus on general algorithmic strategies and common proof techniques. By the end of the course, you will be in a position to begin research in this field, or to lead an interactive-learning software implementation in industry.

Review materials

• Linear Algebra Review by Zico Kolter and Chuong Do
• Linear Algebra by David Cherney, Tom Denton, Rohit Thomas and Andrew Waldron (introductory text)
• Probability Review by Arian Maleki and Tom Do; see also Chapter 5 of [Lattimore–Szepesvári]

Class materials

Lecture slides (PDF) are posted on the schedule before each class. Bring them to lecture if you like to annotate. The course also draws on these texts and notes:

• Lattimore–SzepesváriBandit Algorithms, Tor Lattimore and Csaba Szepesvári
• JamiesonInformal lecture notes on bandits, Kevin Jamieson
• Rakhlin–SridharanStatistical Learning and Sequential Prediction, Alexander Rakhlin and Karthik Sridharan
• Foster–RakhlinFoundations of Reinforcement Learning and Interactive Decision Making, Dylan Foster and Alexander Rakhlin

Grading and policies

Grading

Three homeworks (10% each) and two projects, one at the midterm and one at the end of the quarter (35% each).

Projects

There are two projects, one due at the midterm and one at the end of the quarter. For each, you pick a topic that extends a tool from the course and explain it to your classmates. Each project has two parts:

• A 5-minute YouTube video (±30 seconds; it can be unlisted). Slides with a voiceover, a whiteboard talk, or an animated explainer are all fine. Production quality isn’t graded. Clarity and accuracy are, and at least one big idea should come across clearly. Please add captions; YouTube’s auto-captions, corrected where they’re wrong, are fine.
• A 2–3 page writeup in LaTeX, submitted as a PDF on Gradescope. It introduces the problem, surveys the relevant papers, states results correctly, and identifies what is open.

Share both in an Ed post titled “[Your name]: [Project title]”. Projects are individual (one per person), and topics and the full rubric are in each project’s handout. The late policy does not apply to projects, so plan ahead.

Collaboration policy

With classmates

Homeworks must be done individually: each student hands in their own answers and writes their own code. It is fine to collaborate with classmates in figuring out answers and helping each other solve problems, but list the people you collaborated with on each homework. If you find yourself copying and pasting code, LaTeX, or anything else from another student, you have crossed the line.

With AI: you supply the algorithm, AI can help you type it

You may use AI to turn an algorithm you have fully specified into code: the update rules, what is sampled, what is observed, and what is computed. You may not ask AI to supply the algorithm itself. Naming the method and asking the AI to “implement it” hands over exactly the thinking the homework is meant to exercise.

For code, a rule of thumb: if someone who had never heard of the algorithm could write the code from your prompt alone, you’re on the right side of the line. Routine code, such as making well-formatted plots, loading data, or debugging your own implementation, is always fine. Whatever you use AI for, say so in your submission (see the AI policy).

With AI on proofs: keep it minimal

For proof-based questions, use AI minimally: to clarify a definition or concept, to brainstorm possible approaches, or to help with LaTeX formatting. Do not ask AI to write, complete, or fix your proof. The argument and its details should be yours.

The homework problems have been chosen for their pedagogical value and may be similar or identical to those given in past offerings of this course at UW or similar courses elsewhere. Using pre-existing solutions from these sources, the web, or textbooks violates the academic integrity expected of you and is strictly prohibited. This includes asking an AI tool to find or reproduce such solutions.

Submitting homework

Submit each homework as a single, typeset PDF to Gradescope (no photos or scans). You are enrolled in Gradescope automatically; email us if you don’t see the course. Code for a programming problem goes at the end of that problem, after any requested figures. LaTeX is strongly recommended, and there are convenient packages for listing Python code.

LaTeX resources: Overleaf (online editor) · Learn LaTeX in 30 minutes · Math symbols · Detexify (draw a symbol to find it) · Install LaTeX locally · Gradescope help

Discussion and communication

Ed is our discussion board and your first stop for questions (registered students receive an invite; otherwise email the instructor). We will not use the Canvas discussion board. For private matters, email cse541-staff@cs.washington.edu or the instructor directly. You can also send anonymous course feedback, though I can’t reply to it personally.

Access and accommodations

Your experience in this class is important to me. It is the policy and practice of the University of Washington to create inclusive and accessible learning environments consistent with federal and state law.

• Already have accommodations with DRS? Activate them for CSE 541 through myDRS early in the quarter so we can talk about how they will work in this course (for example, extended time or alternative formats).
• Not yet registered? If you have a temporary health condition or permanent disability that requires accommodations (including, but not limited to, mental health, attention-related, learning, vision, hearing, physical or health impacts), contact Disability Resources for Students (DRS) directly to set up an Access Plan. DRS facilitates the interactive process that establishes reasonable accommodations.

You don’t need to share your diagnosis with the course staff. Reach out to the instructor or cse541-staff@cs.washington.edu anytime with questions about access in this course.

Assignments

Homeworks and project handouts are posted here when released. Tentative due dates are on the schedule. Everything is due at 11:59 PM Pacific.

Schedule

Lectures are added here as we go, each with its slides posted before class. Due dates are tentative. [Jamieson] = the course notes.