Schedule

This schedule is tentative and subject to change. See the detailed course schedule for readings and paper-presentation sign-ups.

WEEKTOPICDATELECTURE/DISCUSSION
1 Intro and Deep RL crash course 09/30 Intro and Deep RL crash course — Lecture
2 RL from Human Feedback (RLHF) (for LLMs) 10/05 RL post-training history and RLHF intro — Lecture
10/07 RLHF, post-training dynamics, and safety — Students present papers and discuss
3 RL post-training 10/12 RL post-training, Personalized RLHF — Lecture
10/14 RLVR and technical aspects of RL post-training — Students present papers and discuss
4 Learning from humans beyond LLMs 10/19 Learning from humans beyond LLMs — Lecture
10/21 PbRL, robot learning from human feedback, and instruction following — Students present papers and discuss
5 Multi-agent RL 10/26 Deep multi-agent RL intro — Lecture
10/28 State-of-the-art MARL, theory, fun MARL papers, and social skills like theory of mind — Students present papers and discuss
6 Human-AI coordination 11/02 Zero-shot coordination with humans — Lecture
11/04 Human-AI coordination (both embodied and for LLMs), population-based training, human simulation — Students present papers and discuss
7 Guest lecture & break 11/09 Guest lecture by Louis Kirsch of Inherent Labs — RL post-training with long-horizon / non-verifiable rewards for scientific research
11/11 Veterans Day — No class
8 Social learning 11/16 Social Learning — Lecture
11/18 Social Learning — Students present papers and discuss
9 Emergent Complexity 11/23 Multi-agent Emergent Complexity — Lecture
11/25 Emergent Complexity and Open-Endedness — Students present papers and discuss
10 Multi-agent RL for LLMs 11/30 MARL for LLMs — Lecture
12/02 MARL and emergent complexity for LLMs — Students present papers and discuss
11 Final project presentations 12/07 Final project presentations
12/09 Final project presentations