CSE 599J | Social Reinforcement Learning

Important dates for final projects

Submit your project abstract to OpenReview by December 7, 2026.
Submit your review bids by December 9, 2026.
Final paper deadline: December 9, 2026 at 11:59 PM.
Peer reviews due: December 16, 2026 at 11:59 PM.

Schedule and Details

Instructor: Natasha Jaques (nj@cs.washington.edu)

TAs:
Kunal Jha (kj@cs.washington.edu)
Yanming Wan (ymwan@cs.washington.edu)

Lecture: MW, 1:30 PM - 2:50 PM (80-minute lectures)
Location: SIG 134
Class meetings: September 30 - December 9, 2026 (no class November 11)

Remote attendance: There is no remote attendance option, although lecture recordings will be available via Canvas.

Office hours:
Natasha Jaques: Wednesdays, 9:00 AM - 10:00 AM, Gates 234
Kunal Jha: Tuesdays, 10:30 AM - 11:30 AM, Gates 277
Yanming Wan: Mondays, 4:00 PM - 5:00 PM, Gates 377

Important Links

Main spreadsheet (detailed schedule, readings, and paper presentation sign-up) (link)
Schedule and lecture slides (link)
EdSTEM discussion board (link)
Gradescope (link)
Project team matching (link)

Schedule overview (by week):

1. Intro and Deep RL crash course
2. RL from Human Feedback (RLHF) (for LLMs)
3. RL post-training
4. Learning from humans beyond LLMs
5. Multi-agent RL
6. Human-AI coordination
7. Guest lecture & break
8. Social learning
9. Emergent Complexity
10. Multi-agent RL for LLMs
11. Final project presentations

Class Format

Classes will be split so that on approximately half the days Natasha will present a ~50 minute lecture followed by ~30 minutes of questions and discussion, and on the remaining days we will have student-led paper presentations and discussions.

Each student in the class will sign up to present in one of the slots in the course schedule. They will prepare a 5-minute presentation on one of the papers for that discussion day. Each presentation will be followed by 1 minute of clarifying questions. After all the presentations finish, we will discuss each of the papers as a class, the themes that connect them, and interesting and impactful research directions that relate to them.

Grading

Class participation (15%)

Because this is primarily a discussion course, to make it work we need students to attend class in person, ask questions, and participate in paper discussions. Therefore, a portion of your grade depends on doing this. To get the full 15%, you need to read the relevant papers, show up on time, and make comments or ask questions in at least 17/20 classes (you are allowed to miss 3; see the Course Policies section). However, we hope you will go beyond this minimum requirement and make the most of the class by actively participating and sharing your thoughts and questions on the research we are learning about.
Paper reflections (10%)

We will have 8 Discussion classes for which you will prepare a 300-word paper reflection on one of the suggested papers, to be uploaded to EdSTEM. Reflections will be graded on a pass/fail basis. As per the Course Policies, you can miss submitting 1 reflection with no penalty. Presenters still need to submit reflections for the papers they are presenting. We are aware that it would be extremely easy to generate these summaries with an LLM. If you want to learn something from the course, you’ll need to actually read the papers. If you are not planning to read the papers, it would be better to drop and give up your spot to one of the other students petitioning to get in.
Lead discussion (10%)

During the quarter, every student is expected to present a paper at least once. Use the class schedule spreadsheet to sign up for a particular presentation time, and choose the paper you would like to present. Add your presentation slides to the shared spreadsheet by at least 11:59 PM two days before class.
Course project (50%)

Proposal (5%): due 11:59 PM October 28, 2026
Writeup (35%): due 11:59 PM December 9, 2026
Project presentation (10%): in class on December 7 and December 9

See the Class Project section for more information.
Peer review (15%)

Peer review is a big part of research, and in this class we will learn how to write high-quality peer reviews. We will upload our course projects to OpenReview. You will be responsible for submitting 3 reviews of other students’ papers, which will be due December 16, 2026 at 11:59 PM. This review procedure will be single-blind, since students will have seen the class project presentations. Each review will be worth 5% of your grade and will be graded based on whether it is thorough, complete, fair, and whether it gives the authors useful feedback for improving their paper. Peer-review feedback will not be used to determine the final grade for students’ papers; they will be graded by the instructors.

Course Policies

Late submissions and absences

To reduce the burden on instructors, create flexibility for students, and maintain consistent treatment across students, we will allow you to drop 1 of the 8 paper reflections with no penalty. Your participation grade will also be based on participating in 17/20 class meetings, so you can miss 3 classes with no penalty. We understand that sometimes you may need to miss class due to travel obligations or other circumstances. Our goal is that for most of these issues you do not need to contact us. It will not affect your grade unless you go beyond 3 classes.
Late assignments will not be graded by default. The due dates have been set to being as late as possible while allowing for feedback on the presentation slides and prompt grading of the course project.

Inclusion, feedback, accommodations, and other policies

DEI: This course welcomes all students of all backgrounds. The computer science and computer engineering industries have a significant lack of diversity. This is due to a lack of sufficient past efforts by the field toward facilitating diversity, equity, and inclusion. The Allen School seeks to create a more diverse, inclusive, and equitable environment for our community and our field. You should expect and demand to be treated by your classmates and myself with respect.
- If any incident occurs that challenges this commitment to a supportive, diverse, inclusive, and equitable environment, please let me know so the issue can be addressed. You can use the course anonymous feedback form. Supporting DEI is a process that requires continual learning and growth, and I value your feedback on how we can improve the course on this dimension.
- You can also submit feedback through the Allen School’s anonymous feedback form: https://feedback.cs.washington.edu/
Generative AI: Generative AI tools can help with research, but if you rely on them too heavily it will inhibit your ability to gain useful skills from taking this course, which is the whole point. Also, I can tell when writing is AI-generated and will mark you down for something that sounds too LLM-sloppy. For writing your paper reflections, do not use AI to generate them. We want evidence that you've actually read the paper, and would prefer sloppy jot notes to an AI-generated summary. If we think your summary sounds AI-generated, we will run it through an AI detector. If the detector indicates that more than 50% of your summary was AI-generated, then we will dock you points proportional to the detected AI-generated percentage. E.g. if the detector says it's 90% AI, you will receive 10% on the summary reflection grade.
DRS accommodations: Embedded in the core values of the University of Washington is a commitment to ensuring access to a quality higher education experience for a diverse student population. Disability Resources for Students (DRS) recognizes disability as an aspect of diversity that is integral to society and to our campus community. DRS serves as a partner in fostering an inclusive and equitable environment for all University of Washington students. The DRS office is in 011 Mary Gates Hall. Please see the UW resources at http://depts.washington.edu/uwdrs/current-students/accommodations/. If you have DRS accommodations that the course staff should know about, please contact us at the beginning of the course.
Religious accommodations: Washington state law requires that UW develop a policy for accommodation of student absences or significant hardship due to reasons of faith or conscience, or for organized religious activities. The UW’s policy, including more information about how to request an accommodation, is available at Religious Accommodations Policy. Accommodations must be requested within the first two weeks of this course using the Religious Accommodations Request form".
Sexual harassment: University policy prohibits all forms of sexual harassment. If you feel you have been a victim of sexual harassment or if you feel you have been discriminated against, you may speak with your instructor, teaching assistant, the chair of the department, or you can file a complaint with the UW Ombudsman’s Office for Sexual Harassment. Their office is located at 339 HUB, (206)543-6028. There is a second office, the University Complaint Investigation and Resolution Office, who also investigate complaints. The UCIRO is located at 22 Gerberding Hall. Please see additional resources at the UW office of Ombud.
Land acknowledgement: The University of Washington acknowledges the Coast Salish peoples of this land, the land which touches the shared waters of all tribes and bands within the Suquamish, Tulalip and Muckleshoot nations.
Resources: For additional resources, see CSE Students and Student Resources.
EdSTEM: We encourage you to submit questions on EdSTEM to get help from other students.

Acknowledgements