CS422: Interactive and Embodied Learning
Winter 2026
Instructor: Nick Haber
Office hours: by appointment
Meeting times: Mo 3-5:50
Location: Green Earth Sciences 131
Course Description
The scaling of self-supervised learning and post-training techniques has brought dramatic success. Yet if we look at our current mental model of human learning (representation learning, but also interactive learning, motivation, very long-context memory...), recent research gives us signals suggesting that the dominant paradigm for scaling is likely far from optimal. In the 2026 class, we’ll look at interesting pretraining ideas, the role of memory, play, and social learning. Class will be a seminar, with discussion. Students will read and take turns presenting current works, and they will produce a proposal of a feasible next research direction. Topics build upon reinforcement learning techniques and include intrinsic motivation, model-based RL, self-supervised representation learning, multi-agent RL, human-AI interaction, and emergent communication.
The primary goal of this course is to provide a survey of techniques related to agents learning rich models of their world through interaction. Readings and discussions are planned with an eye towards germinating new work, and the project component should serve as a way to get started. The course is suitable for students looking to dive into recent works (from e.g. NeurIPS, ICML, and ICLR). Students should walk away from the course understanding not only relevant state-of-the-art approaches, but also its limitations, gaps in our understanding, benchmarks waiting to be created, and useful methods waiting to be fruitfully combined. This is an exciting, rapidly-growing field with many opportunities to quickly get started!
As a seminar in which we continually update readings, this course is intended to be suitable for repeated enrollment. See the 2021 offering here, 2022 offering here, and 2024 offering here.
Prerequisites
- Willing to read current AI literature (from e.g. NeurIPS, ICML, ICLR)
If not experienced in this, at least at a point where you're willing to learn this skill. - Proficiency in Python, and ideally Tensorflow or PyTorch
No assignments require this, but you will get a great deal more out of the course (and potentially make more compelling presentations and project) if you can e.g. look over a paper's github repo and play. - Knowledge in linear algebra, calculus, and statistics (MATH 19, 41, or 51, CS 109, or equivalent)
Again, no assignments require this, but you will get a great deal more out of readings if you are comfortable with these concepts. - CS 229, 231N, 234, or equivalent
Deep reinforcement learning will play a key role.
Class time
Winter quarter, 2026Mo 3-5:50
Forum, project submissions
Visit our Canvas page.Grading
Project 40%Presentations 30%
Discussion participation 30%
Student responsibilities
Students will read required papers for each class, and students/groups of students will take turns presenting. Expect to present twice, as part of a group, this quarter. Note: each class has a required paper as well as a number of supplementary papers. The supplementary lists can get quite long -- this is not meant to be imposing! We simply think that each represents a handy constellation of papers for those interested in getting more acquainted with each subarea. We will try to provide a roadmap as we go, and feel free to ask for more context!
Students will work on projects in which they will scope a novel research project appropriate to the course material. For details and project milestones, see our project timeline.