New Paradigms in AI

General Information

• Instructor: Pedro Domingos
• Time: Fridays, 2:30-3:30pm
• Location: Allen Center 403
• Credit: 1-3 Variable
• Grading: Credit / No Credit
• Mailing list: cse590m1_au26@uw.edu

Overview

Neurosymbolic AI
Key idea: To overcome neural networks' limitations we need to explicitly combine them with elements of symbolic AI.
Representative paper: DeepProbLog: Neural probabilistic logic programming

Predictive coding
Key idea: The brain learns by making predictions and letting each level model the previous one's errors.
Representative paper: Predictive coding in the visual cortex: A functional interpretation of some extra-classical receptive-field effects

Joint-embedding predictive architectures
Key idea: Predictions are best made from a latent representation of the input to a latent representation of the output.
Representative paper: Self-supervised learning from images with a joint-embedding predictive architecture

Recurrent and hierarchical reasoning models
Key idea: Deep reasoning does not require deep models or chain-of-thought, only well-controlled recurrence.
Representative paper: Less is more: Recursive reasoning with tiny networks

Vector symbolic architectures
Key idea: Binding, superposition and permutation of high-dimensional vectors suffice for massively parallel implementation of AI systems.
Representative paper: Vector symbolic architectures as a computing framework for emerging hardware

State space models
Key idea: Sequence models can run in linear time by maintaining a fixed-size latent state.
Representative paper: Mamba: Linear-time sequence modeling with selective state spaces

Diffusion models
Key idea: To generate realistic data, gradually add noise to training data and learn to remove it, then reverse the process.
Representative paper: Denoising diffusion probabilistic models

Tensor product representations
Key idea: Symbolic structures can be represented in neural networks by superpositions of tensor products of their elements.
Representative paper: The emergent symbolic structure of artificial neural networks

Fast weights
Key idea: Input-dependent weights have much higher storage capacity than neural activations but much faster dynamics than standard weights.
Representative paper: Linear transformers are secretly fast weight programmers

Tensor logic
Key idea: Neural and symbolic AI can be unified by viewing logical rules as Einstein summations over Boolean tensors.
Representative paper: Tensor logic: The language of AI.

Schedule

For accessibility assistance, please contact support@cs.washington.edu

UW Site Use Agreement