---

Discussion:

Discord
Slack

---

Topics:

• Fundamental Methods
  • Introduction to AI Agents & Signal Basics
  • Understanding Waves
  • Android/iPhone Programming
  • Using Microphones/Speakers as Sensors
  • Multi-Microphone/Speaker Applications
  • Deep Learning for Enhanced Hearing
  • Programming Models on Phones
  • IMU & GPS systems
• AI & Agentic Systems
  • Real-time Translation systems
  • Proactive Agents
  • LLMs as Judges for agentic systems
  • Augmenting LLMs with Tool Use & Actions
  • AI Scientist systems
  • Benchmarks & Challenges in Evaluating Agents
  • GUI Agents

---

Course schedule:

Date	Topic	Notes
3/31	Introduction and Signal Processing Basics	Slide 1 Additional Readings: Signal and Systems course Video on Aliasing and Nyquist Video on negative frequencies
4/2	Introduction and Signal Processing Basics (cont.)	IMU Slides GPS Slides
4/7	Android and iOS development tutorial	Sample codes - Android Doc and sample codes - iOS iOS+Swift Youtube Tutorial Android Youtube Tutorial
4/9	Understanding Waves	Slide 2
4/14	Using microphones/speakers as sensors
4/16	Using multiple microphones/speakers	BeepBeep: a high accuracy acoustic ranging system using COTS mobile devices Underwater 3D positioning on smart devices
4/21	Deep learning for Enhanced hearing
4/23	Programming models on phones
4/28	Mobile Agents	Papers: Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perception Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaboration
4/30	Mobile Agents	Papers: Mobile-Agent-E: Self-Evolving Mobile Assistant for Complex Tasks PC-Agent: A Hierarchical Multi-Agent Collaboration Framework for Complex Task Automation on PC
5/5	Web Agents	Papers: WE BAR E N A : A REALISTIC WEB ENVIRONMENT FOR BUILDING AUTONOMOUS AGENTS WorkArena: How Capable are Web Agents at Solving Common Knowledge Work Tasks?
5/7	Project brainstorming and proposal	Papers: MemGPT: Towards LLMs as Operating Systems A-MEM: Agentic Memory for LLM Agents
5/12	Proactive agents Presentations (3): XXX (15 minutes) XXX (15 minutes) XXX (15 minutes)	Sign up for presentations!
5/14	LLM as a judge Presentations (3): XXX (15 minutes) XXX (15 minutes) XXX (15 minutes)	Sign up for presentations!
5/19	Augmenting LLMs with Tool use/Actions Presentations (3): XXX (15 minutes) XXX (15 minutes) XXX (15 minutes)	Sign up for presentations!
5/21	AI Scientist Presentations (2): XXX (15 minutes) XXX (15 minutes)	Sign up for presentations!
5/26	Benchmarks & Challenges in Evaluating Agents Presentations (3): XXX (15 minutes) XXX (15 minutes) XXX (15 minutes)	Sign up for presentations!
5/28	GUI Agents Presentations (2): XXX (15 minutes) XXX (15 minutes)	Sign up for presentations!
6/4	Project Presentations
6/9	Project Presentations

---

Assignments:

Implement head tracking using smartphone IMUs Implementing head tracking using smartphone IMUs: Assignment 1 (Due 4/21/2025)
Implementing Yolo video model on smartphone: Assignment 2 (Due 5/9/2021)

---

Project (40% total):

• Project pitch/proposal (5%)
• Project demo (30%)
• Evaluation metrics (5%)

---

Grading:

Late policy: Assignments may be handed in up to ten days late, at a penalty of 10% of the maximum grade per day.

Homework 1 (20%)
Homework 2 (20%)
Final Project (40%)
Class participation (5%)
Class presentation of one paper (15%)