Robot Learning (CMP 728)

Course Description

This graduate-level course introduces the foundations of robot learning, bridging robotics, machine learning, sequential decision making, and modern foundation models. Students will study imitation learning, reinforcement learning, perception for robotics, world models, Vision-Language-Action (VLA) systems, diffusion policies, and agentic robotics. The course emphasizes both conceptual understanding and project-based implementation using modern robotics simulators and benchmarks.

Tentative List of Topics

  • Introduction to Robot Learning
  • Robot Foundations and Kinematics
  • Markov Decision Processes (MDPs)
  • Behavior Cloning
  • Imitation Learning and DAgger
  • Optimal Control and Model-Based Reinforcement Learning
  • Deep Reinforcement Learning (PPO, SAC, Policy Gradients)
  • Vision for Action and Robotic Perception
  • SLAM, NeRF, and 3D Scene Representations
  • Multimodal Robot Learning
  • Vision-Language-Action (VLA) Models
  • JEPA and World Models
  • Diffusion Policies
  • Agentic Robotics and LLM-based Planning
  • Code-Generating Robot Agents
  • Project Presentations

Prerequisites

Basic Python programming is expected. Prior coursework in machine learning, robotics, probability, linear algebra, or computer vision is helpful but not strictly required. Students should be comfortable reading recent research papers.

Learning Outcomes

  • Formulate robot learning problems using MDPs and sequential decision-making frameworks.
  • Understand imitation learning and reinforcement learning algorithms.
  • Analyze perception and representation learning methods for robotics.
  • Understand modern VLA systems, world models, JEPA, and diffusion policies.
  • Design, implement, and evaluate a robot learning project.
  • Critically read and discuss contemporary robot learning research.

Project

Students will complete an individual or team-based semester project. Projects may focus on reinforcement learning, imitation learning, diffusion policies, world models, multimodal robot learning, VLA systems, agentic robotics, or related topics. Implementations are expected to use simulation platforms such as Isaac Lab, Isaac Sim, MuJoCo, ManiSkill, PyBullet, or Gymnasium Robotics.

Grading

Reading & Video Quizzes: 10%
Project Proposal: 5%
Project Progress Review: 10%
Final Project Report and Demonstration: 35%
Final Examination: 40%

Suggested References

Reinforcement Learning: An Introduction, Sutton & Barto.
Modern Robotics, Lynch & Park.
Probabilistic Robotics, Thrun, Burgard & Fox.
Planning Algorithms, Steven M. LaValle.
Recent papers from CoRL, RSS, ICRA, IROS, NeurIPS, ICML, and CVPR.

Expectations

Students are expected to attend lectures regularly, complete assigned readings, participate in discussions, and make steady progress on project milestones. Since the topics build on each other throughout the semester, continuous engagement is strongly encouraged.