Week 01
Introduction to Deep Learning
Course overview and the motivation, history, and core ideas behind deep learning.
Lecture
Monday · 09:30–12:30
D5
Course information
About the course
This course provides a thorough understanding of the fundamental concepts and recent advances in deep learning. The main objective is to provide students practical and theoretical foundations to use and develop deep neural architectures to solve challenging tasks in an end-to-end manner. The course is taught by Erkut Erdem.
The course will use Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville as the textbook, with the draft available online for free at deeplearningbook.org.
Instruction style. During the semester, students are responsible for studying and keeping up with course material outside class time. This may involve reading book chapters, papers or blogs and watching video lectures. After the first four lectures, each week a group of students will present a research paper related to the topics of the week.
Communication. Lecture notes, presentations, assignments and important deadlines will be posted on this website. Other course-related communication will be handled through Ed. Students may use the existing course join link with a departmental email account.
Preparation
Prerequisites
CMP784 is open to graduate students in the CMPE and AIDE departments. Prospective senior undergraduate students may sit in on the class. Non-CMPE/AIDE graduate students should ask the course instructor for approval before the add/drop period.
- Programming: proficiency sufficient to complete the practicals and implement the course project.
- Calculus and linear algebra: differentiation, chain rule, vectors, matrices, eigenvalues and eigenvectors.
- Probability and statistics: random variables, expectations, multivariate Gaussians, Bayes rule and conditional probabilities.
- Machine learning: strongly recommended. Relevant introductory courses include AID202 Foundations of Machine Learning, CMP472 Introduction to Machine Learning and CMP712 Machine Learning. CMP684 Neural Networks is also closely related.
- Optimization: cost functions, gradients and regularization.
Assessment
Course requirements and grading
Assessment
Math prerequisites quiz2%
Practicals · 2 × 8%16%
Final exam25%
Course project32%
Course engagement
Paper presentations15%
Weekly quizzes5%
Class participation5%
Class participation reflects preparation and substantive contribution to in-class discussions, including questions and comments during lectures, paper discussions, and project sessions. The math prerequisites quiz must be completed and passed by every student.
Fall 2026
Weekly schedule
Week 02
Machine Learning Overview
A compact review of the machine-learning concepts needed throughout the course.
Week 03
Multi-Layer Perceptrons
Feed-forward neural networks, backpropagation, representations, and learning.
Week 04
Training Deep Neural Networks
Optimization, regularization, initialization, normalization, and training dynamics.
Week 05
Convolutional Neural Networks
Convolutional architectures and their use in visual recognition.
Week 06
Understanding and Visualizing CNNs
Interpreting learned representations and analyzing convolutional networks.
Week 07
Recurrent Neural Networks
Sequence modeling, RNNs, LSTMs, GRUs, and recurrent computation.
Week 08
Attention and Transformers
Attention mechanisms, self-attention, transformer architectures, and applications.
Week 09
Autoencoders and Deep Generative Models
Representation learning, autoencoders, and foundations of deep generative modeling.
Week 10
Progress Presentations
Project groups present their progress, preliminary results, and next steps.
Week 11
Deep Generative Models · continued
Further study of modern deep generative models.
Week 12
Deep Generative Models · continued
Continued discussion of generative modeling methods and applications.
Week 13
Self-supervised Learning
Pretext tasks, contrastive learning, and representation learning without manual labels.
Week 14
Final Project Presentations
Final in-class presentations and discussion of course projects.