BBM 406: Introduction to Machine Learning
Spring 2016
A scene from the historical Go game where a computer program, Google DeepMind's AlphaGo (white stones), defeated a professional player, Europe's Go champion Fan Hui (black stones), first time ever. Photograph: Google DeepMind
Course Information
Course Description
This is a undergraduate-level introductory course in machine learning (ML) which will give a broad overview of many concepts and algorithms in ML, ranging from supervised learning methods such as support vector machines and decision trees, to unsupervised learning (clustering and factor analysis). The goal is to provide students with a deep understanding of the subject matter and skills to apply these concepts to real world problems.
Time and Location
Lectures: Tuesday at 10:00-10:50 (Room D10) and Thursday at 09:00-10:50 (Room D9)
Tutorials: Friday at 09:00-10:50 (Room D8)
Course Instructor
Aykut Erdem
Email: | aykut@cs.hacettepe.edu.tr |
Web: | http://web.cs.hacettepe.edu.tr/~aykut |
Office Hour: | By appointment (Send email) |
Teaching Assistants
Aysun Kocak
Email: | aysunkocak@cs.hacettepe.edu.tr |
Office Hour: |
Isil Karabey
Email: | isilkarabey@cs.hacettepe.edu.tr |
Office Hour: |
Communication
The course webpage will be updated regularly throughout the semester with lecture notes, programming and reading assignments and important deadlines. All other course related communications will be carried out through Piazza. Please enroll it by following the link https://piazza.com/hacettepe.edu.tr/spring2016/bbm406.
Pre-requisites
Basic probability, linear algebra and calculus. Good programming skills.
Course Requirements and Grading
Grading for BBM 406 will be based on
- a course project (done in pairs) (25%),
- a midterm exam (30%),
- a final exam (40%), and
- class participation (5%).
- a set of quizzes (20%), and
- three problem sets (done individually) which involve both theoretical and programming exercises (20%+30%+30%).
Policies
All work on assignments must be done individually unless stated otherwise. You are encouraged to discuss with your classmates about the given assignments, but these discussions should be carried out in an abstract way. That is, discussions related to a particular solution to a specific problem (either in actual code or in the pseudocode) will not be tolerated.
In short, turning in someone else’s work, in whole or in part, as your own will be considered as a violation of academic integrity. Please note that the former condition also holds for the material found on the web as everything on the web has been written by someone else.
Reference Books
A Course in Machine Learning, Hal Daumé III (online version available)
Artificial Intelligence: A Modern Approach (3rd Edition), Stuart Russell and Peter Norvig. Prentice Hall, 2009
Bayesian Reasoning and Machine Learning, David Barber, Cambridge University Press, 2012 (online version available)
Introduction to Machine Learning (3rd Edition), Ethem Alpaydin, MIT Press, 2014
Machine Learning: A Probabilistic Perspective, Kevin Murphy, MIT Press, 2012
Pattern Recognition and Machine Learning, Christopher Bishop, Springer, 2006
Schedule
Date | Topic | Notes |
---|---|---|
Feb 9 | Course outline and logistics, An overview of Machine Learning [slides] | Reading: The Discipline of Machine Learning, Tom Mitchell Video: The Master Algorithm, Pedro Domingos |
Feb 11 | Machine Learning by Examples, Nearest Neighbor [slides] | Reading: Barber 1,14.1-14.2 Demo: k-Nearest Neighbors Tutorial: Python/numpy |
Feb 16 | Kernel Regression, Distance Functions, Curse of Dimensionality [slides] | Reading: Bishop 1.4, 2.5 |
Feb 18 | Linear Regression, Generalization, Model Complexity, Regularization [slides] | Assg1 out Reading: Bishop 1.1, 3.1, Stanford CS229 note Demo: Linear regression Tutorial: kNN and Linear Regression |
Feb 23 | Regularization (cont'd), Machine Learning Methodology [slides] | Reading: P. Domingos, A few useful things to know about machine learning |
Feb 25 | Learning Theory, Basic Probability Review [slides] | Reading: Daume III 10.1-10.6, Barber 1.1-1.4, CIS 520 note E. Simoncelli, A Geometric Review of Linear Algebra Video: Probability Primer Tutorial: Linear Algebra |
Mar 1 | Basic Probability Review (cont'd.) Statistical Estimation: MLE [slides] | Reading: Murphy 2.1-2.3.2 Video: Daphne Koller, Probabilistic Graphical Models, MLE Lecture |
Mar 3 | Statistical Estimation: MLE (cont'd.), MAP, Naïve Bayes Classifier [slides] | Assg1 due, Assg2 out Reading: Daume III 7, Naïve Bayes, Tom M. Mitchell Optional Reading: Learning to Decode Cognitive States from Brain Images, Tom M. Mitchell et al. Demo: Bayes Theorem Video: Daphne Koller, Probabilistic Graphical Models, MAP Lecture |
Mar 8 | Logistic Regression, Discriminant vs. Generative Classification [slides] | Reading: Barber 17.4, Bishop 4.1.1-4.1.2, 4.5 Optional Reading: On Discriminative vs. Generative classifiers: A comparison of logistic regression and naive Bayes, Andrew Y. Ng, Michael I. Jordan |
Mar 10 | Linear Discriminant Functions, Perceptron [slides] | Course project proposal due Reading: Bishop 4.1.1-4.1.2, Daume III 3 Optional Reading: On Discriminative vs. Generative classifiers: A comparison of logistic regression and naive Bayes, Andrew Y. Ng, Michael I. Jordan Tutorial: Logistic Regression, Linear Discriminant Functions, Perceptron |
Mar 15 | Multi-layer Perceptron [slides] | Reading: Bishop Ch. 5.1 Video: Neural Networks, Andrew Ng Demo: A Neural Network Playground |
Mar 17 | Training Neural Networks: Computational Graph, Back-propagation [slides] | Assg2 due Reading: CS 231 Backpropagation notes Tutorial: Neural Networks |
Mar 22 | No class | |
Mar 24 | Midterm exam | Assg3 out [CIFAR10 data] |
Mar 29 | Introduction to Deep Learning [slides] | Reading: Deep Learning, Yann LeCun, Yoshio Bengio, Geoffrey Hinton |
Mar 31 | Deep Convolutional Networks [slides] | Reading: Conv Nets: A Modular Perspective, Understanding Convolutions, Christopher Olah |
Apr 5 | Support Vector Machines (SVMs) [slides] | Assg3 due Reading: Alpaydin 13.1-13.2 |
Apr 7 | Soft margin SVM [slides] | Reading: Alpaydin 13.3 Demo: Andrea Vedaldi's SVM MATLAB demo |
Apr 12 | Multi-class SVM [slides] | Demo: Multi-class SVM demo |
Apr 14 | Kernels, Kernel Trick, Support Vector Regression [slides] | Reading: Alpaydin 13.5-13.7, CS229 Notes 3.7 Tutorial: Support Vector Machines |
Apr 21 | Decision Tree Learning [slides] | Reading: Mitchell 3, Bishop 14.4 Demo: A Visual Introduction to Machine Learning Tutorial: Decision Trees |
Apr 26 | Ensemble Methods: Bagging, Random Forests [slides] | Reading: Bishop 14.1-14.2, Understanding the Bias-Variance Tradeoff, Scott Fortmann-Roe, Random Forests, Leo Breiman and Adele Cutler Optional Reading: Real-Time Human Pose Recognition in Parts from Single Depth Images, Jamie Shotton et al. Demo: Bootstrapping |
Apr 28 | Ensemble Methods: Boosting [slides] | Reading: Bishop 14.3, Boosting, Robert Schapire Optional Reading: Rapid Object Detection using a Boosted Cascade of Simple Features, Paul Viola and Michael Jones Video: A Boosting Tutorial, Robert Schapire |
May 3 | Clustering: K-Means [slides] | Reading: Bishop 9.1 Demo: Visualizing K-Means equilibria |
May 5 | Clustering: Spectral Clustering, Agglomerative Clustering [slides] | Cluster Analysis: Basic Concepts and Algorithms, Pang-Ning Tan, Michael Steinbach and Vipin Kumar |
May 10 | Principle Component Analysis (PCA) [slides] | Reading: Barber 15.1-15.3, 15.7, Stanford CS229 note Video: PCA, Andrew Ng Demo: Principal Component Analysis Explained Visually |
May 12 | PCA Applications, Autoencoders, Kernel PCA |
Course Project
The students taking the course are required to complete a research oriented project. The students can work individually or in pairs to apply their newly acquired skills towards solving a problem of their own choosing.
For a detailed description of the course project and the related schedule, see this page. In preparing your progress and final project reports, you should use the provided LaTeX template and submit them electronically in PDF format.
Assignments
- Assignment 1 (Due: March 4, 2016 (23:59:59))
- Assignment 2 (Due: March 20, 2016 (23:59:59))
- Assignment 3 (Due: April 7, 2016 (23:59:59))
Resources
Related Conferences
- Advances in Neural Information Processing Systems (NIPS)
- International Conference on Machine Learning (ICML)
- The Conference on Uncertainty in Artificial Intelligence (UAI)
- International Conference on Artificial Intelligence and Statistics (AISTATS)
- IEEE International Conference on Data Mining (ICDM)
Related Journals
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Journal of Machine Learning Research
- Data Mining and Knowledge Discovery
- IEEE Transactions on Neural Networks
Python Resources
- Python/numpy Tutorial.
- scikit-learn: Machine learning in Python
Linear Algebra
- A Geometric Review of Linear Algebra, by Eero Simoncelli
- An Introduction to Linear Algebra in Parallel Distributed Processing, by M.I. Jordan
Resources for scientific writing and talks
- Notes on writing, by Fredo Durand
- How to write a great research paper, by Simon Peyton Jones (video)
- Small Guide To Giving Presentations, by Markus Püschel
- Giving an effective presentation: Using Powerpoint and structuring a scientific talk, by Susan McConnell (video)
- Writing papers and giving talks, by Bill Freeman (notes)