BBM 406: Fundamentals of Machine Learning

Fall 2016

Mark I Perceptron
NVIDIA's self-driving car in action, which is trained using a convolutional neural network*.

Course Information

Course Description

This is a undergraduate-level introductory course in machine learning (ML) which will give a broad overview of many concepts and algorithms in ML, ranging from supervised learning methods such as support vector machines and decision trees, to unsupervised learning (clustering and factor analysis). The goal is to provide students with a deep understanding of the subject matter and skills to apply these concepts to real world problems.

Time and Location

Lectures: Mondays at 09:00-10:50 (Room D8) and Thursday at 11:00-11:50 (Room D8)

Tutorials: Wednesdays at 15:00-16:50 (Room D4)

Course Instructor

Aykut Erdem's avatar

Aykut Erdem

Email: aykut@cs.hacettepe.edu.tr
Web: http://web.cs.hacettepe.edu.tr/~aykut
Office Hour:Wednesday 3-4pm

Teaching Assistants

Aysun Kocak's avatar

Aysun Kocak

Email: aysunkocak@cs.hacettepe.edu.tr

Office Hour: Mondays 1:30-3pm


Burcak Asal's avatar

Burcak Asal

Email: basal@cs.hacettepe.edu.tr

Office Hour: Thursdays 1:30-3pm

Communication

The course webpage will be updated regularly throughout the semester with lecture notes, programming and reading assignments and important deadlines. All other course related communications will be carried out through Piazza. Please enroll it by following the link https://piazza.com/hacettepe.edu.tr/fall2016/bbm406.

Pre-requisites

Basic probability, linear algebra and calculus. Good programming skills.

Course Requirements and Grading

Grading for BBM 406 will be based on

  • a course project (done in pairs) (25%),
  • a midterm exam (30%),
  • a final exam (40%), and
  • class participation (5%).
In BBM 409, the grading will be based on
  • a set of quizzes (20%), and
  • three problem sets (done individually) which involve both theoretical and programming exercises (20%+30%+30%).

Policies

All work on assignments must be done individually unless stated otherwise. You are encouraged to discuss with your classmates about the given assignments, but these discussions should be carried out in an abstract way. That is, discussions related to a particular solution to a specific problem (either in actual code or in the pseudocode) will not be tolerated.

In short, turning in someone else’s work, in whole or in part, as your own will be considered as a violation of academic integrity. Please note that the former condition also holds for the material found on the web as everything on the web has been written by someone else.

back to top

Reference Books

  • A Course in Machine Learning, Hal Daumé III (online version available)

  • Artificial Intelligence: A Modern Approach (3rd Edition), Stuart Russell and Peter Norvig. Prentice Hall, 2009

  • Bayesian Reasoning and Machine Learning, David Barber, Cambridge University Press, 2012 (online version available)

  • Introduction to Machine Learning (3rd Edition), Ethem Alpaydin, MIT Press, 2014

  • Machine Learning: A Probabilistic Perspective, Kevin Murphy, MIT Press, 2012

  • Pattern Recognition and Machine Learning, Christopher Bishop, Springer, 2006

back to top

Schedule

Date Topic Notes
Oct 3 Course outline and logistics, An overview of Machine Learning [slides] Reading: The Discipline of Machine Learning, Tom Mitchell
Video 1: The Master Algorithm, Pedro Domingos
Video 2: The Thinking Machine (1961)
Oct 6 Nearest Neighbor [slides] Reading: Barber 1,14.1-14.2
Demo: k-Nearest Neighbors
Tutorial: Python/numpy
Oct 10 Kernel Regression, Distance Functions, Curse of Dimensionality, Introduction to Linear Regression [slides] Reading: Bishop 1.4, 2.5
Oct 13 Linear Regression, Generalization, Model Complexity, Regularization [slides] Assg1 out
Reading: Bishop 1.1, 3.1, Stanford CS229 note
Demo: Linear regression
Tutorial: kNN and Linear Regression
Oct 17 Model Complexity, Regularization (cont'd), Machine Learning Methodology [slides] Reading: P. Domingos, A few useful things to know about machine learning
Oct 20 Learning Theory, Basic Probability Review [slides] Reading: Daume III 10.1-10.6, Barber 1.1-1.4, CIS 520 note
E. Simoncelli, A Geometric Review of Linear Algebra
Video: Probability Primer
Tutorial: Linear Regression, Cross-Validation
Demo: Conditional probability Explained Visually
Oct 24 Basic Probability Review (cont'd.), Statistical Estimation: MLE and MAP [slides] Reading: Murphy 2.1-2.3.2
Video: Daphne Koller, Probabilistic Graphical Models, MLE Lecture, MAP Lecture
Oct 27 Naïve Bayes Classifier [slides] Assg1 due, Assg2 out
Reading: Daume III 7, Naïve Bayes, Tom M. Mitchell
Optional Reading: Learning to Decode Cognitive States from Brain Images, Tom M. Mitchell et al.
Demo: Bayes Theorem
Oct 31 Logistic Regression, Discriminant vs. Generative Classification, Linear Discriminant Functions [slides] Reading: Barber 17.4, Bishop 4.1.1-4.1.2, 4.5
Optional Reading: On Discriminative vs. Generative classifiers: A comparison of logistic regression and naive Bayes, Andrew Y. Ng, Michael I. Jordan
Nov 3 Perceptron [slides] Reading: Daume III 3
Tutorial: Naive Bayes
Nov 7 Multi-layer Perceptron [slides] Course project proposal due
Video: Neural Networks, Andrew Ng
Demo: A Neural Network Playground
Nov 10 Training Neural Networks: Computational Graph, Back-propagation [slides] Assg2 due
Reading: CS 231 Backpropagation notes
Tutorial: Logistic Regression, Linear Discriminant Functions, Perceptron
Nov 14 Midterm exam
Nov 17 Project discussion Assg3 out
Nov 21 Introduction to Deep Learning, Deep Convolutional Neural Networks [slides] Reading: Deep Learning, Yann LeCun, Yoshio Bengio, Geoffrey Hinton, Conv Nets: A Modular Perspective, Understanding Convolutions, Christopher Olah
Nov 24 Deep Convolutional Networks (cont'd.) [slides]
Nov 28 Support Vector Machines (SVMs) [slides] Assg3 due
Reading: Alpaydin 13.1-13.2
Video: Patrick Winston, Support Vector Machines
Dec 1 Soft margin SVM [slides] Reading: Alpaydin 13.3
Demo: Andrea Vedaldi's SVM MATLAB demo
Dec 5 Multi-class SVM, Kernels [slides] Reading: Alpaydin 13.5-13.7, 13.9, M.A. Hearst, Support Vector Machines, CS229 Notes 3.7
Demo: Multi-class SVM demo
Dec 8 Support Vector Regression [slides] Tutorial: Support Vector Machines
Dec 12 Decision Tree Learning [slides] Project progress reports due
Reading: Mitchell 3, Bishop 14.4
Demo: A Visual Introduction to Machine Learning
Tutorial: Decision Trees
Dec 15 Ensemble Methods: Bagging, Random Forests [slides] Reading: Bishop 14.1-14.2, Understanding the Bias-Variance Tradeoff, Scott Fortmann-Roe, Random Forests, Leo Breiman and Adele Cutler
Optional Reading: Real-Time Human Pose Recognition in Parts from Single Depth Images, Jamie Shotton et al.
Demo: Bootstrapping
Dec 19 Ensemble Methods: Boosting [slides] Reading: Bishop 14.3, Boosting, Robert Schapire
Optional Reading: Rapid Object Detection using a Boosted Cascade of Simple Features, Paul Viola and Michael Jones
Video: A Boosting Tutorial, Robert Schapire
Dec 22 Clustering: K-Means [slides] Reading: Bishop 9.1
Demo: Visualizing K-Means equilibria
Dec 26 Clustering: Spectral Clustering, Agglomerative Clustering [slides] Cluster Analysis: Basic Concepts and Algorithms, Pang-Ning Tan, Michael Steinbach and Vipin Kumar
Dec 29 Principle Component Analysis (PCA) [slides] Reading: Barber 15.1-15.3, 15.7, Stanford CS229 note
Video: PCA, Andrew Ng
Demo: Principal Component Analysis Explained Visually
Jan 2 PCA Applications, Autoencoders, Kernel PCA [slides]
Jan 5 Project presentations

back to top

Class Project

The students taking BBM 406 are required to complete a research oriented project, and this semester the theme is Machine Learning and Food. Please see the project page for the details.

back to top

Assignments

back to top

Resources

Related Conferences

  • Advances in Neural Information Processing Systems (NIPS)
  • International Conference on Machine Learning (ICML)
  • The Conference on Uncertainty in Artificial Intelligence (UAI)
  • International Conference on Artificial Intelligence and Statistics (AISTATS)
  • IEEE International Conference on Data Mining (ICDM)

Related Journals

  • IEEE Transactions on Pattern Analysis and Machine Intelligence
  • Journal of Machine Learning Research
  • Data Mining and Knowledge Discovery
  • IEEE Transactions on Neural Networks

Python Resources

Linear Algebra

Resources for scientific writing and talks

back to top


© 2016 Hacettepe University