BBM 406 - Introduction to Machine Learning (Spring 2015)

Lectures: Tue 10:00-10:50_D8, Thu 09:00-10:50_D10

Frank Rosenblatt's Mark I Perceptron at the Cornell Aeronautical Laboratory, Buffalo, New York, circa 1960

Instructor: Aykut Erdem

aykut-at-cs-hacettepe.edu.tr
111
+90 312 297 7500, 146

Course Description

This is a undergraduate-level introductory course in machine learning (ML) which will give a broad overview of many concepts and algorithms in ML, ranging from classification methods such as support vector machines and decision trees, to unsupervised learning (clustering and factor analysis). The goal is to provide students with a deep understanding of the subject matter and skills to apply these concepts to real world problems.

Prerequisites

Basic probability, linear algebra and calculus. Good programming skills.

Schedule

Week	Date	Topic	Notes
1	Feb 17	Overview of Machine Learning, Your 1st classifier: Nearest Neighbor Classifier [slides] [4-per-page]	Reading: Barber 1,14.1-14.2 Video: What is Machine Learning?, Bernhard Schölkopf Demo: k-Nearest Neighbors
2	Feb 24	Linear Regression, Least Squares [slides] [4-per-page]	PS1 out Reading: Bishop 1.1, 3.1, Stanford CS229 note Demo: Linear regression
3	Mar 3	Machine Learning Methodology, Basic Probability and Linear Algebra Reviews [slides] [4-per-page]	Reading: Murphy 2.1-2.3.2, CIS 520 note, P. Domingos, A few useful things to know about machine learning, E. Simoncelli, A Geometric Review of Linear Algebra Video: Probability Primer
4	Mar 10	Statistical Estimation: MLE, MAP, Naïve Bayes Classifier [slides] [4-per-page]	Reading: Barber 8.6, 8.7, Naïve Bayes, Tom M. Mitchell Optional Reading: Learning to Decode Cognitive States from Brain Images, Tom M. Mitchell et al. Video: Daphne Koller, Probabilistic Graphical Models, MLE Lecture, MAP Lecture
5	Mar 17	Linear Classification Models: Logistic Regression, Linear Discriminant Functions, Perceptron [slides] [4-per-page]	PS1 due, PS2 out Reading: Barber 17.4, Bishop 4.1.1-4.1.2, 4.5 Optional Reading: On Discriminative vs. Generative classifiers: A comparison of logistic regression and naive Bayes, Andrew Y. Ng, Michael I. Jordan
6	Mar 24	Support Vector Machines (part 1)[slides] [4-per-page]	Reading: M.A. Hearst, Support Vector Machines, C.J.C. Burges, A Tutorial on Support Vector Machines for Pattern Recognition
7	Mar 31	Support Vector Machines (part 2)
8	Apr 7	Midterm exam
9	Apr 14	Decision Tree Learning [slides] [4-per-page]	PS2 due Reading: Mitchell 3, Bishop 14.4
10	Apr 21	Ensemble Methods: Bagging, Random Forests, Boosting [slides] [4-per-page]	PS3 out (beach and grassland images) Reading: Murphy 16.4, Boosting, Robert Schapire Optional Reading: Real-Time Human Pose Recognition in Parts from Single Depth Images, Jamie Shotton et al. Video: A Boosting Tutorial, Robert Schapire
11	Apr 28	Clustering [slides] [4-per-page]	Reading: Bishop 9.1, A Tutorial on Spectral Clustering, U. von Luxburg Video: Clustering, GMM, Andrew Ng
12	May 5	Deep Learning [slides] [4-per-page]	Reading: Mitchell 4, Bishop Ch. 5, A. Karpathy, Hacker's guide to Neural Networks Video: Neural Networks, Backpropagation, Andrew Ng> Demo: A toy neural network
13	May 12	Principle Component Analysis	PS3 due
14	May 19	Course wrap-up

Grading

30% Problem Sets
30% Midterm Exam
40% Final Exam

Reference Books

Artificial Intelligence: A Modern Approach (3rd Edition), Stuart Russell and Peter Norvig. Prentice Hall, 2009.
Bayesian Reasoning and Machine Learning, David Barber, Cambridge University Press, 2012. (online version available)
Introduction to Machine Learning (2nd Edition), Ethem Alpaydin, MIT Press , 2010.
Pattern Recognition and Machine Learning, Christopher Bishop, Springer, 2006.
Machine Learning: A Probabilistic Perspective, Kevin Murphy, MIT Press, 2012.

Resources

Related Conferences:

Advances in Neural Information Processing Systems (NIPS)
International Conference on Machine Learning (ICML)
The Conference on Uncertainty in Artificial Intelligence (UAI)
International Conference on Artificial Intelligence and Statistics (AISTATS)
IEEE International Conference on Data Mining (ICDM)

Related Journals:

IEEE Transactions on Pattern Analysis and Machine Intelligence
Journal of Machine Learning Research
Data Mining and Knowledge Discovery
IEEE Transactions on Neural Networks

MATLAB Resources:

Introduction to MATLAB, by Danilo Šćepanović
MATLAB Tutorial, by Stefan Roth
MATLAB Primer, by MathWorks
Code Vectorization Guide, by MathWorks
Writing Fast MATLAB code, by Pascal Getreuer
MATLAB array manipulation tips and tricks, by Peter J. Acklam

Linear Algebra:

A Geometric Review of Linear Algebra, by Eero Simoncelli
An Introduction to Linear Algebra in Parallel Distributed Processing, by M.I. Jordan

Resources for scientific writing and talks:

Notes on writing, by Fredo Durand
How to write a great research paper, by Simon Peyton Jones (video)
Small Guide To Giving Presentations, by Markus Püschel
Giving an effective presentation: Using Powerpoint and structuring a scientific talk, by Susan McConnell (video)
Writing papers and giving talks, by Bill Freeman (notes)

Communication:

The course webpage will be updated regularly throughout the semester with lecture notes, programming and reading assignments and important deadlines. All other course related communications will be carried out through Piazza. Please enroll it by following the link https://piazza.com/hacettepe.edu.tr/spring2014/bbm406

Policies:

All work on assignments must be done individually unless stated otherwise. You are encouraged to discuss with your classmates about the given assignments, but these discussions should be carried out in an abstract way. That is, discussions related to a particular solution to a specific problem (either in actual code or in the pseudocode) will not be tolerated.

In short, turning in someone else’s work, in whole or in part, as your own will be considered as a violation of academic integrity. Please note that the former condition also holds for the material found on the web as everything on the web has been written by someone else.