CMP712 - Machine Learning

Spring 2025

Instructor: Assoc. Prof. Dr. Hacer Yalım Keleş

Email: hacerkeles@cs.hacettepe.edu.tr

Lectures: Fridays, 09:40-12:30

Room: D5

Attendance: Compulsory

Communication: Course Piazza Page 

Prerequisites:

• Programming (Python)
• Basic Probability
• Linear Algebra

Reference Textbooks: 

• Pattern Recognition and Machine Learning, Christopher M. Bishop, Springer (2006) (PRML: available online)
• Pattern Classification, R. O. Duda, P. E. Hart, D. G. Stork, (Second Edn.)
• Deep Learning, I. Goodfellow, Y. Bengio, A. Courville, MIT Press 2016 (available online)

Supplementary Textbooks:

• Bayesian Reasoning and Machine Learning, David Barber. (BRML: available online)
• The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Trevor Hastie, Robert Tibshirani, and Jerome Friedman, Springer, 2008. (ESL: available online)
• An Introduction to Statistical Learning with Applications in R, G. James, D. Witten, T. Hastie, and R. Tibshirani, Springer 2021. (ISL: available online)

Aim:

This graduate level lecture provides the theoretical foundations of the various machine learning models and algorithms together with the related literature. In this context, we will study the basics of statistical machine learning, including Bayesian Decision Theory, Maximum Likelihood Estimation, dimension reduction, basic supervised and unsupervised learning algorithms and optimization algorithms; with discussions on capacity, overfitting, underfitting problems and bias-variance issues. Following the basics, we will also study modern practices of deep networks, including deep feedforward networks and convolutional neural networks. 

Content Summary:

• Supervised Learning: regression, linear models of classification, perceptrons, support vector machines (SVMs), LDA, logistic regression, decision trees (bagging, boosting), neural networks, convolutional neural networks, recurrent neural networks, boosting, nearest neighbor search,
• Unsupervised Learning: density estimation (Parzen windows, kNN), dimensionality reduction (PCA), Clustering, Autoencoders

Tentative Schedule

Assessment:

Research Project: 

• Select a machine learning problem. Possible application areas: computer vision, speech recognition, NLP etc. 
• Find a reference paper from the recent literature (papers published since 2022).  There are some constraints in the paper selection: 
• The selected model must be a deep learning based model
• The selected paper must be cited by at least 5 papers.
• Everyone needs to pick a different paper. 
• Read and understand your reference paper and the related literature. 
• Implement the selected approach (maybe only a part of it that is related to your project).
• During the Finals:
• Prepare a term paper (the format of the paper will be provided later) 
• Submission: Term paper, short video presentation (details will be provided later), and source codes ( .ipynb file of your Jupyter notebook). 

Logistics:

All communication and announcements will be posted in the Piazza page. It is mandatory that you enrolled the piazza using this link.