Course Information

About

This advanced undergraduate course is about the fundamentals of computational photography, an emerging new research area which brings together the advancements in computer graphics, computer vision and image processing to overcome the limitations of conventional photography. The course is structured around basic topics such as cameras and image formation, high dynamic range imaging, edge-aware filtering, gradient-domain processing, deconvolution, blending and compositing, visual quality assessment, deep image enhancement, neural rendering.

The main goal of this course is to introduce students a number of different computational techniques to capture, manipulate and enrich visual media. The students are expected to develop a foundational understanding and knowledge of concepts that underly computational photography. The students will also be expected to gain hand-on experience via a set of programming assignments supplied in the complementary BBM 446 Computational Photography Practicum.

The course is taught by Erkut Erdem, and the teaching assistants are Ali Baran Tasdemir and Ahmet Alkilinc.

                 

Time and Location

Lectures: Mondays at 09:40-12:30 (Seminar Hall)
Practicum: Mondays at 15:40-17:30 (D8)

Reference Books

Policies: All work on assignments must be done individually unless stated otherwise. You are encouraged to discuss with your classmates about the given assignments, but these discussions should be carried out in an abstract way. That is, discussions related to a particular solution to a specific problem (either in actual code or in the pseudocode) will not be tolerated.

In short, turning in someone else’s work, in whole or in part, as your own will be considered as a violation of academic integrity. Please note that the former condition also holds for the material found on the web as everything on the web has been written by someone else.

Communication

The course webpage will be updated regularly throughout the semester with lecture notes, presentations, assignments and important deadlines. All other course related communications will be carried out through ed. Please enroll it by following the link https://edstem.org/eu/join/Up3fMG.

Pre-requisites

Good math (calculus, linear algebra, statistics) and programming skills. An introductory course in image processing (BBM413/AIN430), and/or computer vision (BBM416/AIN431) and/or machine learning (BBM406/AIN311) is highly recommended.

Course Requirements and Grading

Grading for BBM444/AIN434 will be based on

Grading for BBM446 will be based on

Schedule

Date Topic Notes
Feb 19 Introduction, Digital photography [slides] Brian Hayes, Computational Photography, American Scientist 96, 94-99, 2008
Feb 26 Image formation [slides] Szeliski, Chapter 2
Forsyth and Ponce, Chapter 1.1
Antonio Torralba and William T. Freeman, Accidental pinhole and pinspeck cameras, CVPR 2012
Mar 4 Noise and Color [slides] Assg1 out
Szeliski, Chapter 2.3, 3.1.2, 10.1
Forsyth and Ponce, Chapter 3
Michael S. Brown, Understanding the In-Camera Rendering Pipeline and the role of AI/Deep Learning, ICCV 2023 Tutorial
S.J. Gortler, Chapter 19 (Color) of Foundations of 3D Computer Graphics, MIT Press, 2012
Mar 11 Exposure and high-dynamic-range imaging [slides] Szeliski, Chapter 10.1, 10.2
Paul E. Debevec and Jitendra Malik, Recovering High Dynamic Range Radiance Maps from Photographs, SIGGRAPH 1997
Tomoo Mitsunaga and Shree K. Nayar, Radiometric Self Calibration, CVPR 1999
Erik Reinhard et al., Photographic tone reproduction for digital images, CVPR 1999
Mar 18 Edge-aware filtering [slides] Assg1 due, Assg2 out
Szeliski, Chapter 3.2, 3.3
Carlo Tomasi and Roberto Manduchi, Bilateral Filtering for Gray and Color Images, ICCV 1998
Sylvain Paris et al., A Gentle Introduction to the Bilateral Filter and Its Applications, SIGGRAPH 2008 class
Georg Petschnigg et al., Digital photography with flash and no-flash image pairs, SIGGRAPH 2004
Kaiming He et al., Guided image filtering, ECCV 2010
Antoni Buades et al., A non-local algorithm for image denoising, CVPR 2005
Levent Karacan et al., Structure Preserving Image Smoothing via Region Covariances, SIGGRAPH Asia 2013
Qi Zhang et al., Rolling Guidance Filter, ECCV 2014
Mar 25 Gradient-domain image processing [slides] Course project proposal due
Szeliski, Chapter 3.1.3, 3.5.5, 10.4.3
Pérez et al., Poisson Image Editing, SIGGRAPH 2003
Agrawal et al., Removing Photography Artifacts Using Gradient Projection and Flash-Exposure Sampling, SIGGRAPH 2005
Agrawal and Raskar, Gradient Domain Manipulation Techniques in Vision and Graphics, ICCV 2007 tutorial
Bhat et al., GradientShop: A Gradient-Domain Optimization Framework for Image and Video Filtering, ACM Trans. Graphics, 2010
Tumblin et al., Why I want a gradient camera?, CVPR 2005
Hua et al., A Survey on Gradient-Domain Rendering, Eurographics 2019 State of the Art Reports (STAR)
Gallego et al., Event-based Vision: A Survey, IEEE TPAMI, 2020
Apr 1 Focal stacks and lightfields [slides] Assg2 due
Szeliski, Chapter 12.1.3, 14.3
Suwajanakorn et al., Depth from Focus with Your Mobile Phone, CVPR 2015
Levoy and Hanrahan, Light Field Rendering, SIGGRAPH 1996
Gortler et al., The Lumigraph, SIGGRAPH 1996
Ng et al., Light field photography with a hand-held plenoptic camera, Stanford TR 2005-02
Apr 8 Feedback session on projects Assg 3 out
Apr 15 Midterm Exam
Apr 22 Deconvolution, Coded photography [slides] Assg3 due
Szeliski, Chapter 3.4.3, 3.4.4, 10.1.4, 10.3
R. Fergus et al., Removing camera shake from a single image, SIGGRAPH 2006
Cho and Lee, Fast motion deblurring, SIGGRAPH Asia 2009
Levin et al., Understanding and evaluating blind deconvolution algorithms,CVPR 2009
Levin et al., Image and depth from a conventional camera with a coded aperture, SIGGRAPH 2007
Levin et al., Motion-Invariant Photography, SIGGRAPH 2008
Nagahara et al., Flexible depth of field photography, ECCV 2008
Apr 29 Convolutional Neural Networks [slides] Project progress reports due
Assg4 out
Szeliski, Chapter 5.1,5.3,5.4
May 6 Deep Generative Models and their applications [slides] Szeliski, Chapter 5.5.4
Goodfellow, Generative Adversarial Networks, NeurIPS 2016 Tutorial
May 13 Visual quality assessment [slides] Assg4 due
Ma and Fanf, Image Quality Assessment in the Modern Age, ACM MM 2021 tutorial
Sheikh et al., A statistical evaluation of recent full reference image quality assessment algorithms, IEEE TIP 2006
Ding et al., Comparison of full-reference image quality assessment models for optimization of image processing systemsIJCV 2021
Wang and Bovik, Reduced-and no-reference image quality assessment, IEEE SPM 2011
Talebi and Milanfar, NIMA: Neural Image Assessment, IEEE TIP 2018
May 20 Project presentations, Course wrap-up Final project reports due

Resources

Related Conferences

  • IEEE International Conference on Computer Vision (ICCV)
  • European Conference on Computer Vision (ECCV)
  • IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • SIGGRAPH
  • SIGGRAPH Asia
  • IEEE International Conference on Computational Photography (ICCP)
  • Advances in Neural Information Processing Systems (NeurIPS)
  • International Conference on Learning Representations (ICLR)

Reference Journals

  • ACM Transactions on Graphics (ACM TOG)
  • IEEE Transactions on Image Processing (IEEE TIP)
  • IEEE Transactions on Multimedia (IEEE TMM)
  • IEEE Transactions on Pattern Analysis and Machine Intelligence (IEEE TPAMI)
  • International Journal of Computer Vision (IJCV)
  • Computer Vision and Image Understanding (CVIU)
  • Image and Vision Computing (IMAVIS)

Python Resources

Linear Algebra

Resources for scientific writing and talks