BIL 721 - Computational Photography (Spring 2015)

Lectures: Tue 13:00-15:45_D5

Philipp Otto Runge's Sphere (Die Farbenkugel) (1810) Animated movie still from The Royal Tenenbaums (2001), by If We Don't, Remember Me.

Instructor: Aykut Erdem

aykut-at-cs-hacettepe.edu.tr
111
+90 312 297 7500, 146

Course Description

This course introduces graduate students to a broad selection of topics in computational photography, an emerging new field at the intersection of computer graphics, computer vision and photography. The main focus of the course will be on software-based solutions, and to that end, we will discuss ways of using computational techniques to overcome the limitations of conventional photography and to enhance how we capture, manipulate and interact with images.

Prerequisites

Good programming skills (especially Matlab) and basic knowledge of linear algebra and calculus. Previous knowledge of computer graphics, computer vision or image processing will be helpful but not absolutely required.

Schedule (Tentative)

Week Date Topic Notes
1 Feb 17 Introduction to the course [slides] [4-per-page] Readings:
  • Brian Hayes, Computational Photography, American Scientist 96, 94-99, 2008
  • 100 Ideas That Changed Photography (Brain Pickings)
2 Feb 24 Cameras and sensors [slides] [4-per-page] PA1 is out
Readings:
  • A. Levin, R. Fergus, F. Durand, and B. Freeman, Image and Depth from a Conventional Camera with a Coded Aperture, SIGGRAPH 2007
  • How Your Camera Works (by Daniel Eggert)
  • How Digital Cameras Work (HowStuffWorks)
Demo: Photography applets (by Marc Levoy, Andrew Adams, Katie Dektar, and Nora Willett)
3 Mar 3 Image processing basics [slides] [4-per-page] Paper selections due
Readings:
  • A. Oliva, A. Torralba, and P.G. Schyns, Hybrid Images, ACM SIGGRAPH 2006
4 Mar 10 Texture synthesis, video textures [slides] [4-per-page] PA1 and project proposals due
Readings:
  • A. Schödl, R. Szeliski, D. Salesin, and I. Essa, Video Textures, SIGGRAPH 2000
  • A.A. Efros and W.T. Freeman, Image quilting for texture synthesis and transfer, SIGGRAPH 2001
  • A. Hertzmann, C.E. Jacobs, N. Oliver, B. Curless, and D.H. Salesin, Image Analogies, SIGGRAPH 2001
Paper presentation:
  • D. Dai, H. Riemenschneider, and L. Van Gool, The Synthesizability of Texture Examples, CVPR 2014 (Samet Hicsonmez)
    5 Mar 17 Image resizing and blending [slides] [4-per-page] PA2 is out
    Readings:
    • S. Avidan and A. Shamir, Seam Carving for Content-Aware Image Resizing, ACM Trans. Graph., 26(3), 2007.
    • P. Perez, M. Gangnet, and A. Blake, Poisson Image Editing, SIGGRAPH 2003.
    Paper presentation:
    • S.-M. Hu, F.-L. Zhang, M. Wang, R.R. Martin, and J. Wang, PatchNet: A Patch-based Image Representation for Interactive Library-driven Image Editing, ACM SIGGRAPH Asia 2013 (Orkun Ozturk)
    6 Mar 24 Stitching and matting [slides] [4-per-page] Readings:
    • Y. Li, J. Sun, C.-K. Tang, and H.-Y. Shum, Lazy Snapping, SIGGRAPH 2004
    • A. Agarwala, M. Dontcheva, M. Agrawala, S. Drucker, A. Colburn, B. Curless, D. Salesin, M. Cohen, Interactive Digital Photomontage, SIGGRAPH 2004.
    • A. Levin, D. Lischinski, and Y. Weiss, A Closed Form Solution to Natural Image Matting, CVPR 2006.
    • Y.-Y. Chuang, B. Curless, D. Salesin, R. Szeliski, A Bayesian Approach to Digital Matting, CVPR 2001.
    Paper presentation:
    • M.W. Tao, M.K. Johnson, and S. Paris, Error-tolerant Image Compositing, ECCV 2010 (Levent Karacan)
    7 Mar 31 Warping and morphing [slides] [4-per-page] PA2 due
    Readings:
    • T. Beier and S. Neely, Feature-Based Image Metamorphosis, SIGGRAPH 1992.
    • S.M. Seitz and C.R. Dyer, View Morphing, SIGGRAPH 2006.
    Paper presentation:
    • S. Darabi, E. Shechtman, C. Barnes, D.B. Goldman, and P. Sen, Image Melding: Combining Inconsistent Images using Patch-based Synthesis, ACM SIGGRAPH 2012 (Buket Ogul)
    8 Apr 7 Panoramas and mosaics [slides] [4-per-page] PA3 is out [starter code]
    Readings:
    • M. Brown and D.G. Lowe, Recognising Panoramas, ICCV 2003.
    • M. Brown, R. Szeliski, and S. Winder, Multi-Image Matching using Multi-Scale Oriented Patches, CVPR 2005.
    Paper presentation:
    • J. Kopf, M.F. Cohen, and R. Szeliski, First-person hyperlapse videos, ACM SIGGRAPH 2014 (Ugur Eray Tahta)
    9 Apr 14 HDR and tone mapping [slides] [4-per-page] Readings:
    • P.E. Debevec and J. Malik, Recovering High Dynamic Range Radiance Maps from Photographs, SIGGRAPH 1997.
    • F. Durand and J. Dorsey, Fast Bilateral Filtering for the Display of High-Dynamic-Range Images, SIGGRAPH 2002.
    • G.J. Brown, High dynamic range digital photography, Royal Photographic Society Journal, pp. 428-431, 2006.
    Paper presentation:
    • J. Hu, O. Gallo, K. Pulli, X. Sun, HDR Deghosting: How to Deal with Saturation?, CVPR 2013 (Nihat Tekeli)
    10 Apr 21 Faces [slides] [4-per-page] PA3 due
    Readings:
    • D.A. Rowland and D.I. Perrett, Manipulating Facial Appearance through Shape and Color, IEEE CG&A 1995.
    • L. Sirovich and M. Kirby, Low-dimensional procedure for the characterization of human faces, Journal of the Optical Society of America A 4(3):519–524, 1987
    Paper presentation:
    • I. Kemelmacher-Shlizerman, S. Suwajanakorn, and S.M. Seitz, Illumination-Aware Age Progression, CVPR 2014 (Mehran Sarandi Far)
    11 Apr 28 Better photos [slides] [4-per-page] Project progress reports due
    Readings:
    • Y. Ke, X. Tang, and F. Jing, The Design of High-Level Features for Photo Quality Assessment, CVPR 2006.
    • D. Joshi, R. Datta, E. Fedorovskaya, Q.-T. Luong, J.Z. Wang, J. Li, and J. Luo, Aesthetics and Emotions in Images: A Computational Perspective, IEEE Signal Processing Magazine, 28(5):94-115, 2011.
    • S. Bae, S. Paris, and F. Durand, Two-­scale Tone Management for Photographic Look, SIGGRAPH 2006.
    Paper presentation:
    • L. Kratz and K. Nishino, Factorizing Scene Albedo and Depth from a Single Foggy Image, ICCV 2009 (Ozgur Altun)
    12 May 5 Big visual data (part 1) [slides] [4-per-page] Readings:
    • A. Torralba, R. Fergus, W.T. Freeman, 80 million tiny images: A large dataset for non-parametric object and scene recognition, IEEE Trans. Pattern Anal. Mach. Intell., 30(11):1958-1970, 2008
    • J. Hays and A.A. Efros, Scene Completion Using Millions of Photographs, ACM SIGGRAPH 2007
    Paper presentation:
    • J.-Y. Zhu, Y.J. Lee, and A.A. Efros, AverageExplorer: Interactive Exploration and Alignment of Visual Data Collections, ACM SIGGRAPH 2014 (Firat Ercis)
    13 May 12 Big visual data (part 2) [slides] [4-per-page] Readings:
    • J. Hays and A.A. Efros, IM2GPS: Estimating geographic information from a single image, CVPR 2008A. Shrivastava, T. Malisiewicz, A. Gupta, and A.A. Efros, Data-­driven visual similarity for cross-domain image matching, SIGGRAPH Asia 2011.
    • M. Johnson, G. Brostow, J. Shotton, O. Arandjelovic, V. Kwatra, and R. Cipolla, Semantic Photo Synthesis, Computer Graphics Forum, 25(3):407–413, 2006.
    Paper presentation:
    • P.-Y. Laffont, Z. Ren, X. Tao, C. Qian, and J. Hays, Transient Attributes for High-Level Understanding and Editing of Outdoor Scenes, ACM SIGGRAPH 2014 (Semih Yagcioglu)
    14 May 19 No class (Commemoration of Ataturk, Youth and Sports Day)

    Grading

    • 20% Quizzes
    • 20% Programming Assignments
    • 20% Paper presentations/Class participation
    • 40% Project and final term paper

    Paper presentations and Quizzes

    The students will be required to present at least one research paper either of their choice or from the suggested reading list. These papers should be read by every student as the quizzes about the presented papers will be given on the weeks of the presentations.

    Programming Assignments

    There will be four assignments related to the topics covered in the class. Each assignment will involve implementing an algorithm, carrying out a set of experiments to evaluate it, and writing up a report on the experimental results. All assignments have to be done individually, unless stated otherwise.

    Project

    The students will be required to do a project in computational photography. For a detailed description of the course project, follow this link. In preparing your progress and final project reports, you should use the provided template and submit them electronically in PDF format.

    Reference Books

    • Computer Vision: Algorithms and Applications, Richard Szeliski, Springer, ISBN: 978-1-84882-934-3 (online version available)
    • Photography (11th edition), Barbara London, Jim Stone, and John Upton, Pearson, 2014, ISBN: 978-0-205-93380-8
    • Stanford CS178, Digital Photography, Course notes

    Resources

    • Related Conferences:
      • ACM SIGGRAPH
      • ACM SIGGRAPH Asia
      • Eurographics (EG)
      • ACM Multimedia (MM)
      • IEEE International Conference on Computer Vision (ICCV)
      • European Conference on Computer Vision (ECCV)
      • IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
      • British Machine Vision Conference (BMVC)
      • Asian Conference of Computer Vision (ACCV)
      • Advances in Neural Information Processing Systems (NIPS)
      • IEEE International Conference on Computational Photography (ICCP)
      • IEEE International Conference on Pattern Recognition (ICPR)
    • Related Journals:
      • ACM Transactions on Graphics (TOG)
      • International Journal of Computer Vision
      • IEEE Transactions on Pattern Analysis and Machine Intelligence
      • Computer Vision and Image Understanding
      • Pattern Recognition
      • Journal of Mathematical Imaging and Vision
      • Image and Vision Computing
      • The Visual Computer
    • MATLAB Resources:
      • Introduction to MATLAB, by Danilo Šćepanović
      • MATLAB Tutorial, Stefan Roth
      • MATLAB Primer, MathWorks
      • Code Vectorization Guide, MathWorks
      • Writing Fast MATLAB code, Pascal Getreuer
      • MATLAB array manipulation tips and tricks, Peter J. Acklam
    • Linear Algebra:
      • A Geometric Review of Linear Algebra, Eero Simoncelli
      • An Introduction to Linear Algebra in Parallel Distributed Processing, M.I. Jordan
    • Resources for scientific writing and talks:
      • How to read a paper, S. Keshav
      • Notes on writing, Fredo Durand
      • How to write a great research paper, Simon Peyton Jones (video)
      • Small Guide To Giving Presentations, Markus Püschel
      • Giving an effective presentation: Using Powerpoint and structuring a scientific talk, Susan McConnell (video)
      • How to give a talk (that doesn't put your audience to sleep), Ramesh Raskar
      • Writing papers and giving talks, Bill Freeman (notes)

    Communication:

    The course webpage will be updated regularly throughout the semester with lecture notes, programming and reading assignments and important deadlines. All other course related communications will be carried out through Piazza. Please enroll it by following the link https://piazza.com/hacettepe.edu.tr/spring2014/bil721

    Policies:

    All work on assignments must be done individually unless stated otherwise. You are encouraged to discuss with your classmates about the given assignments, but these discussions should be carried out in an abstract way. That is, discussions related to a particular solution to a specific problem (either in actual code or in the pseudocode) will not be tolerated.

    In short, turning in someone else’s work, in whole or in part, as your own will be considered as a violation of academic integrity. Please note that the former condition also holds for the material found on the web as everything on the web has been written by someone else.

    Acknowledgements:

    The materials used in this class largely rely on lecture notes by other researchers. In particular, the slides are adapted from those of Alexei A. Efros, Steve Seitz, Rick Szeliski, Paul Debevec, Stephen Palmer, Paul Heckbert, David Forsyth, Steve Marschner, Lana Lazebnik, Silvio Savarese, Rob Fergus, Marc Levoy, Peter N. Belhumeur, Derek Hoiem, James Hays, Tamara Berg, Fredo Durand, Bill Freeman (as credited within).


    © 2015 Hacettepe University