Computational Photography (CS 445) – Fall 2024

Instructor: Yuxiong Wang

Where: 1404 Siebel Center for Comp Sci

When: Tue/Thu 12:30 PM - 01:45 PM (CT)

Instructor Office Hours: Tue 2:00 PM - 3:00 PM (CT) Siebel 3316

TA: Jipeng Lyu (jipeng2), Xin Xu (xinx8), Qianlan Yang (qianlan2)

Office Hours: (Jipeng) Mon 10:00 AM - 11:00 AM (CT) Siebel 3102

Office Hours: (Jipeng) Tue 10:00 AM - 11:00 AM (CT) Siebel 3102

Office Hours: (Qianlan) Wed 4:00 PM - 5:00 PM (CT) Siebel 3102

Office Hours: (Xin) Thu 11:00 AM - 12:00 PM (CT) Siebel 3102

Office Hours: (Qianlan) Fri 2:00 PM - 3:00 PM (CT) Siebel 3102

Syllabus

Take-Home Questions

Campuswire Discussion Board

Gradescope (project submission and grades)

Tutorial on Linear Algebra

Tutorial on NumPy & Jupyter Notebook

Textbook: Computer Vision: Algorithms and Applications by Rick Szeliski

Projects

Project 1: Hybrid Images	Project 2: Image Quilting	Project 3: Gradient Domain Fusion
Project 4: Image-based Lighting	Project 5: Video Stitching	Final Project

Class Schedule

Week	Date	Topic	Link	Reading/Notes
1	Aug 27	Introduction	ppt ; pdf
		Basics of Working with Images
1	Aug 29	Pixels and image filters	ppt ; pdf	S3.2 (linear filtering) S3.3 (non-linear filtering)
2	Sep 3	Thinking in frequency	ppt ; pdf	S3.4 (fourier transforms) S2.3.3 (compression) FFT Demo (original) ; Python demo by Jae
2	Sep 5	Templates and image pyramids	ppt ; pdf	S3.5.2 (image pyramids) S8.1.1 (pyramid alignment) Other reading: Burt Adelson 1983 – The Laplacian Pyramid
3	Sep 10	Light and color	ppt ; pdf	S2.2 (light), S2.3.2 (color) or Forsyth and Ponce Ch 3
3	Sep 12	Histograms and color adjustment	ppt ; pdf	S3.1 (histograms and color adjustment) Color balance demo
	Sep 18 (Wed)	Project 1 (Hybrid images) due
		The Digital Canvas: Coloring, Blending, Cutting, Synthesizing, and Warping Images
4	Sep 17	Growing: Texture synthesis and hole filling	ppt ; pdf	Texture Synthesis – Efros Leung (1999) Image Quilting – Efros Freeman (2001) Image Analogies – Hertzmann et al. (2001) Patch Match – Barnes et al. (2009)
4	Sep 19	Cutting: Intelligent Scissors and Graph Cuts	ppt ; pdf	Intelligent Scissors – Mortenson Barret (1995) GrabCut – Rother et al. (2004)
5	Sep 24	Pasting: Compositing and blending	ppt ; pdf	Poisson Image Editing – Perez et al. (2003) Burt and Adelson, A multiresolution spline with application to image mosaics, ACM ToG (1983)
5	Sep 26	Image warping (translation, rotation, scale, etc.)	ppt ; pdf	S3.6 (warping) Alignment demo
	Oct 2 (Wed)	Project 2 (Image quilting) due
6	Oct 1	Image morphing	ppt ; pdf
		Modeling the Physical World
6	Oct 3	The Pinhole Camera	ppt ; pdf	S2.1.5 (3D to 2D projection)
7	Oct 8	Single-view Metrology + Camera Model continued	ppt ; pdf	HoiemSavarese-SVG Chapter
7	Oct 10	Single-view 3D Reconstruction	ppt ; pdf	Tour into the picture (Horry et al. 1997)
8	Oct 14 – Oct 20	Midterm Exam (on PrarieLearn)
9	Oct 22	Image-based Lighting: ray tracing, environment maps, light probes	ppt ; pdf	Reading (do read this): Debevec, Image-based Lighting Tutorial (2005)
9	Oct 24	Image-based Lighting cont.: HDR light probes, relighting	ppt ; pdf	Optional Reading: Debevec & Malik, “Recovering High Dynamic Range Radiance Maps from Photographs”, SIGGRAPH 1997 Debevec, Rendering Synthetic Objects in Real Scenes, 1998
	Oct 30 (Wed)	Project 3 (Gradient domain fusion) due
10	Oct 29	The image as a virtual stage	ppt ; pdf	Rendering Synthetic Objects (Karsch et al. 2011)
		Correspondence and Recognition
10	Oct 31	Matching and alignment with interest points	ppt ; pdf	Grauman/Leibe Draft Chapter on Local Features Optional: Lowe - SIFT paper
	Nov 1 (Fri)	Final Project Proposal due
11	Nov 5	Automatic Photo Stitching and RANSAC	ppt ; pdf	Brown Lowe 2007 ; S9 (stitching); slides
11	Nov 7	Object recognition, retrieval, and augmented reality	ppt ; pdf	Grauman/Leibe Draft Visual Vocabularies; S14.3
	Nov 13 (Wed)	Project 4 (Image-based lighting) due
12	Nov 12	Opportunities of scale: texture synthesis, multi-view reconstruction, im2gps, tiny images, etc.	ppt ; pdf	Reading: Hays & Efros, Scene Completion Using Millions of Photographs
		More Topics of Interest
12	Nov 14	Generating and detecting fakes	ppt ; pdf	Pix2pix Cycle GAN Deep fakes in news
13	Nov 19	How the Kinect works	ppt ; pdf
13	Nov 21	Computational approaches to cameras	ppt ; pdf	Adelson & Bergen, The Plenoptic Function and the Elements of Early Vision (1991) Ng et al., Light Field Photography with a Hand-held Plenoptic Camera (2005) Levin et al., Image and Depth with a Coded Aperture (2007)
14	Nov 23 – Dec 1	Fall Break
	Dec 4 (Wed)	Project 5 (Video stitching) due
15	Dec 3	Understanding Faces	ppt ; pdf
15	Dec 5	Video Magnification	ppt ; pdf
16	Dec 10	Last day – cutting edge and wrap up	ppt ; pdf
	Dec 15 (Sun)	Final Project due (cannot be late)
16-17	Dec 13 – Dec 19	Final Exam (on PrairieLearn)

Credits and Course Notes

I am grateful for the many preceding efforts to design excellent courses and course notes for this topic. I am particularly indebted to Derek Hoiem for designing this course and borrowing much of his course and project design, to Rick Szeliski for making his book available, to James Hays for course suggestions and materials, and to Alyosha Efros, Steve Seitz, Lana Lazebnik, Silvio Savarese, among others for their slides (as credited within). Anyone who would like to use my slides is more than welcome; please do credit the original sources where appropriate.