Computational Photography (CS 445) – Fall 2024  

  

Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: <i>The Key to the Fields</i> (Magritte 1936)

  Instructor:  Yuxiong Wang

 

 

 

  Where: 1404 Siebel Center for Comp Sci

  When: Tue/Thu 12:30 PM - 01:45 PM (CT)

 

  Instructor Office Hours: Tue 2:00 PM - 3:00 PM (CT) Siebel 3316

 

  TA: Jipeng Lyu (jipeng2), Xin Xu (xinx8), Qianlan Yang (qianlan2)

  Office Hours: (Jipeng) Mon 10:00 AM - 11:00 AM (CT) Siebel 3102

  Office Hours: (Jipeng) Tue 10:00 AM - 11:00 AM (CT) Siebel 3102

  Office Hours: (Qianlan) Wed 4:00 PM - 5:00 PM (CT) Siebel 3102

  Office Hours: (Xin) Thu 11:00 AM - 12:00 PM (CT) Siebel 3102

  Office Hours: (Qianlan) Fri 2:00 PM - 3:00 PM (CT) Siebel 3102

 

  Syllabus

  Take-Home Questions

  Campuswire Discussion Board

  Gradescope (project submission and grades)

 

  Tutorial on Linear Algebra

  Tutorial on NumPy & Jupyter Notebook

 

 

 

 

  Textbook: Computer Vision: Algorithms and Applications by Rick Szeliski

                                                                                                           

  

   Projects

Project 1: Hybrid Images

Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: leopard elephant

Project 2: Image Quilting

    Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: C:\Users\Hoiem\Documents\Classes\ComputationalPhotography - Fall 2012\projects\quilting\face_toast_cropped.png

   

Project 3: Gradient Domain Fusion

Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: cloning_teaser

 

Project 4: Image-based Lighting

https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQJ6tcY1JugjgyMMpZMj9BgU6OaGyIv8jL4T8J-Qsgnn0UOrHT2tw

 

 

Project 5: Video Stitching

 

 

Final Project

Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: http://t2.gstatic.com/images?q=tbn:ANd9GcSEXEDpOJHSX2MOS0mvolJmNhoGKiQZ6t4ky5dyEio1mph1-L1x

 

 

 

  

  Class Schedule  

Week

Date

Topic

Link

Reading/Notes

1

Aug 27

Introduction

ppt ; pdf

 

 

Basics of Working with Images

 

 

1

Aug 29

Pixels and image filters

ppt ; pdf

S3.2 (linear filtering)

S3.3 (non-linear filtering)

2

Sep 3

Thinking in frequency

ppt ; pdf

S3.4 (fourier transforms)

S2.3.3 (compression)

FFT Demo (original) ; Python demo by Jae

2

Sep 5

Templates and image pyramids

ppt ; pdf

S3.5.2 (image pyramids)

S8.1.1 (pyramid alignment)

Other reading:

   Burt Adelson 1983 – The Laplacian Pyramid

3

Sep 10

Light and color

ppt ; pdf

S2.2 (light), S2.3.2 (color)

or Forsyth and Ponce Ch 3

3

Sep 12

Histograms and color adjustment

ppt ; pdf

S3.1 (histograms and color adjustment)

Color balance demo

 

Sep 18 (Wed)

Project 1 (Hybrid images) due

 

 

 

The Digital Canvas: Coloring, Blending, Cutting, Synthesizing, and Warping Images

 

 

4

Sep 17

Growing: Texture synthesis and hole filling

ppt ; pdf

Texture Synthesis – Efros Leung (1999)

Image Quilting – Efros Freeman (2001)

Image Analogies – Hertzmann et al. (2001)

Patch Match – Barnes et al. (2009)

4

Sep 19

Cutting: Intelligent Scissors and Graph Cuts

ppt ; pdf

Intelligent Scissors – Mortenson Barret (1995)

GrabCutRother et al. (2004)

5

Sep 24

Pasting: Compositing and blending

ppt ; pdf

Poisson Image Editing – Perez et al. (2003)

Burt and Adelson, A multiresolution spline with application to image mosaics, ACM ToG (1983) 

5

Sep 26

Image warping (translation, rotation, scale, etc.)

ppt ; pdf

S3.6 (warping)

Alignment demo

 

Oct 2 (Wed)

Project 2 (Image quilting) due

 

 

6

Oct 1

Image morphing

ppt ; pdf

 

 

Modeling the Physical World

 

 

6

Oct 3

The Pinhole Camera

ppt ; pdf

S2.1.5 (3D to 2D projection)

7

Oct 8

Single-view Metrology + Camera Model continued

ppt ; pdf

HoiemSavarese-SVG Chapter

7

Oct 10

Single-view 3D Reconstruction

ppt ; pdf

Tour into the picture (Horry et al. 1997)

8

Oct 14 – Oct 20

Midterm Exam (on PrarieLearn)

 

 

9

Oct 22

Image-based Lighting: ray tracing, environment maps, light probes

ppt ; pdf

Reading (do read this):

Debevec, Image-based Lighting Tutorial (2005)

9

Oct 24

Image-based Lighting cont.: HDR light probes, relighting

ppt ; pdf

Optional Reading:

Debevec & Malik, “Recovering High Dynamic Range Radiance Maps from Photographs”, SIGGRAPH 1997

Debevec, Rendering Synthetic Objects in Real Scenes, 1998

 

Oct 30 (Wed)

Project 3 (Gradient domain fusion) due

 

 

10

Oct 29

The image as a virtual stage

ppt ; pdf

Rendering Synthetic Objects (Karsch et al. 2011)

 

Correspondence and Recognition

 

 

10

Oct 31

Matching and alignment with interest points

ppt ; pdf

Grauman/Leibe Draft Chapter on Local Features Optional: Lowe - SIFT paper

 

Nov 1 (Fri)

Final Project Proposal due

 

 

11

Nov 5

Automatic Photo Stitching and RANSAC

ppt ; pdf

Brown Lowe 2007 ; S9 (stitching);  slides

11

Nov 7

Object recognition, retrieval, and augmented reality

ppt ; pdf

Grauman/Leibe Draft Visual Vocabularies; S14.3

 

Nov 13 (Wed)

Project 4 (Image-based lighting) due

 

 

12

Nov 12

Opportunities of scale: texture synthesis, multi-view reconstruction, im2gps, tiny images, etc.

ppt ; pdf

Reading: Hays & Efros, Scene Completion Using Millions of Photographs

 

 

More Topics of Interest

 

 

12

Nov 14

Generating and detecting fakes  

ppt ; pdf

Pix2pix

Cycle GAN

Deep fakes in news

13

Nov 19

How the Kinect works

ppt ; pdf

13

Nov 21

Computational approaches to cameras

ppt ; pdf

Adelson & Bergen, The Plenoptic Function and the Elements of Early Vision (1991)

Ng et al., Light Field Photography with a Hand-held Plenoptic Camera (2005)

Levin et al., Image and Depth with a Coded Aperture (2007)

14 

Nov 23 – Dec 1  

Fall Break

 

 

 

Dec 4 (Wed)

Project 5 (Video stitching) due

 

 

15

Dec 3

Understanding Faces

ppt ; pdf

 

15

Dec 5

Video Magnification

ppt ; pdf

 

 

 

 

 

 

16

Dec 10

Last day – cutting edge and wrap up

ppt ; pdf

 

Dec 15 (Sun)

Final Project due (cannot be late)

 

 

16-17

Dec 13 – Dec 19

Final Exam (on PrairieLearn)

 

 





 


 

 

 

Credits and Course Notes

I am grateful for the many preceding efforts to design excellent courses and course notes for this topic.  I am particularly indebted to Derek Hoiem for designing this course and borrowing much of his course and project design, to Rick Szeliski for making his book available, to James Hays for course suggestions and materials, and to Alyosha Efros, Steve Seitz, Lana Lazebnik, Silvio Savarese, among others for their slides (as credited within).  Anyone who would like to use my slides is more than welcome; please do credit the original sources where appropriate.

 

 

 


Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: GoStats.com