Computational Photography (CS 445) – Spring 2022  

  

Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: <i>The Key to the Fields</i> (Magritte 1936)

  Instructor:  Yuxiong Wang

 

 

 

  Where: Coursera

 

  Instructor Office Hours: Mon 2:30 PM - 3:30 PM (CT) Zoom

 

  TA: Eric Lee (kylee5), Yuanyi Zhong (yuanyiz2), Ziqi Pang (ziqip2)

  Office Hours: (Yuanyi) Tue 5:00 PM - 6:00 PM (CT) Zoom

  Office Hours: (Eric) Wed 10:00 AM - 11:00 AM (CT) Zoom

  Office Hours: (Ziqi) Thu 9:00 PM - 10:00 PM (CT) Zoom

 

  Syllabus

  Take-Home Questions

  Campuswire Discussion Board

 

  Tutorial on Linear Algebra

  Tutorial on NumPy & Jupyter Notebook

 

 

 

 

  Textbook: Computer Vision: Algorithms and Applications by Rick Szeliski

                                                                                                           

  

   Projects

Project 1: Hybrid Images

Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: leopard elephant

Project 2: Image Quilting

    Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: C:\Users\Hoiem\Documents\Classes\ComputationalPhotography - Fall 2012\projects\quilting\face_toast_cropped.png

   

Project 3: Gradient Domain Fusion

Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: cloning_teaser

 

Project 4: Image-based Lighting

https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQJ6tcY1JugjgyMMpZMj9BgU6OaGyIv8jL4T8J-Qsgnn0UOrHT2tw

 

 

Project 5: Video Stitching

 

 

Final Project

Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: http://t2.gstatic.com/images?q=tbn:ANd9GcSEXEDpOJHSX2MOS0mvolJmNhoGKiQZ6t4ky5dyEio1mph1-L1x

 

 

 

  

  Class Schedule  

Week

Date

Topic

Link

Reading/Notes

1

Jan 18 – Jan 23

Introduction

ppt ; pdf

 

 

Basics of Working with Images

 

 

1

Pixels and image filters

ppt ; pdf

S3.2 (linear filtering)

S3.3 (non-linear filtering)

2

Jan 24 – Jan 30

Thinking in frequency

ppt ; pdf

S3.4 (fourier transforms)

S2.3.3 (compression)

FFT Demo (original) ; Python demo by Jae

2

Templates and image pyramids

ppt ; pdf

S3.5.2 (image pyramids)

S8.1.1 (pyramid alignment)

Other reading:

   Burt Adelson 1983 – The Laplacian Pyramid

3

Jan 31 – Feb 6

Light and color

ppt ; pdf

S2.2 (light), S2.3.2 (color)

or Forsyth and Ponce Ch 3

3

Histograms and color adjustment

ppt ; pdf

S3.1 (histograms and color adjustment)

Color balance demo

 

Feb 9 (Wed)

Project 1 (Hybrid images) due

 

 

 

The Digital Canvas: Coloring, Blending, Cutting, Synthesizing, and Warping Images

 

 

4

Feb 7 – Feb 13

Growing: Texture synthesis and hole filling

ppt ; pdf

Texture Synthesis – Efros Leung (1999)

Image Quilting – Efros Freeman (2001)

Image Analogies – Hertzmann et al. (2001)

Patch Match – Barnes et al. (2009)

4

Cutting: Intelligent Scissors and Graph Cuts

ppt ; pdf

Intelligent Scissors – Mortenson Barret (1995)

GrabCut – Rother et al. (2004)

5

Feb 14 – Feb 20

Pasting: Compositing and blending

ppt ; pdf

Poisson Image Editing – Perez et al. (2003)

Burt and Adelson, A multiresolution spline with application to image mosaics, ACM ToG (1983) 

5

Image warping (translation, rotation, scale, etc.)

ppt ; pdf

S3.6 (warping)

Alignment demo

 

Feb 23 (Wed)

Project 2 (Image quilting) due

 

 

6

Feb 21 – Feb 27

Image morphing

ppt ; pdf

 

 

Modeling the Physical World

 

 

6

The Pinhole Camera

ppt ; pdf

S2.1.5 (3D to 2D projection)

7

Feb 28 – Mar 6

Single-view Metrology + Camera Model continued

ppt ; pdf

HoiemSavarese-SVG Chapter

7

Single-view 3D Reconstruction

ppt ; pdf

Tour into the picture (Horry et al. 1997)

8

Mar 5 – Mar 11

Midterm Exam (on PrarieLearn)

 

 

8-9 

Mar 12 – Mar 20  

Spring Break

 

 

10

Mar 21 – Mar 27

Image-based Lighting: ray tracing, environment maps, light probes

ppt ; pdf

Reading (do read this):

Debevec, Image-based Lighting Tutorial (2005)

10

Image-based Lighting cont.: HDR light probes, relighting

ppt ; pdf

Optional Reading:

Debevec & Malik, “Recovering High Dynamic Range Radiance Maps from Photographs”, SIGGRAPH 1997

Debevec, Rendering Synthetic Objects in Real Scenes, 1998

 

Mar 30 (Wed)

Project 3 (Gradient domain fusion) due

 

 

11

Mar 28 – Apr 3

The image as a virtual stage

ppt ; pdf

Rendering Synthetic Objects (Karsch et al. 2011)

 

Correspondence and Recognition

 

 

11

Matching and alignment with interest points

ppt ; pdf

Grauman/Leibe Draft Chapter on Local Features Optional: Lowe - SIFT paper

 

Apr 1 (Fri)

Final Project Proposal due

 

 

12

Apr 4 – Apr 10

Automatic Photo Stitching and RANSAC

ppt ; pdf

Brown Lowe 2007 ; S9 (stitching);  slides

12

Object recognition, retrieval, and augmented reality

ppt ; pdf

Grauman/Leibe Draft Visual Vocabularies; S14.3

 

Apr 13 (Wed)

Project 4 (Image-based lighting) due

 

 

13

Apr 11 – Apr 17

Opportunities of scale: texture synthesis, multi-view reconstruction, im2gps, tiny images, etc.

ppt ; pdf

Reading: Hays & Efros, Scene Completion Using Millions of Photographs

 

 

More Topics of Interest

 

 

13

Generating and detecting fakes  

ppt ; pdf

Pix2pix

Cycle GAN

Deep fakes in news

14

Apr 18 – Apr 24

How the Kinect works

ppt ; pdf

14

Computational approaches to cameras

ppt ; pdf

Adelson & Bergen, The Plenoptic Function and the Elements of Early Vision (1991)

Ng et al., Light Field Photography with a Hand-held Plenoptic Camera (2005)

Levin et al., Image and Depth with a Coded Aperture (2007)

 

Apr 27 (Wed)

Project 5 (Video stitching) due

 

 

15

Apr 25 – May 1

Understanding Faces

ppt ; pdf

 

15

Video Magnification

ppt ; pdf

 

 

 

 

 

 

16

May 2 – 4

Last day – cutting edge and wrap up

ppt ; pdf

 

May 8 (Sun)

Final Project due (cannot be late)

 

 

16-17

May 6 – May 13

Final Exam (on PrairieLearn)

 

 





 


 

 

 

Credits and Course Notes

I am grateful for the many preceding efforts to design excellent courses and course notes for this topic.  I am particularly indebted to Derek Hoiem for designing this course and borrowing much of his course and project design, to Rick Szeliski for making his book available, to James Hays for course suggestions and materials, and to Alyosha Efros, Steve Seitz, Lana Lazebnik, Silvio Savarese, among others for their slides (as credited within).  Anyone who would like to use my slides is more than welcome; please do credit the original sources where appropriate.

 

 

 


Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: GoStats.com