Top Banner
15-463 (15-862): Computational Photography
53

15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Aug 25, 2018

Download

Documents

lamngoc
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

15-463 (15-862): Computational Photography

Page 2: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

15-463 (15-862): Computational Photography

Staff• Prof: Alexei Efros (efros@cs), 4207 NSH

• TAs: Alvaro Collet (acollet@cs) and Laura Trutoiu (trutoiu@cs)

Web Page• http://graphics.cs.cmu.edu/courses/15-463/

Discussion Forum:• googlegroups

Page 3: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Today

Introductions

Why Computational Photography?

Overview of the course

Administrative stuff

Page 4: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

A bit about me

Alexei (Alyosha) Efros

Assistant Professor in Robotics and CSD

also work with colleagues in Paris and Oxford

Teaching

The plan is to have fun and learn cool things, both

you and me!

Social warning: I don’t see well

Research

Graphics, Vision

Page 5: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

PhD Thesis on Texture and Action Synthesis

Antonio’s son cannot walk but he can fly

Smart Erase button in MS Digital Image Pro:

Page 6: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

More recent work

Derek Hoiem, Alexei Efros, Martial Hebert

Page 7: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros
Page 8: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros
Page 9: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros
Page 10: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros
Page 11: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros
Page 12: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

A super-brief History of Art

and its futile Search for Realism

Why Computational Photography?

Page 13: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Depicting Our World: The Beginning

Prehistoric Painting, Lascaux Cave, France

~ 13,000 -- 15,000 B.C.

Page 14: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

The Empress Theodora with her court.

Ravenna, St. Vitale 6th c.

Depicting Our World: Middle Ages

Page 15: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Nuns in Procession. French ms. ca. 1300.

Depicting Our World: Middle Ages

Page 17: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Depicting Our World: Renaissance

Piero della Francesca,

The Flagellation (c.1469)

Page 18: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Depicting Our World: Toward Perfection

Jan van Eyck, The Arnolfini Marriage (c.1434)

Page 19: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Depicting Our World: Toward Perfection

Lens Based Camera Obscura, 1568

Page 20: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Depicting Our World: Perfection!

Still Life, Louis Jaques Mande Daguerre, 1837

Page 21: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Depicting Our World: Realism?

Page 22: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Flickr Paris

Page 23: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Real Paris

Page 24: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Real Notre Dame

Page 25: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Depicting Our World: Ongoing Quest

Pablo Picasso David Hockney

Page 26: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Better than realism?

David Hockney, Place Furstenberg, (1985)

Page 27: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Alyosha Efros

Place Furstenberg, 2009

Which one is right?

Multiple viewpoints Single viewpoint

David Hockney,

Place Furstenberg, 1985

Page 28: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Depicting Our World: Ongoing Quest

Antonio Torralba & Aude Oliva (2002)

Page 29: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Enter Computer Graphics...

Page 30: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

GRAPHICS

Traditional Computer Graphics

3D geometry

physics

Simulation

projection

Page 31: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

State of the Art

•Amazingly real

•But so sterile, lifeless, futuristic (why?)

Page 32: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

The richness of our everyday world

Photo by Svetlana Lazebnik

Page 33: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Beauty in complexity

University Parks, Oxford

Page 34: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Which parts are hard to model?

Photo by Svetlana Lazebnik

Page 35: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

People

From “Final Fantasy”

On the Tube, London

Page 36: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Faces / Hair

Photo by Joaquin Rosales Gomez

From “Final Fantasy”

Page 37: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Hyper-humans

Page 38: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Urban Scenes

Virtual LA (SGI)

Photo of l LA

Page 40: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

The Realism Spectrum

+ easy to create new worlds

+ easy to manipulate

objects/viewpoint

- Very hard to look realistic

+ instantly realistic

+ easy to aquire

- very hard to manipulate

objects/viewpoint

Computer Graphics PhotographyComputational

Photography

Realism

Manipulation

Ease of capture

Page 41: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Virtual Real World

Campanile Moviehttp://www.debevec.org/Campanile/

Page 42: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Course Outline

Page 43: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Programming Project 0

The Vertigo Effect

Page 44: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Programming Project 1

Images of the Russian Empire -- colorizing the

Prokudin-Gorskii photo collection

Page 45: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Programming Project 2

Image Resizing by Scene Carving

Page 46: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Programming Project 3

Face warping and morphing

Page 47: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Programming Project 4

Photo Mosaics

Full screen panoramas (cubic): http://www.panoramas.dk/

Mars: http://www.panoramas.dk/fullscreen3/f2_mars97.html

2003 New Years Eve: http://www.panoramas.dk/fullscreen3/f1.html

Page 48: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Programming Project 4

Automatic Mosaic Stitching

Page 49: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Programming Project 5

Tour Into the Picture

Page 50: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Final Project

Something cool!!!

Page 51: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Administrative Stuff

Grading• Written and Programming Assngments (60%)

• Exam (20%)

• Final Project (20%)

• Class Participation: priceless

Late Policy• Five late days total, to be spent wisely

• 20% off from each extra late day

Cheating• Let’s not embarrass ourselves

Hardware/Software• CMU clusters

• MATLAB!!!

Page 52: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

General Comments

Prerequisites

• Linear algebra!!!

• Some computer graphics, vision, or image processing is

useful, but not required.

Emphasis on programming projects!• Building something from scratch (Matlab!)

Why you should not take this class?

plenty of reasons…

Page 53: 15-463 (15-862): Computational Photography - …graphics.cs.cmu.edu/courses/15-463/2010_spring/Lectures/... · 15-463 (15-862): Computational Photography Staff • Prof: Alexei Efros

Cameras

Really cool

Not too expensive nowadays (<$150)

e.g. Canon A1100