Computational Photography and Image Manipulation CMPT 469 / 985, Fall 2019
Computational Photography and Image Manipulation
CMPT 469 / 985, Fall 2019
Instructor: Yağız Aksoy
3
Layer editing Compositing
Soft Color Segmentation
Aksoy, Aydin, Smolic, Pollefeys, “Unmixing-Based Soft Color Segmentation for Image Manipulation”, ACM Transactions on Graphics, 2017
4
5
6
Semantic Soft Segmentation
Input
Aksoy, Oh, Paris, Pollefeys, Matusik, “Semantic Soft Segmentation”, ACM Transactions on Graphics (Proc. SIGGRAPH), 2018
7
Computational Photography
Computational Photography
How can I use computational techniques to capture light in new ways?
How can I use computational techniques to breathe new life into the photograph?
How can I use computational techniques to synthesize and organize photo collections?
[Hays]
Comp Photo and Related Fields
• Computer Graphics: Models to Images
• Computer Vision: Images to Models
• Computational Photography: Images to Images
Google Pixel phones
Portrait mode iOS
Course outline(Tentative!)
[Wikipedia]
Alim Khan, emir of Bukhara,1911.
Week 1 – Imaging basics
Week 2 – Camera basics
The (in-camera) image processing pipeline
analog front-end
RAW image (mosaiced,
linear, 12-bit)white
balanceCFA
demosaicingdenoising
color transforms
tone reproduction
compressionfinal RGB
image (non-linear, 8-bit)
Week 3 – Frequency domain
Week 4 - High dynamic range (HDR) imaging
[example from www.dpreview.com] [Debevec and Malik, SIGGRAPH 1997]
Week 5 – Bilateral filtering
Week 6 - Color
Retinal vs perceived color.
Berlin, 1945/2010, Mehringdamm
Week 7 – Image compositions
Week 8 – Boundary minimization techniques
Scaling
Michael Rubinstein — MIT CSAIL – [email protected]
Week 9 – Transforms and panoramas
[Brown and Lowe, IJCV 2007]
+ =
Public Library, Stereoscopic Looking Room, Chicago, by Phillips, 1923
Week 10 - Stereo
Week 11 – Motion and Optical Flow
24G. Johansson, “Visual Perception of Biological Motion and a Model For Its Analysis", Perception and Psychophysics
14, 201-211, 1973.
Course structure
• Prerequisites: (i) A computer vision, computer graphics or visual computing course(ii) College-level linear algebra course
• Coding assignments - 4 x 15% = 60%Paper presentation - 30%Paper discussion - 3 x 3.333333% = 10%
Presentations
• Tuesdays: Paper presentations by you!
• CMTP 469: 8 minutes CMTP 985: 15 minutes
• Papers announced on Thursdays for the next Tuesday
• You can propose which papers you would like to present the week before your presentation related to the topic of the week
• SIGGRAPH, SIGGRAPH Asia, Transactions on Graphics, Eurographics, CVPR, ICCV, ECCV, ICCP
• Week assignments for everyone next week
Paper discussions
• 4 minutes discussion after each presentation
• Everyone will have 3 weeks assigned for paper discussion• 2, 5, and 7 weeks after your own presentation (roll over to week 1)
• Interesting closely-related work, shortcomings/strengths of the paper, application scenarios etc.
• You can switch your week assignments with each other, but notify me at least 1 week before the assignment.
Coding assignments
• 4 coding assignments, 2 weeks completion time for each
• Problems related to image processing and manipulation
• Code in whatever you want, but MATLAB is encouraged
• Late policy:• 2 days of penalty-free late submission throughout the semester
• 5% bonus if you don’t use them!
• After that, one-day late: 50% penalty, two-day late: 100% penalty• 1 hour late = 23 hrs late
• Plagiarism policy
Misc.
• We will use Piazza, Coursys, or Canvas, link will be posted later
• Course website: http://yaksoy.github.io/cpim/
• Please no laptops during lectures & presentations• The slides will be posted online
• The best way to take notes is on paper!
• Min. req.
http://yaksoy.github.io/cpim/
Textbook
http://szeliski.org/Book/
http://szeliski.org/Book/
Independent research projects
Slide credits
• Many slides thanks to courses by Alyosha Efros, James Tompkin, and Ioannis (Yannis) Gkioulekas along with their acknowledgements.
34