CS 423 (CS 423/CS 523) Computer Vision Lecture 1 INTRODUCTION TO COMPUTER VISION.

CS 423 (CS 423/CS 523)

Computer Vision

Lecture 1INTRODUCTION TO COMPUTER VISION

About the Course

2

http://vvgl.ozyegin.edu.tr

Objective

Introduction to the theory, tools, and algorithms of computer vision

Instructor

Assist. Prof. M. Furkan Kıraç

E-mail: [email protected]

Room: 219

Hours

Mondays, 9:40-12:30, Room: 246

Grading

Projects: 4x15%

Midterm Exam: 40%

Syllabus

3

Projects:Late submissions are not accepted. Copying answers from others’ work is not permitted.

Midterm Exam:At least 3 of the 4 Projects must be turned in by the due date in order to qualify for the Final Exam. No Composite Exam (Bütünleme Sınavı), as there is no final exam.  

Grading

4

Computer Vision: Algorithms and Applications, Richard Szeliski, Springer, 2010.

Computer Vision: A Modern Approach, David A. Forsyth and Jean Ponce, Prentice-Hall, 2002.

  Introductory Techniques for 3D Computer

Vision, Emanuele Trucco and Alessandro Verri, Prentice-Hall 1998.

Recommended Books

5

OpenCV Computer Vision Application Programming Cookbook Second Editon, Robert Laganiere, Packt Publishing, 2014.

Learning OpenCV, Gary Bradski and Adrian Kaehler, O'Reilly, 2008.

Mastering OpenCV with Practical Computer Vision Projects, Daniel Lelis Baggio, et al., Packt Publishing, 2012.

OpenCV Resources

6

Applications of Computer Vision

7

Image Stitching

Image Matching

Object Recognition

3D Reconstruction

Interior Modeling

12

3D Augmented Reality

13

3D Camera Tracking

14

15

Stereo Conversion for 3DTV

Depth Estimation and View Interpolation for 3DTV

16

Human Tracking

17

License Plate Recognition

18

Human Pose Estimation

19

Course Outline

20

Linear Filters, Frequency Domain Filtering, Edge and Boundary Detection Feature Detection Fitting, Alignment Histograms Covariance, Principle Component Analysis (PCA) Face Detection and PCA Optical Flow and Motion Tracking and Mean-Shift Randomized Decision Trees, Pose Estimation Bag of Features Context, Two-View Geometry Summary

Topics to be covered...

21

Relation to Other Fields

22

Computer Vision

23

Figure from "Computer Vision: Algorithms and Applications,” Richard Szeliski, Springer, 2010.

Lights and materials Shading Texture mapping Environment effects Animation 3D scene modeling 3D character modeling (OpenGL)

Computer Graphics

24

Computer Graphics

25

Resampling Enhancement Noise filtering Restoration Reconstruction Segmentation Image compression (MATLAB and OpenCV)

Image Processing Topics

26

Image Processing

27

Motion estimation Frame-rate conversion Multi-frame noise filtering Multi-frame restoration Super-resolution Video compression (MATLAB & OpenCV)

Video Processing Topics

28

Video acquisition-display chain

29

Capture Representation Coding

Transmission Decoding Rendering

Human vs. Computer

30

Optical illusions

Actual vs. Perceived Intensity (Mach band effect)

32

Brightness Adaptation of the Eye

33

Optical illusions

Optical illusions

Why is Computer Vision Difficult?

Human perception

Human perception

Human Visual System

40

Human Eye

Photoreceptors: Rods & Cones

Rods vs. Cones

RodsPerceive brightness onlyNight vision

ConesPerceive colorDay visionRed, green, and blue cones

Cone Distribution

64%

32%

2%

Blue is less-focused

Visual Threshold drop during Dark Adaptation

Spatial Resolution of the Human Eye Photopic (bright-light) vision:

Approximately 7 million cones Concentrated around fovea

Scotopic (dim-light) vision Approximately 75-150 million rods Distributed over retina

(HDTV: 1920x1080 = 2 million pixels)

49

Frequency Responses of Cones

Same amount of energy produces different sensations of brightness at different wavelengths

Green wavelength contributes most to the perceived brightness.

50

Trichromatic Color Mixing

Any color can be obtained by mixing three primary colors Red, Green, Blue (RGB) with the right proportion

valuessTristimulu :,3,2,1

kk

kk TCTC

Image Formation

53

Human Eye vs. Camera

Camera components Eye components

Lens Lens, cornea

Shutter Iris, pupil

Film Retina

Cable to transfer images Optic nerve to send the incident light information to the brain

Human Vision

Image formation

Pin-Hole Camera Model

Point Spread Effect

Out-of-Focus Blur

Shrinking the Aperture

Converging Lens

Correction with a Converging Lens

Perfectly In-Focus for a Certain Distance Only

“circle of confusion”

Depth-of-Field

Depth-of-Field

“Sharp Image” within Depth-of-Field due to Finite Sensor Size

NZFZ

Focal Length (F) and Depth (Z)

Z

YFy

F

y

Z

Y

Z

XFx

Aperture Size Affects Depth-Of-Field

f / 5.6

f / 32

Aperture

2dA

Camera f-number

d

Ff

2

f

FA

Exposure Time

Motion Blur Effect due to Finite Exposure Time

Decrease in aperture implies…

Increase in depth-of-field Decrease in motion blur Decrease in exposure

2D Image Representation

75

76

Image Capture

(Courtesy Gonzalez & Woods)

Digital Image Capture

Digital Image Capture

Light sensitive diodes convert photons to electrons

Color Image Capture: Single vs. Three CCD Arrays

RGB splitter(three separate imaging sensors, higher resolution)

Bayer filter(cheaper but introduces spatial resolution loss)

Digital Camera Issues

Noise caused by low light

Color color fringing (chromatic aberration) artifacts from Bayer patterns

Blooming charge overflowing into neighboring pixels

In-camera processing over-sharpening can produce halos

Compression creates blocking artefacts

Digitization: Sampling and Quantization

Sampling Rate Problem

Over Quantization

83

84

Images as Matrices of Integers

126 127 126

125 126 127

123 126 125

128 127 124

123 120 144

121 128 155

126 123 127

120 122 124

119 121 123

122 142 162

130 157 161

145 162 164

158

163

160

164

166

165

m

n

(0,0)

0 ≤ s(m,n) ≤ 255 } quantization

0 ≤ m ≤ M-1

0 ≤ n ≤ N-1

MxN 8-bit gray-scale (intensity, luminance) image

sampling

0 → black, 255 → white

Images as Functions

We can think of an image as a function, f, from R2 to R: f( x, y ) gives the intensity at position ( x, y ) Realistically, we expect the image only to be defined

over a rectangle, with a finite range:• f: [a,b]x[c,d] [0,1]

A color image is just three functions pasted together. We can write this as a “vector-valued” function:

( , )

( , ) ( , )

( , )

r x y

f x y g x y

b x y

RGB Color Bands (Channels)

Red

Green Blue

YUV Bands

Also called Y Cb Cr Y : Luma

Cb : Chrominance_blueCr : Chrominance_red

Y

U (Cb)

V(Cr)

Color

YUV-RGB Conversion

Summary

89

Human visual system

Pin-hole camera model

Image representation

Summary

90