M. Wu: ENEE631 Digital Image Processing (Spring'09) Human Visual Perception Spring ’09 Instructor: Min Wu Electrical and Computer Engineering Department,

M. Wu: ENEE631 Digital Image Processing (Spring'09)

Human Visual PerceptionHuman Visual Perception

Spring ’09 Instructor: Min Wu

Electrical and Computer Engineering Department,

University of Maryland, College Park

bb.eng.umd.edu (select ENEE631 S’09) [email protected]

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

; 2

00

9)

ENEE631 Spring’09ENEE631 Spring’09Lecture 2 (1/28/2009)Lecture 2 (1/28/2009)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec2 – HVS [2]

OverviewOverview

Last Class– Introduction to DIP/DVP: applications and examples– Image as a function over two spatial coordinates– Sampling and quantization to obtain Digital image

Today– Human visual perception: monochrome vision and color vision

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)


Recap: What is an Image? Recap: What is an Image?

What we perceive as a grayscale image is a pattern of light intensity over a 2-D plane (aka “image plane”)

– Described by a nonnegative real-valued function I(x,y) of two continuous spatial coordinates on an image plane.

– I(x,y) is the intensity of the image at the point (x,y).– An image is usually defined on a bounded rectangle for processing

I: [0, a] [0, b] [0, inf )

Color image:

Represented by 3 functions

R(x,y) for red, G(x,y) for green, B(x,y) for blue.

x

y

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

07

)

x

y

I (x, y)


Obtain Digital Image by Sampling + QuantizationObtain Digital Image by Sampling + Quantization

Computer handles “discrete” data.

Sampling– Sample the value of the image at the nodes of a

regular grid on the image plane.

– A pixel (picture element) at (i, j) is the image intensity value at grid point indexed by the integer coordinate (i, j).

Quantization– Is a process of transforming a real valued sampled

image to one taking only a finite number of distinct values.

– Each sampled value in a 256-level grayscale image is represented by 8 bits.

=> Stay tuned for the theories on these in future weeks.

0 (black)

255 (white)

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


Color Perception and RepresentationColor Perception and Representation


LightLight

Light is an electromagnetic wave– with wavelength of 350nm to 780nm stimulating human visual response

Expressed as spectral energy distribution I()

– The range of light intensity levels that human visual system can adapt is huge: ~ on 10 orders of magnitude (1010) but not simultaneously

– Brightness adaptation: small intensity range to discriminate simultaneously

Figure is from slides at Gonzalez/ Woods DIP book website (Chapter 2)

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)


Color of LightColor of Light

Perceived color depends on spectral content (wavelength composition)

– e.g., 700nm ~ red– “spectral color”: a light with very narrow bandwidth

A light with equal energy in all visible bands appears white

“Spectrum” from http://www.physics.sfasu.edu/astro/color.html

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

& R

.Liu

© 2

00

2)


Perceptual Attributes of Color Perceptual Attributes of Color

Value of Brightness (perceived luminance)

Chrominance– Hue

specify color tone (redness, greenness, etc.)

depend on peak wavelength

– Saturation describe how pure the color is depend on the spread

(bandwidth) of light spectrum reflect how much white light is

added

RGB HSV Conversion ~ nonlinear– Learn more from Gonzalez/Wood Chapter 6

HSV circular cone is from online documentation of Matlab image processing toolbox

http://www.mathworks.com/access/helpdesk/help/toolbox/images/color10.shtml

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

& R

.Liu

© 2

00

2)


The EyeThe Eye


UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)

– Cross section illustration– Retina ~ the “film” in eyes to hold our visual sensors


Two Types of Photoreceptors at RetinaTwo Types of Photoreceptors at Retina

Rods– Long and thin– Large quantity (~ 100 million)– Provide scotopic vision (i.e., dim light vision or at low illumination)– Only extract luminance information and provide a general overall picture

Cones– Short and thick, densely packed in fovea (center of retina)– Much fewer (~ 6.5 million) and less sensitive to light than rods– Provide photopic vision (i.e., bright light vision or at high illumination)– Help resolve fine details as each cone is connected to its own nerve end– Responsible for color vision

– Mesopic vision provided at intermediate illumination by both rod and cones

our interest (well-lighted display)

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

/20

04

)


Absorption of Light by R/G/B ConesAbsorption of Light by R/G/B Cones


UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)


Representation by Three Primary ColorsRepresentation by Three Primary Colors

Three types of photoreceptors (cones) in human retina

– Absorption response Si() has peaks around 450nm (blue), 550nm (green), 620nm (red orange) ~ i.e. short, medium, long wavelength

– Color sensation depends on the spectral response {1(C), 2(C), 3(C) }, instead of the complete light spectrum C()

A color can be reproduced by mixing an appropriate set of three primary colors (Thomas Young, 1802)

S1() C() d

S2() C() d

S3() C() d

C()

color light

1(C)

2(C)

3(C)

Identically perceived colors if i (C1) = i (C2) for i=1,2,3U

MC

P E

NE

E6

31

Slid

es

(cre

ate

d b

y M

.Wu

© 2

00

1/2

00

4)


Example: Seeing Yellow Without YellowExample: Seeing Yellow Without Yellow

mix green and red light to obtain perception of yellow, without shining a single yellow photon

520nm 630nm570nm

=

UM

CP

EN

EE

40

8G

/63

1 S

lide

s (c

rea

ted

by

M.W

u &

R.L

iu ©

20

02

/20

04

)

“Seeing Yellow” figure is from B.Liu ELE330 S’01 lecture notes @ Princeton; R/G/B cone response is from slides at Gonzalez/ Woods DIP book website


RGB Primaries and Color RepresentationRGB Primaries and Color Representation

– Use red, green, blue light to represent a large number of visible colors– The contribution from each primary is normalized to [0, 1]

Color-cube figures: left figure is from B.Liu ELE330 S’01 lecture notes @ Princeton, right figure is from slides at Gonzalez/ Woods DIP book website

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

& R

.Liu

© 2

00

2)


Color Coordinate for PrintingColor Coordinate for Printing

Primary colors for pigment– Defined as one that subtracts/absorbs a

primary color of light & reflects the other two

CMY – Cyan, Magenta, Yellow – Complementary to RGB– Proper mix of them produces black

UM

CP

EN

EE

40

8G

/63

1 S

lide

s (c

rea

ted

by

M.W

u &

R.L

iu ©

20

02

/20

04

)



ExamplesExamples

HSV

YUV

RGB

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

& R

.Liu

© 2

00

2)


Color Coordinates Used in TV TransmissionColor Coordinates Used in TV Transmission

Facilitate sending color video via 6MHz mono TV channel

YIQ for NTSC (National Television Systems Committee) transmission system

– Use receiver primary system (RN, GN, BN) as TV receivers standard

– Transmission system use (Y, I, Q) color coordinate Y ~ luminance, I & Q ~ chrominance I & Q are transmitted in through orthogonal carriers at the

same freq.

YUV (YCbCr) for PAL and digital video– Y ~ luminance, Cb and Cr ~ chrominance

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

& R

.Liu

© 2

00

2)


Human Visual Perception Human Visual Perception

(Monochrome Vision)(Monochrome Vision)


Information Processing by Human ObserverInformation Processing by Human Observer

Visual perception

– Concerns how an image is perceived by a human observer preliminary processing by eye this lecture further processing by brains

– Important to develop image fidelity measures when design and evaluate DIP/DVP algorithms &

systems

imageimage eyeeye perceived perceived imageimage

understanding of content

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


Recall: Two Types of Photoreceptors at RetinaRecall: Two Types of Photoreceptors at Retina

Rods– Long and thin– Large quantity (~ 100 million)– Provide scotopic vision (i.e., dim light vision or at low illumination)– Only extract luminance information and provide a general overall picture

Cones– Short and thick, densely packed in fovea (center of retina)– Much fewer (~ 6.5 million) and less sensitive to light than rods– Provide photopic vision (i.e., bright light vision or at high illumination)– Help resolve fine details as each cone is connected to its own nerve end– Responsible for color vision

– Mesopic vision provided at intermediate illumination by both rod and cones

our interest (well-lighted display)

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

/20

04

)


Luminance vs. BrightnessLuminance vs. Brightness

Luminance (or intensity)– Independent of the luminance of surroundings

I(x,y,) -- spatial light distribution

V() -- relative luminous efficiency function of visual system

(bell shape; different for scotopic vs. photopic vision; highest for green wavelength, second for red, and least for blue )

Brightness– Perceived luminance => Depends on surrounding luminance

Same lum. Different brightness

Different lum.

Similar brightness

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

/20

04

)


Luminance vs. Brightness (cont’d)Luminance vs. Brightness (cont’d)

Example: visible digital watermark– How to make the watermark

appears the same graylevelall over the image?

from IBM Watson web page“Vatican Digital Library”

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


Contrast and Weber’s LawContrast and Weber’s Law

From psychovisual research– HVS more sensitive to luminance contrast than absolute luminance– Eye-brain response to the % changes in intensity is approx. constant

Weber’s Law: | Ls – L0 | / L0 const (0.02)

– Luminance of an object (L0) is set to be just noticeable from luminance of surround (Ls)

– For just-noticeable luminance difference L:

Define C L / L d( log L ) 0.02 equal increments in log luminance are perceived as equally

different

Empirical Luminance-to-Contrast models

C = 50 log10 L (logarithmic law, widely used)

for L [1, 100] and C [0, 100]

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

/20

04

)


Mach BandsMach Bands

Visual system tends to undershoot or overshoot around the boundary of regions of different intensities

Demonstrates that the perceived brightness is not a simple function of light intensity


UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)


dot

dot

Visual Angle and Spatial FrequencyVisual Angle and Spatial Frequency

Visual angle matters more than absolute size and distance– Smaller but closer object vs. larger but farther object– Eyes can distinguish about 25-30 lines per degree in bright illumination

25 lines per degree translate to 500 lines for distance=4 x screenheight

Spatial Frequency– Measures the extent of spatial transition

in unit of “cycles per visual degree”

Visibility thresholds– Eyes are most sensitive to medium spatial freq.

and least sensitive to high frequencies ~ similar to a band-pass filter

– More sensitive to horizontal and vertical changes than other orientations

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

& R

.Liu

© 2

00

2);

EE

63

1 S

’07


Overall Monochrome Vision ModelOverall Monochrome Vision Model

From Jain’s Fig.3.9 (pp57)


Image Fidelity CriteriaImage Fidelity Criteria

Subjective measures– Examination by human viewers– Goodness scale: excellent, good, fair, poor, unsatisfactory– Impairment scale: unnoticeable, just noticeable, … – Comparative measures

with another image or among a group of images

Objective (Quantitative) measures– Mean square error and variations– Pro:

Simple, less dependent on human subjects, & easy to handle mathematically

– Con: Not always reflect human perception

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


Mean-square CriterionMean-square Criterion

Average (or sum) of squared difference of pixel luminance between two images

Signal-to-noise ratio (SNR)– SNR = 10 log10 ( s

2 / e2 ) in unit of decibel (dB)

s2 image variance

e2 variance of noise or error

– PSNR = 10 log10 ( A2 / e2 )

A is peak-to-peak value PSNR is about 12-15 dB higher than SNR

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


SummarySummary Color vision

– Color representations and three primary colors

Monochrome human vision

– visual properties: luminance vs. brightness, etc.– image fidelity criteria

Next time: Pixel-based operation

Reading Assignment:

– Gonzalez’s book: Chapter 2.1-2.2; 6.1-6.2– For further explorations:

Chapter 3 of Jain’s book; J. Woods’ book: Chapter 6.1-6.5 Color Image Proc. Special issue, IEEE Sig Proc Magazine,

Jan. 2005

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

; 2

00

7;2

00

9)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Human Visual Perception Spring ’09 Instructor: Min Wu Electrical and Computer Engineering Department,

Documents