Computer Vision CS 776 Spring 2014 Color Prof. Alex Berg (Slide credits to many folks on individual slides)

Computer VisionCS 776 Spring 2014

Color

Prof. Alex Berg

(Slide credits to many folks on individual slides)

Computer Vision - A Modern Approach

Set: Color Slides by D.A. Forsyth

What is color?

Color is the result of interaction between physical light in the environment and our visual system.

Color is a psychological property of our visual experiences when we look at objects and lights, not a physical property of those objects or lights.

-- S. Palmer, Vision Science: Photons to Phenomenology

The Physics of Light

Any source of light can be completely describedphysically by its spectrum: the amount of energy emitted (per time unit) at each wavelength 400 - 700 nm.

© Stephen E. Palmer, 2002

400 500 600 700

Wavelength (nm.)

# Photons(per ms.)

Relativespectral

power



Causes of colorThe sensation of color is caused by the brain. Some ways to get this sensation include:

• Pressure on the eyelids• Dreaming, hallucinations, etc.

Main way to get it is the response of the visual system to the presence/absence of light at various wavelengths.

Light could be produced in different amounts at different wavelengths (compare the sun and a fluorescent light bulb).

Light could be differentially reflected (e.g. some pigments).It could be differentially refracted - (e.g. Newton’s prism)Wavelength dependent specular reflection - e.g. shiny copper

penny (actually most metals).Flourescence - light at invisible wavelengths is absorbed and

reemitted at visible wavelengths.



Radiometry for colour

All definitions are now “per unit wavelength”

All units are now “per unit wavelength”

All terms are now “spectral”Radiance becomes spectral radiance

• watts per square meter per steradian per unit wavelength

Radiosity --- spectral radiosity

Electromagnetic spectrum

Human Luminance Sensitivity Function

Slide by Svetlana Lazebnik

Computer Vision - A Modern ApproachSet: Color

Slides by D.A. Forsyth

Measurements of relative spectral power of sunlight, made by J. Parkkinen and P. Silfsten. Relative spectral power is plotted against wavelength in nm. The visible range is about 400nm to 700nm. The color names on the horizontal axis give the color names used for monochromatic light of the corresponding wavelength --- the “colors of the rainbow”. Mnemonic is “Richard of York got blisters in Venice”.

Violet Indigo Blue Green Yellow Orange Red



Relative spectral power of two standard illuminant models --- D65 models sunlight,and illuminant A models incandescent lamps. Relative spectral power is plotted against wavelength in nm. The visible range is about 400nm to 700nm. The color names on the horizontal axis give the color names used for monochromatic light of the corresponding wavelength --- the “colors of the rainbow”.

Violet Indigo Blue Green Yellow Orange Red



Measurements of relative spectral power of four different artificial illuminants, made by H.Sugiura. Relative spectral power is plotted against wavelength in nm. The visible range is about 400nm to 700nm.

Spectra of Light Sources

.

# P

hoto

ns

D. Normal Daylight

Wavelength (nm.)

B. Gallium Phosphide Crystal

400 500 600 700

# P

hoto

ns

Wavelength (nm.)

A. Ruby Laser

400 500 600 700

400 500 600 700

# P

hoto

ns

C. Tungsten Lightbulb

400 500 600 700

# P

hoto

ns

Some examples of the spectra of light sources


Rel

. pow

erR

el. p

ower

Rel

. pow

erR

el. p

ower

Reflectance Spectra of Surfaces

Some examples of the reflectance spectra of surfaces

Wavelength (nm)

% L

ight

Ref

lect

ed Red

400 700

Yellow

400 700

Blue

400 700

Purple

400 700




Spectral albedoes for several different leaves, with color names attached. Notice that different colours typically have different spectral albedo, but that different spectral albedoes may result in the same perceived color (compare the two whites). Spectral albedoes are typically quite smooth functions. Measurements by E.Koivisto.

Interaction of light and surfaces

Reflected color is the result of interaction of light source spectrum with surface reflectance


Interaction of light and surfaces

What is the observed color of any surface under monochromatic light?

Olafur Eliasson, Room for one color

http://www.olafureliasson.net/works/room_for_one_colour.html

http://www.olafureliasson.net/works/room_for_one_colour.html

The Eye

The human eye is a camera!• Lens - changes shape by using ciliary muscles (to focus on

objects at different distances)• Pupil - the hole (aperture) whose size is controlled by the

iris• Iris - colored annulus with radial muscles• Retina - photoreceptor cells

Slide by Steve Seitz

Rods and cones, fovea

Rods are responsible for intensity, cones for color perception

Rods and cones are non-uniformly distributed on the retina• Fovea - Small region (1 or 2°) at the center of the visual field containing

the highest density of cones – and no rods


cone

rod

pigmentmolecules

Demonstration of visual acuity

With one eye shut, at the right distance, all of these letters should appear equally legible (Glassner, 1.7).


Blind spot

With left eye shut, look at the cross on the left. At the right distance, the circle on the right should disappear

(Glassner, 1.8).


Rod / Cone sensitivity

Why can’t we read in the dark?Slide by A. Efros


.

400 450 500 550 600 650

RE

LATI

VE

AB

SO

RB

AN

CE

(%)

WAVELENGTH (nm.)

100

50

440

S

530 560 nm.

M L

Three kinds of cones:

Physiology of Color Vision

• Ratio of L to M to S cones: approx. 10:5:1• Almost no S cones in the center of the fovea

Physiology of Color Vision: Fun facts

“M” and “L” pigments are encoded on the X-chromosome• That’s why men are more likely to be color blind http://

www.vischeck.com/vischeck/vischeckURL.php• “L” gene has high variation, so some women may be

tetrachromatic

Some animals have one (night animals), two (e.g., dogs), four (fish, birds), five (pigeons, some reptiles/amphibians), or even 12 (mantis shrimp) types of coneshttp://www.mezzmer.com/blog/how-animals-see-the-world/

http://en.wikipedia.org/wiki/Color_visionSlide by D. Hoiem

http://www.vischeck.com/vischeck/vischeckURL.php



http://www.mezzmer.com/blog/how-animals-see-the-world/

http://en.wikipedia.org/wiki/Color_vision

Color perception

Rods and cones act as filters on the spectrum• To get the output of a filter, multiply its response curve

by the spectrum, integrate over all wavelengths– Each cone yields one number

S

M L

Wavelength

Power

• How can we represent an entire spectrum with 3 numbers?• We can’t! Most of the information is lost

– As a result, two different spectra may appear indistinguishable» such spectra are known as metamers


Metamers

Standardizing color experience

We would like to understand which spectra produce the same color sensation in people under similar viewing conditions

Color matching experiments

Wandell, Foundations of Vision, 1995

Color matching experiment 1

Source: W. Freeman


p1 p2 p3 Source: W. Freeman




p1 p2 p3

The primary color amounts needed for a match

Source: W. Freeman


Source: W. Freeman






p1 p2 p3 p1 p2 p3

We say a “negative” amount of p2 was needed to make the match, because we added it to the test color’s side.

The primary color amounts needed for a match:

p1 p2 p3

Source: W. Freeman

Trichromacy

In color matching experiments, most people can match any given light with three primaries• Primaries must be independent

For the same light and same primaries, most people select the same weights• Exception: color blindness

Trichromatic color theory• Three numbers seem to be sufficient for encoding

color• Dates back to 18th century (Thomas Young)


Grassman’s Laws (~linearity)

Color matching appears to be linearIf two test lights can be matched with the

same set of weights, then they match each other: • Suppose A = u1 P1 + u2 P2 + u3 P3 and B = u1 P1 + u2 P2 +

u3 P3. Then A = B.

If we mix two test lights, then mixing the matches will match the result:• Suppose A = u1 P1 + u2 P2 + u3 P3 and B = v1 P1 + v2 P2 +

v3 P3. Then A + B = (u1+v1) P1 + (u2+v2) P2 + (u3+v3) P3.

If we scale the test light, then the matches get scaled by the same amount:• Suppose A = u1 P1 + u2 P2 + u3 P3.

Then kA = (ku1) P1 + (ku2) P2 + (ku3) P3. Slide by Svetlana Lazebnik

Linear color spaces

Defined by a choice of three primaries The coordinates of a color are given by

the weights of the primaries used to match it

mixing two lights producescolors that lie along a straight

line in color space

mixing three lights produces colors that lie within the triangle

they define in color space


Linear color spaces

How to compute the weights of the primaries to match any spectral signal?

p1 p2 p3

?

Given: a choice of three primaries and a target color signal

Find: weights of the primaries needed to match the color signal

p1 p2 p3


Linear color spaces

In addition to primaries, need to specify matching functions: the amount of each primary needed to match a monochromatic light source at each wavelength

RGB matching functionsRGB primaries


Linear color spaces

How to compute the weights of the primaries to match any spectral signal?

Let c(λ) be one of the matching functions, and let t(λ) be the spectrum of the signal. Then the weight of the corresponding primary needed to match t is

λ

Matching functions, c(λ)Signal to be matched, t(λ)

dtcw )()(


RGB space

Primaries are monochromatic lights (for monitors, they correspond to the three types of phosphors)

Subtractive matching required for some wavelengths

RGB matching functionsRGB primaries


Comparison of RGB matching functions with best 3x3 transformation of cone responses

Wandell, Foundations of Vision, 1995


Linear color spaces: CIE XYZPrimaries are imaginary, but matching

functions are everywhere positiveThe Y parameter corresponds to brightness

or luminance of a color2D visualization: draw (x,y), where

x = X/(X+Y+Z), y = Y/(X+Y+Z) Matching functions

http://en.wikipedia.org/wiki/CIE_1931_color_spaceSlide by Svetlana Lazebnik

http://en.wikipedia.org/wiki/CIE_1931_color_space

Uniform color spacesUnfortunately, differences in x,y coordinates do

not reflect perceptual color differencesCIE u’v’ is a projective transform of x,y to make

the ellipses more uniform

McAdam ellipses: Just noticeable differences in color


Nonlinear color spaces: HSV

Perceptually meaningful dimensions: Hue, Saturation, Value (Intensity)

RGB cube on its vertex


Color perception

Color/lightness constancy• The ability of the human visual system to perceive

the intrinsic reflectance properties of the surfaces despite changes in illumination conditions

Instantaneous effects• Simultaneous contrast• Mach bands

Gradual effects• Light/dark adaptation• Chromatic adaptation• Afterimages

J. S. Sargent, The Daughters of Edward D. Boit, 1882 Slide by Svetlana Lazebnik

Checker shadow illusion

http://web.mit.edu/persci/people/adelson/checkershadow_illusion.html


Checker shadow illusion

Possible explanations• Simultaneous contrast• Reflectance edges vs. illumination edges



Color Constancy by “Gamut” Approach…

Limited range of material reflectance properties

Limited range of possible illuminantsGiven observed colors we can work out

possible range of illuminants, combine those with some prior over known illuminants and obtain

More reading & thought problems

David Forsyth’s “Gamut” based reasoning for color constancy (IJCV 1990) http://luthuli.cs.uiuc.edu/~daf/papers/colorconst.pdf

I am very interested in pursing the line of reasoning I mentioned in class, where the above ideas for color constancy are extended to texture recognition.

http://luthuli.cs.uiuc.edu/~daf/papers/colorconst.pdf



Computer Vision CS 776 Spring 2014 Color Prof. Alex Berg (Slide credits to many folks on individual slides)

Documents

color slides

forsyth slide

color names

relative spectral power

sensation of color

spectral radiosity slide

forsyth causes of color

phenomenology slide