Oct 10, 20051 Image Formation: Light Sources + Reflectance + Sensors Light is produced in different amounts at different wavelengths by each light source.

Oct 10, 2005 1

Image Formation: Light Sources + Reflectance + Sensors

• Light is produced in different amounts at different wavelengths by each light source

• Light is differentially reflected at each wavelength, which gives objects their natural colours (surface albedoes)

• The sensation of colour is determined by the human visual system, based on the product of light and reflectance

Credits: Many slides from Jim Rehg, Frank Dellaert and David Forsyth

Oct 10, 2005 2

Image Formation

Light Sources

Object

Transmittance

Absorption

Reflectance

Observer

Ambient

Point Uniform

Oct 10, 2005 3

The multiplicative model E()() holds for many types of surfaces (not all, e.g. feathers). Interpretation: wavelengths do not interfere

Image Formation

Oct 10, 2005 4

Sensor response: Spectral integration

E() spectral power distribution of the light source

) spectral albedo

f() spectral sensitivity of sensor f; for human vision and standard cameras f 2 {R,G,B}

Illumination Reflectance Observer

Oct 10, 2005 5

The Dichromatic Reflection Model

• So far, we have ignored geometry.

i: incidence angle

e: exit angle

g: phase angle

i: interface reflected light

b: body reflected light

Oct 10, 2005 6

Dichromatic Reflection Model

•For most surfaces, the spectral albedo’s dependence on photometric angles can be described by the dichromatic model:

(λ, i, e, g) = i(λ, i, e, g) + b(λ, i, e, g)

furthermore

)=cb()mb()+cI()mI(),

where cb()mb(is the body reflectance and cI()mI() is the interface reflectance

The vector (i,e,g) consists of the three photometric angles.

Note that the form constraints the colour of a single surface patch under a given illumination to lie in a plane.

Oct 10, 2005 7

Dichromatic Reflection Model

Reflected Light

Body Reflection

Interface (Surface) Reflection

Shafer S.A. 1985

)()()()()( ,,, yxyxyx IIBB CmCmC

Type INeutral Interface Reflection,Objects with high oil, water content

Type II”Full” Dichromatic Reflection ModelObjects as silk, wool, coloured

paper

Type IIISpecial Version of the Dichromatic Reflection ModelAdaptable for Metals

Tominaga, 1994

•Interface reflection is mirror-like, mI() is close to a delta function. The commonly used Phong model assumes mI() =cosk(), where is measured from the mirror direction, k is a use parameter.•Body reflection is often isotropic, the patch has the same intensity from all viewpoints. Patch intensity depends on the orientation of the normal w.r.t. the illumination source. The commonly used Lambertian model assumes $mb() = cos(), where iis the incident angle, i.e. the angle between the surface normal and the light source direction.

Oct 10, 2005 8

Dichromatic Reflection Model

)()()()()( ,,, yxyxyx SSBB CmCmC

Oct 10, 2005 9

R

G

B

Illuminant color

Diffuse component

T

S

Skewed-T in Histogram

A Physical Approach to colour Image Understanding – Klinker, Shafer, and Kanade. IJCV 1990

Figure courtesy ofD. Forsyth

Oct 10, 2005 10R

G

B

R

G

B

Diffuseregion

Boundary ofspecularity

Figure courtesy ofD. Forsyth

Skewed-T in Histogram

Oct 10, 2005 11

Dichromatic Reflection Model in Chromaticity Representation

Chromaticity = Colour modulo intensity

BGRR

R

BGR

GG

BGRB

B

1 BGR

Oct 10, 2005 12

Gamma Correction

• The dichromatic model is only valid for linear cameras!

• The phosphor dots are not a linear system (voltage vs. intensity)

Oct 10, 2005 13

Gamma correction

• Without gamma correction, how will (0,255,127) look like?• Normally gamma is within 1.7 and 2.8• Who is responsible for Gamma correction?• SGI does it for you• PC/Mac etc, you should do it yourself

Oct 10, 2005 14

No gamma correction

Oct 10, 2005 15

Gamma corrected to 1.7

Oct 10, 2005 16

Residual Gamma or System Gamma

• Systems such as SGI monitor has a gamma of 2.4, but they only gamma correct for 1.7.

• The residue gamma is 2.4/1.7 = 1.4, why?• Depends on how you see it? Bright screen, dark room causes

changes in your eye transfer function too.• What about web pages? Which screen do you intend for?

Oct 10, 2005 17

Illumination 1.

If Spectra of Light Source Changes

Spectra of Reflected Light Changes

Oct 10, 2005 18

Illumination 2.

Daylight

Tungsten

Fluorescence

Oct 10, 2005 19

Modelling Source of Light: Most Common Lights are Close to Black Body Radiators.

Judd et al., 1964CIE standard

“Typical Daylight” 5700KRange 4000 – 25000K

Black Body

T, Kelvin

Oct 10, 2005 20

Black body radiators

• Construct a hot body with near-zero albedo (black body)– Easiest way to do this is to build a hollow metal object with a

tiny hole in it, and look at the hole.• The spectral power distribution of light leaving this object is a

function of temperature (degrees Kelvin)• This leads to the notion of colour temperature --- the

temperature of a black body that would look the same– Candle flame or sunset: about 2000K– Incandescent light bulbs: 3000K– Daylight (sun): 5500K– Blue sky (shadowed from sun): 15,000K

• Camera film must be chosen according to light source

Oct 10, 2005 21

The black-body locus (the colours of heated black-bodies).

Oct 10, 2005 22

Measurements of relative spectral power of sunlight, made by J. Parkkinen and P. Silfsten. Relative spectral power is plotted against wavelength in nm. The visible range is about 400nm to 700nm. The colour names on the horizontal axis give the colour names used for monochromatic light of the corresponding wavelength.

Violet Indigo Blue Green Yellow Orange Red

Oct 10, 2005 23

Relative spectral power of two standard illuminant models --- D65 models sunlight,and illuminant A models incandescent lamps. Relative spectral power is plotted against wavelength in nm. Violet Indigo Blue Green Yellow Orange Red

Oct 10, 2005 24

Oct 10, 2005 25

Spectral albedoes for several different flowers, with colour names attached. Notice that different colours typically have different spectral albedo, but that different spectral albedoes may result in the same perceived colour (compare the two whites). Spectral albedoes are typically quite smooth functions. Measurements by E.Koivisto.

Spectral albedo’s of common materials:

Oct 10, 2005 26

Lighting and Illuminants(3)

Oct 10, 2005 27

Lighting and Illuminants(5)

Oct 10, 2005 28

reflectance spectrophotometer to acquire spectral reflectance data

Oct 10, 2005 29

Spectral Albedo

Oct 10, 2005 30

Spectral Albedo

Oct 10, 2005 31

Additive colour Mixing

Oct 10, 2005 32

Subtractive colour Mixing

Oct 10, 2005 33

colour matching experiments - I

• Show a split field to subjects; one side shows the light whose colour one wants to measure, the other a weighted mixture of primaries (fixed lights).

Oct 10, 2005 34

colour Matching Process

Basis for industrial colour standards

Oct 10, 2005 35

colour Matching Experiment 1

Image courtesy Bill Freeman

Oct 10, 2005 36

colour Matching Experiment 1

Image courtesy Bill Freeman

Oct 10, 2005 37

colour Matching Experiment 1

Image courtesy Bill Freeman

Oct 10, 2005 38

colour Matching Experiment 2

Image courtesy Bill Freeman

Oct 10, 2005 39

colour Matching Experiment 2

Image courtesy Bill Freeman

Oct 10, 2005 40

colour Matching Experiment 2

Image courtesy Bill Freeman

Oct 10, 2005 41

Colour matching experiments - II

• Many colours can be represented as a positive weighted sum of A, B, C

• write M=a A + b B + c C

where the = sign should be read as “matches”• This is additive matching. • Gives a colour description system - two people who agree on A, B,

C need only supply (a, b, c) to describe a colour.

Oct 10, 2005 42

Subtractive matching

• Some colours can’t be matched like this: instead, must write

M+a A = b B+c C• This is subtractive matching.• Interpret this as (-a, b, c)• Problem for building monitors: Choose R, G, B such that positive

linear combinations match a large set of colours

Oct 10, 2005 43

The principle of trichromacy

• Experimental facts:– Three primaries will work for most people if we allow subtractive

matching• Exceptional people can match with two or only one primary.• This could be caused by a variety of deficiencies.

– Most people make the same matches.• There are some anomalous trichromats, who use three

primaries but make different combinations to match.

Oct 10, 2005 44

Human Photoreceptors

Fovea Periphery

Oct 10, 2005 45

Human Cone Sensitivities

• Spectral sensitivity of L, M, S cones in human eye

Oct 10, 2005 46

Grassman’s Laws

Oct 10, 2005 47

Linear colour spaces

• A choice of primaries yields a linear colour space --- the coordinates of a colour are given by the weights of the primaries used to match it.

• Choice of primaries is equivalent to choice of colour space.

• RGB: primaries are monochromatic energies are 645.2nm, 526.3nm, 444.4nm.

• CIE XYZ: Primaries are imaginary, but have other convenient properties. colour coordinates are (X,Y,Z), where X is the amount of the X primary, etc.

Oct 10, 2005 48

• monochromatic• 645.2, 526.3, 444.4 nm.• negative parts -> some colours can be matched only subtractively.

RBG colour Matching

Figure courtesy ofD. Forsyth

Oct 10, 2005 49

CIE XYZ: colour matching functions are positive everywhere, but primaries are imaginary. Usually draw x, y, where x=X/(X+Y+Z)y=Y/(X+Y+Z)So overall brightness is ignored.

CIE XYZ colour Matching

Figure courtesy ofD. Forsyth

Oct 10, 2005 50

Geometry of colour (CIE)

• White is in the center, with saturation increasing towards the boundary

• Mixing two coloured lights creates colours on a straight line

• Mixing 3 colours creates colours within a triangle

• Curved edge means there are no 3 actual lights that can create all colours that humans perceive!

Oct 10, 2005 51

RGB colour Space

The colours that can be displayed on a typical computer monitor (phosphor limitations keep the space quite small)

Oct 10, 2005 52

Uniform colour spaces

• McAdam ellipses (next slide) demonstrate that differences in x,y are a poor guide to differences in colour– Each ellipse shows colours that

are perceived to be the same• Construct colour spaces so that

differences in coordinates are a good guide to differences in colour.

• Two spaces are commonly used: Lab and Luv

• The “uniformity” applies only to small differences in colour

008856.0 if , 116

16787.7

008856.0 if , )( where

)()(200

)()(500

008856.0 if , )(3.903

008856.0 if , 16)(116

3

1

*

*

3

1

*

tt

tttf

Z

Zf

Y

Yfb

Y

Yf

X

Xfa

Y

Y

Y

YY

Y

Y

Y

L

nn

nn

nn

nn

Oct 10, 2005 53

Figures courtesy ofD. Forsyth

McAdam ellipses

10 times actual size Actual size

Oct 10, 2005 54

Human colour Constancy

• Colour constancy: determine hue and saturation under different colours of lighting

• Lightness constancy: gray-level reflectance under differing intensity of lighting

• Humans can perceive– colour a surface would have under white light – colour of reflected light (separate surface colour from measured

colour)– colour of illuminant (limited)

Oct 10, 2005 55

Land’s Mondrian Experiments

• Squares of colour with the same colour radiance yield very different colour perceptions

Photometer: 1.0, 0.3, 0.3 Photometer: 1.0, 0.3, 0.3

Audience: “Red” Audience: “Blue”White light Red light

Blue

Red

Blue

Red

Oct 10, 2005 56

Basic Model for Lightness Constancy

• Assumptions:– Planar frontal scene– Lambertian reflectance– Linear camera response

• Modeling assumptions for scene– Piecewise constant surface reflectance– Slowly-varying Illumination

)()()( xpxIkxC c

Oct 10, 2005 57

1-D Lightness “Retinex”

Threshold gradient image to find surface (patch) boundariesFigure courtesy ofD. Forsyth

Oct 10, 2005 58

1-D Lightness “Retinex”

Integration to recover surface lightness (unknown constant)

Figure courtesy ofD. Forsyth

Oct 10, 2005 59

colour Retinex

Images courtesy John McCann

Oct 10, 2005 60

Colour constancy

• Following methods have been used:– Average reflectance across scene is known (often fails)– Brightest patch is white– Gamut (collection of all colours) falls within known range– Known reference colour (colour chart, skin colour…)– Specular reflections have the colour of the illumination

• This is an open problem