1 Chapter 6: Color Preview 。 The world is colorless 。 Color is caused by the vision system responding differently to different wavelengths of light 。 Brightness.

1

Chapter 6: ColorPreview。 The world is colorless。 Color is caused by the vision system responding differently to different wavelengths of light。 Brightness is caused by summing different magnitudes of wavelengths of light

Cortex: Ordered Feature Map

2

6.1 Physics of Color A color we perceive is resulting from (a) the color of object surface (b) the colors of light sources

e.g. Tonotopic maps: auditory cortex Geographic maps: hippocampal cortex Somatic maps: somatosensory cortex Retinotectal maps: visual cortex

visual cortex auditory cortex

3

6.1.1 Colored Lights

○ Spectral (wavelength) units (quantities)

-- Units with the phrase “per unit wavelength”

e.g., Spectral radiance

Spectral irradiance

Spectral BRDF

Spectral exitance

4

The distribution of spectral radiation

where : color temperature, h : Plank’s constant k : Boltzmann’s constant c : speed of light, : wavelength

51 1

( ) ( )( )exp( / ) 1

Ehc k T

6.1.2 The Color of Sources

○ Black body: absorbs light without reflection

T

5

○ Sun – a distant bright point source

Light from the sun (i) is scattered by the air, (ii) strikes a surface, and (iii) is reflected into camera or eye

Airlight (Skylight)

Sunlight (Daylight)

6

(b) Natural model – air emits a constant amount of light per unit volume

sky is substantially brighter at the horizon than at the zenith because a viewing ray along the horizon passes through more sky

However,

○ Sky: (a) Crude geometrical model -- a hemisphere with constant exitance

7

○ Illumination during the day by: (a) Daylight (sunlight), (b) Skylight (airlight)

wavelength

8

○ Application – Dehazing

9

An image is contributed by two light sources I = D + A, where D: direct light, A: airlight

10

Direct light : the light emitted from the object and passes through the air

( ) ( ) ( )D x J x t z

x : image pixel; z: object distanceJ(x) : light emitted from object surfacet(z) : atmosphere transmittance

0( ) exp[ β( , ) ]

zt z z dz

0 0( ) exp[ β( ) ] exp[ β( ) ]

exp[ β( ) ]

z zt z dz dz

z

scatter function

11

Airlight A: the amount of light within the conical volume

12

13

14

15

(b) The intensity of spectral radiation scattered by a unit volume of air depends on the 4th power of frequency, i.e.,

(c) The sun looks yellow; the sky looks blue

atmosphere

○ (a) Light of a long wavelength can travel farther than light of a short wavelength (Rayleigh scattering)

4 4R f

16

○ Application - Shadow Detection

Cast shadows Self shadowsShadow

17

(1) Intensity test -- a shadow area should be darker than its corresponding background areas

Input Background Foreground

Dark regions

18

(2) Blue ratio test -- at a shadow point p

sun sky ,L L L 0 1

sun skyL L L

Shadow areas

Non-shadow area

sky sun ,i iiL L 1 0b r g

sun sky sun sun sun( )i i i i i ii iL L L L L L

, , ,i r g b

19

sun

sun

( )

(1 ) 1

i ii i

i ii i

L L

L L

( ) ( ) ( ) ( )

( ) ( ) ( ) ( ) 1

i i ii

i i ii

I p L P R P L P

I p L P R P L P

( )( ) ( ),

(1 ) (1 ) (1 )gb r

b r g

( ) ( ) ( )

,( ) ( ) ( )

b r g

b r g

I p I p I p

I p I p I p

(needs to be proven)

20

Shadow areas Dark regions

(3) Reflectance test -- distinguish between cast and self shadows

sun sky( )i i i iI L L R

Non-shadow image area

Shadow image areas

sun sky( )i i i iI L L R

sun(1 )i i i i iI I I L R

21

sun

sun

(1 ),

(1 ) 3

i i i i

i

I L R Rr

I L R R

Normalization:

( , , )r g b TI I I I , , ,i r g b

Training of different materials( , , )r r g br r r

Self shadows

Cast shadows

22

Fluorescent light: high speed electrons strike gas; gas releases ultraviolet radiation; the radiation causes phosphors to fluoresce

○ Artificial Illuminants

Incandescent light: metal filament (e.g., tungsten) is heated to a high temperature

Arc lamp: contains gaseous metal (e.g., mercury) and inert gases; light is produced by electrons in metal atoms dropping from an excite state to a lower energy state

23

24

6.1.3 The Color of Surfaces○ Spectral reflectance

25

6.2 Human Color Perception ○ Types of photoreceptors: Rod : sensitive to light Cone: sensitive to color

Types of cone:S (blue) – short wavelength light M (green) – medium wavelength lightL (red) – long wavelength light

26

○ Principle of Uni-Variance -- Receptors respond strongly or weakly, but do not signal the wavelength of the light falling on them The response of the kth type of receptor

27

6.2.1 Color Matching -- is to figure out how a color is composed of primaries Two ways of color matching: Additive matching, Subtractive matching

○ Additive matching

28

○ Subtractive matching For some colors, their may be negative. Subtractive matching adds some amount of some primaries to the test light.

i s

○ Principle of Trichromacy (1) The primaries must be independent (2) Both additive and subtractive matching are allowed

6.3 Representing ColorUnit radiance source:

1 1 2 2 3 3( ) ( ) ( ) ( )U f P f P f P

1 2 3, , P P P : primaries

: color matching function

29

Single wavelength source:

Source:

( ) ( )S U

1 1 2 2 3 3

1 1 2 2

3 3 1 1 2 2 3 3

( ) ( )

{ ( ) ( ) ( ) } ( )

{ ( ) ( ) } +{ ( ) ( ) }

{ ( ) ( ) }

S U S d

f P f P f P S d

f S d P f S d P

f S d P w P w P w P

30

○ Grassman’s Laws -- matching is linear

31

。 RGB Color Space R,G,B are real primaries Color matching functions may be negative

。 CIE XYZ Color Space CIE: Commission International D’eclairage X,Y,Z are not real primaries Color matching functions are positive everywhere

○ Color Matching Function 1 2 3( ), ( ), ( )f f f

32

1

0

( ) ( ) ( ),N

x i i ii

X k f l r

1

0

( ) ( ) ( ),N

y i i ii

Y k f l r

Definitions:

1

0

( ) ( ) ( )N

z i i ii

Z k f l r

33

6.3.1 Linear Color Spaces -- A color lies on a straight line connecting two colors. The color can be formed by a linear combination of the two colors

-- A color lies on a planar patch formed by connecting three colors. The color can be formed by a linear combination of the three colors

34

○ RGB Color Space R: 645.16 nm, G: 526.32nm, B: 444.44nm

0.299 0.587 0.114

0.596 0.275 0.321

0.212 0.523 0.311

Y R

I G

Q B

○ YIQ color space

35

○ CIE XYZ Color Space The volume of visible colors in the XYZ space is a cone whose vertex is at the origin

○ YUV color space

0.299 0.587 0.114

0.493( ), 0.877( )

Y R G B

U B Y V R Y

36

。 The relationship between RGB and XYZ

0.431 0.342 0.178

0.222 0.707 0.071

0.02 0.130 0.939

X R

Y G

Z B

3.063 1.393 0.476

0.969 1.876 0.042

0.068 0.229 1.069

R X

G Y

B Z

37

。 CIE xy Space -- The space results from intersecting the XYZ space with plane 1X Y Z

Chromaticity Diagram

38

(i) Spectral locus: the curved boundary along which the colors are experienced (ii) Neutral point: the color whose weights are equal for all three primaries(iii) Colors that lie farther away from the neutral point are more saturated

39

。 A pigment removes the colors other than the pigment color from the incident light, which is then reflected from surface e.g., Red ink removes green and blue lights; red light passes through the ink and is reflected from the paper

○ CMY -- primaries of pigments Cyan = White – Red, Magenta = White – Green, Yellow = White – Blue

40

6.3.2 Nonlinear Color Spaces -- The coordinates of a color in a linear space may not encode properties that are familiar to human

3

R G BI

○ HSI Space: Hue, Saturation, Intensity

RGB HSI

1

2 1/ 2

1[( ) ( )]

2cos { }[( ) ( )( )]

R G R BH

R G R B G B

360H H B Gif

31 min( , , )S R G B

R G B

41

1 cos1 ,

3 cos(60 )

S Hr

H

0 120 :H

1 ( )g r b

HSI RGB

1(1 ),

3b S

120H H 1 cos

1 ,3 cos(60 )

S Hg

H

120 240 :H

1 ( )b r g

1(1 ),

3r S

1 cos1 ,

3 cos(60 )

S Hb

H

240 360 :H

1 ( )r g b

1(1 ),

3g S 240H H

, , R G B

r g bR G B R G B R G B

42

0

0

* 13 *( )

* 13 *( )

u L u u

v L v v

1/3

0 0

0 0

25(100 ) 16 0.008856

* ,903.3 ,0.008856

Y Y

Y YL

Y Y

Y Y

3 30

0 0

0 0

9 * 16 12 3 20, ( ) , Z ( )

4 25 4* *

where , 13 * 13 *

, : reference white

u L u vX Y Y Y Y

v vu v

u u v vL L

u v

○ Lu*v* color space

4 9where ,

15 3 15 3

X Yu v

X Y Z X Y Z

43

44

45

○ Uniform Color Space。 Noticeable difference – the difference when modifying a color until one can tell it has changed. The noticeable difference of a color forms the boundary of the color and can be fitted with an ellipse (macadam ellipse)

。 The color difference in the CIE xy space is poor (a) the ellipses at the top are larger than those at the bottom (b) the ellipses rotate as they move up

46

CIE u’v’ Space – a more uniform space than

CIE xy space 4 9

( , ) ( , )15 3 15 3

X Yu v

X Y Z X Y Z

47

1/3 1/3 1/3

0 0 0

1/3 1/3

0 0

* 25[100 ] 16, * 500[( ) ( ) ]

* 200[( ) ( ) ]

Y X YL a

Y X Y

X Zb

X Z

0 0 0, , : reference whiteX Y Z

1/3 3 30 0

1/3 30

* 1 * 16 * 16[ ( ) ( )] , ( )500 100 25 251 * 16 *

[( ) ( ) ]100 25 200

a L LX X Y Y

L bZ Z

○ La*b* color space is a substantial uniform space

48

6.3.3 Spatial and Temporal Effects

○ Chromatic adaptation – the color system adapts

(the color diagram is skewed) when the visual

system has been exposed to an illuminant for

some time

Contrast -- the surrounding colors of a color cause the color to move away from the surrounding colors

Assimilation – the surrounding

colors of a color cause the color to move toward the surrounding colors

49

Image color depends on (a) Camera (b) Physical effects

(i) The color of object surface (ii) The colors of light sources

6.4 Surface Color from Image Color

50

○ Cameras 。 A color camera contains an imaging device that is composed of a set of sensory elements CCD (charge coupled device) 。 Each CCD contains one of three filters, each realizing a spectral sensitivity function (SSF)

。 Gamma correction is a form of compression for compressing the incoming dynamic range e.g.,

。 In terms of SSF, CCDs are arranged in a mosaic with a particular pattern, called the Bayer pattern

1/I , where I: intensity

51

。 The spectrum of the reflected light of a patch

○ Physical effects。 The color of light arriving a camera is determined by (a) the spectral radiance of the light (b) the spectral reflectance of surface

52

○ The response of a photoreceptor of the kth type

to the patch ( ) ( ) ( )k kp E d

53

○ The value at an image pixel

( ) ( ) ( ) ( ) ( ) ( )d sg g C x i x x d x x s x where

( )d x

( )xdg

( )s x

( )xsg

( )i x

: image color of a frontal surface

: change in brightness due to the orientation of the surface

: image color of specularity from a flat frontal surface

: change in specular energy due to the orientation of the surface

: colored light

54

1. Specularities appear as small bright patches 2. Specularities are often sufficiently bright to saturate the camera so that the color can be hard to measure ＊ Looking for small, bright patches is an effective way to find specularities without relying on color.

○ Finding Specularities

55

○ Specularities on electric and dielectric surfaces look different 1. Light striking an electric surface can not penetrate it, which is either absorbed or reflected. Electric surfaces have a specular component that is wavelength dependent of the light2. Light striking a dielectric surface ca

n penetrate it. Dielectric surfaces have

a specular component that is waveleng

th independent of the light

56

○ Example: Dielectric object with single color

( ) ( ) ( )p x x d x sd sg g Pixel value:

( )x ddg - Produces a line that extends to pass through the origin- The points on the line have the same color but different intensity values

57

( )x ssg - Produces a line colliding with a face of the color cube- The points on the line have the same color as the source but different intensity values

58

。 Example: Plastic object on black background

59

(c) Boundary region a plan-like cluster Weighted combinations of two different colors (specular and surface colors)

○ Specularity marking algorithm: Find (i) the dog-leg pattern, (ii) the specular line

(b) Diffuse region a line-like cluster

The object surface has a single color but has different intensities from point to point

A window of pixels in

(a) Background region a point-like cluster of points in the color space

All background pixels have the same color and intensity

60

○ Surfaces reveal different colors when imaging under lights with different colors or intensities

6.5 Inferring Lightness and Color

61

○ Humans can easily achieve

Color constancy – Intensity-independent description of color

Lightness constancy – Color-independent description of intensity

62

( ) ( ) ( )cC k I x x x

○ Model of image intensity 。 Radiance arriving at a pixel depends on (a) The illumination of the light source (b) The BRDF of the surface (c) The configuration of the surface (d) Camera responses 。 Simplifications: (a) Scene surfaces are planar and frontal (b) Surfaces are Lambertian (c) The camera response is linear

63

Take logarithm

Assumptions:(i) No albedo change of an object(ii) Albedo changes occur only when one object occludes another

(iii) Illumination I changes slowly over space

log ( ) log log ( ) log ( )cC k I x x x

64

。 Example: Recovering lightness

Horn approach:(1) Differentiate the log transform(2) Throw away small gradients(3) Integrate the result

65

Rephrase as an optimization problemFind whose gradient is m

ost like the thresholded , i.e., find

that minimizes

log log /d d xlog log /d C dx

2log log

thresholded d d C

d d

x x

66

The response of receptor of the kth type

1 1

, ,

1, 1 1, 1

( ) ( ) ( ) ( )[ ( )][ ( )]

( ( ) ( ) ( ) )

n m

k k k j j i ij i

m n m n

i j k j i i j ijki j i j

p E d r e d

e r d e r g

Albedo: Irradiance:

1

( ) ( )n

j jj

r

1

( ) ( )m

i ii

E e

where ( ) ( ) ( )ijk k j ig d can be learned

○ Finite-Dimensional Linear Models -- Models (i) surface albedo and (ii) illuminant irradiance as a weighted sum of basis functions

67

○ Assume the average of albedo is constant and known

1

n

j jj

r

The average of the response of the kth receptor is

,

1, 1

m n

j ijk jki j

p e g r

In vector-matrix form, Ap e where

1

[ ]n

j ijkj

A r g

Solving for illumination e, the surface reflectance at each pixel can then be known.

68

◎ Gamut mapping The gamut of an image: the set of all pixel values

Let G: the convex hull of the gamut of the given imageW: the convex hull of the gamut of an image of many different colors under white light

eM : mapping an image seen under illuminant e to an image seen under white light

69

70

( )eM G W The only illuminants to be considered are those

Once the family of potential illuminants has been found, it remains to determine an appropriate illuminantThe strategies of determination depend on applications

s. t.e