Part 1: Course Introduction130.243.105.49/Research/Learning/courses/dip/2011/... · human visual perception, optical illusions, e-m spectrum example application – person tracking

2011-04-05

Digital Image Processing

Achim J. Lilienthal

AASS Learning Systems Lab, Dep. Teknik

Room T1209 (Fr, 11-12 o'clock)

[email protected]

Course Book Chapters 1 & 2

Part 1: Course Introduction


1. Introduction digital images

human visual perception, optical illusions, e-m spectrum

example application – person tracking with mobile robots

example image understanding – tiny images approach

2. Course Contents

3. Digital Image Acquisition image formation model

image sampling and quantization, zooming and shrinking

Contents


IntroductionDigital Images

→ Contents


digital image of a rat

Introduction – Digital Images1

Digital Images a finite set of digital values (picture elements = pixels)

each pixel is associated to a position in a 2D region

each pixel has a value

magnification of the rat’s nose



Digital Images can be thought of as a matrix (raster image / raster map)

of grey levels / intensity values

94 100 104 119 125 136 143 153 157 158

103 104 106 98 103 119 141 155 159 160

109 136 136 123 95 78 117 149 155 160

110 130 144 149 129 78 97 151 161 158

109 137 178 167 119 78 101 185 188 161

100 143 167 134 87 85 134 216 209 172

104 123 166 161 155 160 205 229 218 181

125 131 172 179 180 208 238 237 228 200

131 148 172 175 188 228 239 238 228 206

161 169 162 163 193 228 230 237 220 199

magnification of the rat’s nose



Digital Images types ← dimensionality and nature of pixel values

binary (bilevel)

grey scale

color

false-color

multi-spectral

semantic (thematic), ...

3D Digital Images picture elements are called voxels

(from "volumetric" and "pixel") → not addressed here


Introduction – Electromagnetic Spectrum1

The Electromagnetic Spectrum we perceive only a small range of colours of the

electromagnetic spectrum (~ 430nm – 790nm) gamma rays, X rays, ultraviolet light, visible spectrum,

infrared, microwaves, radio waves, ...



The Electromagnetic Spectrum fundamental equations

relation between wavelength (λ) and frequency (ν):

relation between energy and frequency:νλ c=

νhE =



The Electromagnetic Spectrum we perceive only a small range of colours of the

electromagnetic spectrum (~ 430nm – 790nm)

objects are perceived by the light they reflect achromatic light: all wavelengths are reflected equally

chromatic light: some wavelengths are reflected predominantly


IntroductionBiological Vision

→ Contents


Introduction – Visual Perception1

Metaphysics All men by nature desire to know.

An indication of this is the delight we take in our senses; for even apart from their usefulness they are loved for themselves; and above all others the sense of sight.

Aristotle (384 BC – 322 BC)



The Human Eye



What happens? photons are reflected at objects

pattern of reflected photons is sensed … biological vision: with photoreceptors ( pixel)

computer vision: with a (digital) camera

… and further processed as a multidimensional signal biological vision: in the visual cortex

computer vision: DIP, computer vision

from Per-Erik Forssén "Visual Object Detection"

Vision



Image Formation – Pinhole Camera Model

from Per-Erik Forssén "Visual Object Recognition"



Image Formation – Pinhole Camera Model focal length between

17 mm (min. refractive power, objects farther than 3m) and

14 mm (max. refractive power)

focal length (min. refractive power)

15 / 100 = h / 17 ⇒ h = 2.55 mm



The Human Eye sphere (diameter ~ 20 mm)



The Human Eye cornea

constant thickness

lens with fixed focal length

responsible for ~ 75% of the refraction




constant thickness



lens can be contracted

zoom (to a plane)

shape of lens is varied to focus on objects at different distances

IR and UV light are absorbed by proteins in the lens structure




constant thickness



lens can be contracted

zoom (to a plane)

2D image on the retina represents the light pattern reflected from a thin plane in the 3D spatial world, the lens is focused on



The Human Eye pupil

opening varies from 2 to 8 mm

regulates the amount of light reaching the retina



The Human Eye pupil



aperture of a camera

source: Wikipedia (http://en.wikipedia.org/wiki/Aperture)



The Human Eye pupil



aperture of a camera

light reaches the retinal surface(spherical, inner wall of the eyeball)

photoreceptors "translate" light into electrical pulses

distributed over the retinal surface

non-uniform resolution



Foveal/Peripheral View












The Human Eye pupil



aperture of the eye

light reaches the retinal surface(spherical, inner wall of the eyeball)

photoreceptors are distributed over the retinal surface cones & rods



The Human Eye two classes of light receptors

distributed over the retinal surface cones (bright-light vision – phototopic)

• 6-7 million around fovea

• colour & bright-light vision

• fine details

• cones with peak sensitivity for long, medium and short wavelengths (red, green, blue)

• only cones in the fovea



The Human Eye two classes of light receptors

distributed over the retinal surface cones (bright-light vision – phototopic)

• 6-7 million around fovea

• colour & bright-light vision

• fine details

• red, green, blue

rods (dim-light vision – scotopic)

• 75-150 million

• coarse details

• "night vision"



Receptor Distribution in the Human Eye no receptors where the optic nerve emerges (blind spot)

radially symmetric distribution around the fovea except from the blind spot

distribution of rods and cones around the fovea

Digital Image ProcessingDigital Image Processing


Why do we sometimes have red eyes in photos?





The Fovea responsible for sharp vision (reading, watching television, ...)

circular indentation (diameter ~ 15 mm)

approx. 330 000 cones in this area (~ a 15 x 15 mm2 square sensor)



The Fovea responsible for sharp vision (reading, watching television, ...)

circular indentation (diameter ~ 15 mm)

approx. 330 000 cones in this area (~ a 15 x 15 mm2 square sensor)

resolution that can be achieved with a CCD chip? 10 MP camera

• 7.2 x 5.3 mm2

(260 000 pixels / mm2)

• 590 000 "pixels" on 1.5 x 1.5 mm2

(260 000 "pixels" / mm2)



Receptor Position in the Human Eye photo-receptors turned away from the lens!


Introduction1

Brightness Adaptation in the Human Eye human eye can adapt over 10 orders of magnitude!

6 orders in phototopic vision (cones)

accomplished by brightness adaptation(changes in the overall sensitivity)

much smaller range for eachbrightness adaptation level Ba

subjective brightness is a log function of the light intensity

brightness discrimination poor at low levels of illumination

better with increasing illumination


Introduction – Sensation vs Perception1

Ganglion Cells ≈ 125 million rods & cones ≈ 1 million ganglion cells

implement local neighbourhood operations(local receptive field)

respond if there is a differencebetween "center and surround"(center-surround cells)

contrast-sensitive vision absolute intensity / color

not available to the brain important for colour constancy



Image Formation in the Human Eye perceived breightness is not a simple function of intensity!

Mach bands• stripes appear darker

near a more intense stripe (and vice versa)

• caused by inhibitory neural connections







simultaneous contrast• a regions' perceived breightness

depends on the intensity in the neighbourhood



perceived breightness is not a simple function of intensity! simultaneous contrast

• a regions perceived breightness depends on the intensity in the neighbourhood


Introduction – Sensation vs Perception1

Sensation operation of basic sensory systems result of physical stimuli and low-level processes

Perception involve higher-level processes in the percipient

memories expectations emotions state of fatigue or alertness

→ "The Great Ideas of Psychology" (TTC)



Biological Vision development responded to evolutionary necessities



Biological Vision bear pixels?



Importance of Context

Torralba et al., CVPR 2007, Short Course



Importance of Context

Torralba et al., CVPR 2007, Short Course







simultaneous contrast• a regions perceived breightness

depends on the intensity in the neighbourhood

optical illusions


Introduction – Optical Illusions1

Optical Illusions the eye / brain fills in nonexisting information

perceives geometrical properties of an object wrongly

characteristic of the human visual system and not yet fully understood ... (some examples follow)



concentrate on the dot in the middle ...... and move your head back and forth



movement created only in the brain



concentrate on the cross in the middle ...... and the moving circle turns green! ... after a while the violet circles disappear!!



1. Relax and stare for 30s - 45s to the four dots in the centre2. Then look slowly to a white wall (large uniformly coloured area) close to you 3. You will see a bright spot forms at the wall4. Now blink a few times5. What do you see? Whom do you see?




IntroductionImage Processing

→ Contents


Introduction – Image Processing1

Image Processing versus Image Analysis

world

data image

image analysis

computer graphics

imaging

“knowledge”

image understanding, computer vision

image processing



Image Processing versus Image Analysis

world

data image

image analysis

computer graphics

image processing

imagingvisualisation

“knowledge”

image understanding, computer vision


Fundamental Steps in Digital Image Processing problem

Lara Croft has to get out of a room




image acquisition




image acquisition

preprocessing




image acquisition

preprocessing

segmentation




image acquisition

preprocessing

segmentation

representation and description model of objects




image acquisition

preprocessing

segmentation


recognition and interpretation what are these objects?




image acquisition

preprocessing

segmentation


recognition and interpretation what are these objects?

solution



Course Contents

→ Contents


Course Contents2

Filtering in the Spatial Domain(Image Enhancement)

"Lena" with noise Median filtering edge detection


Course Contents2

Fourier Transform

original image power spectrum after Fourier transformation

inverse transform of filtered power spectrum


Course Contents2

Image Restoration

?


Course Contents2

Binary Image Operations

original image thresholding closing


Course Contents2

Segmentation

original image segmented (binary) image

?


Course Contents2

Morphological Image Processing & Shape Description

grey image

... after segmentation

... after morphological closing

... after skeletonization


Course Contents2

Colour Representation and Use

RGB space CIE’s chromaticity diagram


Course Contents2

Classification and Introduction to Pattern Recognition

original image result of classification

?


Digital Image Acquisition

→ Contents


Digital Image Acquisition3

Digital Image Representation f(x,y) as a matrix of real numbers

elements of the matrix are called pixels (2D)

)(

)1,1(...)1,1()0,1(

)1,1(...)1,1()0,1()1,0(...)1,0()0,0(

),( ija

NMfMfMf

NfffNfff

yxf =

−−−−

−−

=


Image Formation and Image Sampling3

Image Formation Model illumination i(x,y) from a source reflectivity r(x,y) = reflection / absorption in the scene

f(x,y) = i(x,y) r(x,y) i ~

0.1 lm/m2 (full moon) 1000 lm/m2 (office) 10'000 lm/m2 (cloudy day) 90'000 lm/m2 (sunny day)




f(x,y) = i(x,y) r(x,y) r =

0.01 (black velvet) 0.65 (stainless steel) 0.80 (flat white wall) 0.90 (silver-plated metal) 0.93 (snow)




f(x,y) = i(x,y) r(x,y)

Image Sampling digital image can be seen as a 2D function f(x,y)

x and y are the spatial coordinates

f(x,y) is the grey level / intensity at position (x,y)

a digital image must be sampled (digitized) in space (x,y): image sampling

in amplitude f(x,y): grey-level quantization



Image Sampling and Quantization conversion of continuous input signal to a digital form

continuous signal digitized image




sample f(x,y) inboth coordinates(sampling)

continuous signal





continuous signal





sample f(x,y) inamplitude(quantization)



Image Sampling uniform – same sampling frequency everywhere

adaptive – higher sampling frequency in areas with greater detail (not very common)

determines the spatial resolution



Image Sampling spatial resolution:

smallest discernible detail in the image(line pairs per mm, for example)

5122561286432



Image Quantisation greylevel quantization

2563282

2011-04-05


Achim J. Lilienthal

AASS Learning Systems Lab, Dep. Teknik

Room T1209 (Fr, 11-12 o'clock)

[email protected]

Part 1: Course Introduction

Course Book Chapters 1 & 2

Thank you!

Part 1: Course Introduction130.243.105.49/Research/Learning/courses/dip/2011/... · human visual perception, optical illusions, e-m spectrum example application – person tracking

Documents