General Research Image models Repetition Image Analysis - Lecture 1 Kalle Åström 30 August 2016 Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition
Image Analysis - Lecture 1
Kalle Åström
30 August 2016
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition
Lecture 1I Administrative thingsI What is image analysis?I Examples of image analysisI Image modelsI Image InterpolationI Digital geometryI Gray-level transformationsI Histogram equalization
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Image analysis Computer vision Perceptual problems
Information
Lectures: 16× 2h, tue 8:15, thu 10:15 and wed 8:15 (weeks 3and 5)Assignments: 4 (compulsory -> grade 3)Question/supervision sessions: Times and rooms will beposted on the homepage -Project: Next study period (optional)Credits: 7.5Pass on course (grade 3): Assignments okPass on course (grades 3, 4 and 5): Assignments ok + Writtenexam (hemtenta) + Oral exam
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Image analysis Computer vision Perceptual problems
The Course
F1 - Introduction, image models, interpolation, transformationsF2 - Linear algebra on images, Fourier transformF3 - Linear filters, convolutionF4 - Scale space theory, edge detectionF5 - Machine learning 1F6 - TextureF7 - Multispectral ImagingF8 - Segmentation: FittingF9 - Machine learning 2F10 - Applications 1F11 - Segmentatoin: Clustering and graph cutsF12 - Applications 2: System building, benchmarking, big data.F13 - Statistical Image AnalysisF14 - Computer VisionF15 - Medical Image Analysis.F16 - extra.
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Image analysis Computer vision Perceptual problems
Image analysis
Image processing: Enhance the image (image -> image)Image analysis: Interpret the image (image -> interpretation)Computer vision: Mimic human vision, geometry, interpretationComputer graphics: Generate images from models
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Image analysis Computer vision Perceptual problems
Computer vision
Computer vision - attempt to mimic human visual functionExamples:
I RecognitionI NavigationI ReconstructionI Scene understanding
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Image analysis Computer vision Perceptual problems
Perceptual problems
Example 1:
What is true ?1. In the figure a = b.2. In the figure a > b.
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Image analysis Computer vision Perceptual problems
Perceptual problems (ctd.)
Exemple 2:
1. This is an image of a vase2. This is an image of two faces.
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Mathematical Imaging Group Related courses Research areas
Mathematical Imaging Group,Centre for mathematical sciences
I Research projects: EU, VR, SSF, IndustryI Masters thesis projectsI SSBAI Industry research: NDC, Decuma, Ludesi, Gasoptics,
Exini, Cellavision, Precise Biometrics, Anoto, Wespot,Cognimatics, Polar Rose, Nocturnal Vision
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Mathematical Imaging Group Related courses Research areas
Related courses
I Computer Graphics 7.5 hp (Study period 1)I Language Technology 9hp (Study period 1)I Machine Learning 7.5 hp (Study period 2)I Medical Image Analysis 7.5 hp (Study period 2)I Multispectral imaging 7.5hp (Study period 2)I Spatial Statistics with Image Analysis 7.5 hp (Study period
2)I High Performance Computer Graphics 7.5 hp (Study
period 2)I Computer vision 7.5 hp (Study period 3)
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Mathematical Imaging Group Related courses Research areas
Research areas
I Geometry and computer visionI Medical image analysisI Cognitive vision
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Continuous model
An image can be seen as a function
f : Ω 7→ R+ ,
where Ω = (x , y) | a ≤ x ≤ b, c ≤ y ≤ d ⊆ R2 andR+ = x ∈ R | x ≥ 0. f (x , y) = intensity at point (x , y) =gray-level(f does not have to be continuous)0 ≤ Lmin ≤ f ≤ Lmax ≤ ∞[Lmin,Lmax ] = gray-scale
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Continuous model (ctd.)
Change to gray-scale [0,L] where 0=’black’ and L=’white’.
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Discrete model
Discretise x , y , called sampling.Discretise f , called quantification.Sampling:Point grid in xy -plane.
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Sampling
f (x , y) 7→
f0,0 . . . f0,N−1... fj,k
...fM−1,0 . . . fM−1,N−1
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Quantification
Use G gray-levelsUsually G = 2m for some m.NMm bits are required for storing an imageEx: 512 · 512 · 8 ∼ 262kB(256 gray-levels)M, N decreased⇒ Chess-patternm decreased⇒ False contours
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Sampling
Given an image with continuous representation it isstraightforward to convert it into a discrete one by sampling.
Common model for image formation is smoothing followed bysampling
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Interpolation
Given an image with discrete representation one can obtain acontinuous version by interpolation.
Problem: (Interpolation)Given f (i , j), i , j ∈ Z2.”compute” f (x , y), x , y ∈ R2
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Re-sampling
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Re-sampling (ctd.)
Problem: (Re-sampling)Given f (i , j), i , j ∈ Z2.”Compute” f (x , y), x , y ∈ R2
Discrete image -> Interpolation -> continuous image ->sampling -> New discrete image in different resolution
Used frequently on computers when displaying an image in adifferent size, thus needing a different resolution.
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Nearest neighbour (pixel replication)
f (x , y) = f (i , j),
where (i , j) is the grid point closest to (x , y).
Called pixel replication.
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Nearest neighbour
Pixel replication can be seen as interpolation with
f (x , y) =∑i,j
h(x − i , y − j)f (i , j),
where
Notice the similaraty to convolution.
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Linear interpolation
In one dimension
f (x) = (x − i)f (i + 1) + (i + 1− x)f (i), i < x < i + 1
Called linear inperpolation.
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Linear interpolation (ctd.)
Linear interpolation can be expressed as
f (x , y) =∑i,j
h(x − i , y − j)f (i , j),
with a different interpolation function h:
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Two dimensions
f (x , y) =(i + 1− x)(j + 1− y)f (i , j)+
+ (x − i)(j + 1− y)f (i + 1, j)+
+ (i + 1− x)(y − j)f (i , j + 1)+
+ (x − i)(y − j)f (i + 1, j + 1),
i < x < i + 1, j < y < j + 1
Called bilinear interpolation. Between grid points the intensityis
f (x , y) = ax + by + cxy + d ,
where a,b, c,d is determined by the gray-levels in the cornerpoints.
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Bilinear interpolation
For two-dimensional signals (images) we can apply linearinterpolation, first in x-direction and then y -direction.
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Cubic interpolation (Cubic spline)
Define a function k such that
k(x) =
a3x3 + a2x2 + a1x + a0 x ∈ [0,1]
b3x3 + b2x2 + b1x + b0 x ∈ [1,2]
0 x ∈ [2,∞)
andI k symmetric around the originI k(0) = 1, k(1) = k(2) = 0I k and k ′ continuous at x = 1I k ′(0) = k ′(2) = 0
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Cubic spline function
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Determination of ai and bi
These conditions give
k(x) =
(a + 2)x3 − (a + 3)x2 + 1 x ∈ [0,1]
ax3 − 5ax2 + 8ax − 4a x ∈ [1,2]
where a is a free parameter.Common choice is a = −1.Interpolation is expressed as
f (x) =∑
i
f (i)k(x − i)
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Cubic interpolation for images
For images one first interpolates in x-direction and then iny -direction.
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Sinc interpolation
Assume that f (x) is a band-limited signal.Sampling theorem:
f (x) =∑
k
sinc(2π(x − k))f (k)
Sketch of proof: Fouriertransform F (ω) is band limited. Thus itcan be written as a fourier series, where the coefficients aref (k). Inverse fouriertransform completes the proof.Drawback: sinc has unlimited support⇒ large filter⇒ timeconsuming.Solution: Cut sinc after the first or the first few oscilations⇒almost like cubic interpolation.
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Sinc interpolation for images
For images one interpolates first in x-direction and then iny -direction.
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Gauss interpolation
Interpolate with
f (x) =∑
k
e−(x−k)2/a2f (k)
where a determines ’scale/resolution/blurriness’.
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Scale selection
Gives a scale-space pyramid with the same image at differentscales by changing a. More about this later.
This is called Gaussian pyramid or scale space pyramid orscale space representation.
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Digital Geometry
Let Z be the set of integers 0,±1,±2, . . . .
Grid: Z2,
· · · ·· · · ·· · · ·· · · ·
Grid point: (x , y)
Definition4-neigbourhood to (x , y):
N4(x , y) =
· × ·× (x , y) ×· × ·
.
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Neighbours, connectedness, paths
Definitionp and q are 4-neighbours if p ∈ N4(q).
DefinitionA 4-path from p to q is a sequence
p = r0, r1, r2, . . . , rn = q ,
such that ri and ri+1 are 4-neighbours.
DefinitionLet S ⊆ Z2. S is 4-connected if for every p,q ∈ S there is a4-path in S from p to q.There are efficient algorithms for dividing sets M ⊆ Z2 inconnected components. (For example, see MATLAB’s bwlabel).
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
D- and 8-neighbourhoods
Similar definitions with other neighbourhood structures
DefinitionD-neighbourhood to (x , y):
ND(x , y) =
× · ×· (x , y) ·× · ×
.
Definition8-neighbourhood to (x , y):
N8(x , y) = N4(x , y) ∪ ND(x , y) =
× × ×× (x , y) ×× × ×
.
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Gray-level transformation
A simple method for image enhancement
DefinitionLet f (x , y) be the intensity function of an image. A gray-leveltransformation, T , is a function (of one variable)
g(x , y) = T (f (x , y))
s = T (r) ,
that changes from gray-level f to gray-level g. T usually fulfilsI T (r) increasing in Lmin ≤ r ≤ Lmax ,I 0 ≤ T (r) ≤ L.
In many examples we assume that Lmin = 0 och Lmax = L = 1.The requirements on T being increasing can be relaxed, e.g.with inversion.
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Thresholding
Let
T (r) =
0 r ≤ m1 r > m,
for some 0 < m < 1.
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Thresholding (ctd.)
i.e.f (x , y) ≤ m⇒ g(x , y) = 0 (black),
f (x , y) > m⇒ g(x , y) = 1 (white).
The result is an image with only two gray-levels, 0 and 1. This iscalled a binary image.
The operation is called thresholding.
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Continuous images
I Let s = T (r) be a gray-scale transformation (r = T−1(s))I Let pr (r) be the frequency function for the original image.I Let ps(s) be the frequency function for the resulting image.
It follows that ∫ s
0ps(t)dt =
∫ r
0pr (t)dt .
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Continuous images (ctd.)
Differentiate with respect to s
ps(s) = pr (r)drds
(s = T (r)) .
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Histogram equalization
Take T so that ps(s) = 1 (constant).∫ r
0pr (t)dt =
∫ s
01dt = s ⇒ s = T (r) =
∫ r
0pr (t)dt
ordsdr
= pr (r)
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Histogram equalization (ctd.)
This transformation is called histogram equalization.
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Histogram equalization for digital images
pr (rk ) =nk
n,
whereI n=number of pixelsI nk=number of pixels with intensity rk
i.e. a histogram.Histogram equalization is obtained by
sk = T (rk ) =k∑
j=0
nj
n.
Note that sk does not have to be an allowed gray-scale⇒perfect equalization cannot be obtained.
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Example OCR (Optical Character Recognition)
I Image of textI Image enhancement, filtering.I Segmentation
I ThresholdingI Connected components with digital metrics.
I Classification
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Images show how a system for OCR (Optical CharacterRecognition) can be used in a mobile telephone.The binary image is interpreted into ascii characters.
Original image and rectified image.
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Cut-out of OCR number after thresholding.
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition Continuous model Discrete model Digital Geometry Gray-level transformation
Masters thesis suggestion of the day: The automaticbook database
Create a system for taking inventory of your books by takingimages of them and analysing the images.Images - segmentation - OCR - Database - Search - Missing
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition
Repetition - Lecture 1
I What is image analysis?I Image models (continuous - discrete - sampling -
quantification, sampling and interpolation)I Digital geometry (4-, D-, 8- neighbours, paths, connected
components)I Gray-level transformations (thresholding, histogram
equalization)
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition
Recommended reading
I Forsyth & Ponce: 1. Cameras.I Szeliski: 1. Introduction and 3.1 Point operators.
Kalle Åström Image Analysis - Lecture 1
General Research Image models Repetition
i.e. id est that is det vill sägae.g. exempli gratia for example till exempelcf. confer compare with (see) jämför, se
Kalle Åström Image Analysis - Lecture 1