IMAGE ANALYSIS AND COMPUTER VISION. Orientation basis Definition and history Image processing Basics and classification Digital image Image processing.

IMAGE ANALYSIS AND COMPUTER VISION

Image processing basic steps

1. Image Enhancement

2. Image Restoration

3. Image Analysis

4. Image compression

5. Image Synthesis

Goal in image analysis• Image analysis operations are used in applications, that

require the measurement and classification of image information

• Examples:• Cell recognition from tissue sample• Object recognition from conveyor belt• Zip code reading from envelope

Group discussion• List application possibilities for image analysis!

Image analysis

• Basis is visual image, whose content should be interpreted

• As a result mostly non-image data• As a goal is to understand images content classifying its content

Example: Robot vision

Example: Robot vehicle

Example: Traffic analysis

Image analysis operations

Image analysis operations• Segmentation

– operation that highlights individual objects within an image.

• Feature Extraction– after segmentation->measure the individual features of each

object

• Object Classification– classify the object to particular category

feature extraction

classification

segmentspace

distinctobjects

featurespace

feature

classificationspace

class

Segmentation

image

Segmentation operations

• Image Preprocessing• Initial Object Discrimination• Image Morphological Operations

feature extraction

classification

segmentspace

distinctobjects

featurespace

feature

classificationspace

class

Segmentation

image

Image Preprocessing

• In preprocessing eg. change images contrast, filter noise and remove distracting image background

• Image enhancement operations is used

Initial Object Discrimination

• Separates image objects into rough groups with like characteristics using image enhancement operations

• Outlining and contrast enhancement often used

Initial object discrimination - example

Original Binary contrast enhanced

Initial object discrimination - example• Sobel edge-enhancement

1 Filter with horizontal mask2 Filter with vertical mask 3 Add

-1 0 1 -1 -2 -1

-2 0 2 0 0 0

-1 0 1 1 2 1

Morphological processing

• In preprocessed image the boundaries are very rough-> need to “clean up”

• Morphological operations

Morphological operations• Binary operations

–Erosion and dilation –Opening and Closing–Outlining–Skeletonization

• Gray-scale operations–Top-Hat and Well transformations–Morphological gradient–Watershed edge detection

Binary morphology• Focus on two brightness values. black=0, white=255• Technically same as spatial convolution• combines pixel brightness with a structuring element,

looking for specific pattern• Array of logical values• (cut=AND, union=OR, complement=NOT)

Binary morphology - equation

O(x,y) = 0 or 1 (predefined) if X =I (x,y) ANDX0 =I (x+1,y) ANDX1 =I (x+1,y-1) ANDX2 =I (x,y-1) ANDX3 =I (x-1,y-1) ANDX4 =I (x-1,y) ANDX5 =I (x-1,y+1) ANDX6 =I (x,y+1) ANDX7 =I (x+1,y+1)otherwise, O(x,y) = opposite state

Erosion• Reduces the size of the objects in relation to their

background

Mask 1 1 1 O(x,y) = 1 if “Hit”

1 1 1

1 1 1 = 0 if “Miss”

1 1 1

Example (1/2)

Original Binary contrast enhanced

Example (2/2)

Eroded imageBinary contrast enhanced

Dilation• Uniformly expands the size of object

Mask 0 0 0 O(x,y) = 0 if “Hit”

0 0 0 = 1 if “Miss”

0 0 0

Example

Binary contrast enhanced

Dilated image

Opening• Erosion then Dilation• Removes one pixel mistakes like erosion• Object size remains

Binary contrast enhanced

Erosion Dilation

Closing• Dilation + erosion• Fills pixel wide holes• Object size remains

Binary contrast enhanced

Dilation Erosion

Cleaning

Original binary contrast enhanced image

Opening Cleaned = Opened and Closed

Outlining• Forms one-pixel-wide outlines and tends to be more

immune to image noise than most edge enhancement operations

• Implementation: • Eroded image subtract from original

Outlining

Original Binary contrast enhanced

Outlining

Eroded image Original - Eroded image

Skeletonizing• Make “wireframe” model from image • Uses different erosion masks• Analogy: fire, which burns object from each side

Gray-scale operations

• Used when binary operations degrade an image• Gray-scale operation can be followed binary operation • Mask terms -255 ... 255 or “Don’t care”

Erosion and Dilation• Erosion reduces the size of objects by darken the bright

areas in image• Dilation is inverse operation

Erosion and Dilation example

Erosion

Dilation

Original

Opening and Closing• Opening = erosion + dilation• Opening reduces noise pixels• Closing = dilation + erosion• Closing fills one-pixel-wide holes

Opening example

Opening - example 2

Morphological gradient

Images outlines as a result Make copy from image. Erosion to other

image and dilation to other. Then images subtract from each other using a dual-image point process.

Original

Eroded

Dilated

Eroded - dilated =Gradient

Image

Morphological gradient

Original Erosion

Dilation Gradient=Erosion-Dilation

Feature Extraction

• Operation followed by segmentation• Choose essential features and measure them from

objects• Goal is to find features, which help find out object’s

class easier

feature extraction

classification

segmentspace

distinctobjects

featurespace

feature

classificationspace

class

Segmentation

image

Features• Brightness and color• Texture• Shape• Spatial moments• Edge shape

Pornographic image analysis

Feature: Brightness and color

Histogram can show

• Color (sorting by colors)• Brightness

• average brightness• Standard deviation brightness• mode brightness• sum of all pixel brightnesses<-> energy (zero-order spatial

moment)

Example: Fruit sorting

Problem:• Boxes goes on conveyor belt, which has green apples

(Granny Smith) and red apples (Red Delicious) and also oranges

• Sort boxes to the correct follow on conveyor belts automatically

Example: Fruit sorting

Solution:• Capture image with camera to RGB images• Convert RGB to HSL• Explore Hue color component which fruit it is

RGB HSL

Feature: Texture• Images spatial frequency shows is the one question

smooth texture (low frequency) or coarse texture (high frequency)

Texture features - meters• High-pass filter• Fourier -transform• Standard deviation of the brightnesses (big deviation =

coarse texture, small deviation = smooth texture)

Feature: Shape• Most common object measurement method• Objects physical measures • The goal is to use fewest necessary measure methods• Often standard shapes, square, circular, elliptical• Even the weirdest shapes can be measured

Shape measures

• Perimeter• Area (pixels)• Area to perimeter ratio

Roundness=(4*Area)/Perimeter²

Value between 0 and 1.

Shape measures• Major axis (x1,y1 and x2,y2)• Major axis length sqr((x2-x1)²+(y2-y1)²)• Major axis angle

Shape measures• Minor axis• Minor axis width• Minor axis width to Major axis length ratio

Value between 0 (elongated) and 1 (circle)

Shape measures• Bounding box area

= Major axis length * Minor axis length

• Number of holes• Total hole area• Total hole area to object area ratio

Value between 0 (no holes) and 1 (entirely a hole)

Feature: Spatial moment• Statistical shape measures• Zero-order spatial moment

Sum of object’s pixel brightness.

Same as object’s area in binary image and

object’s ‘energy’ in gray-scale images

Spatial moment• First-order spatial moment

m

xsum = i * xi i=0

where

i = pixel’s x-coordinate xi = pixel’s value at im= image’s horizontal dimension

y-moment n

Ysum = j * yj j=0

Center of mass (=Centroid)

Center of mass= first-order / zero-order

Binary imagessum of x-pixel coordinates

CoMx= number of pixels in object

sum of y-coordinatesCoMy=

number of pixel in image

Feature: Boundary Descriptions• Explicit description• Chain codes• Line segment representation• Fourier descriptors

Boundary shape, accuracy

Sequential list of the boundary pixels

• save the boundary (x,y)-values• No change of location, orientation or size

Boundary shape, Chain code

• Boundary can be followed and recorded using chain code

• pick a starting pixel and detect direction to the next boundary pixel (values 0..7)

Chain codes• Absolute chain code

• Each code represents absolute direction, so can’t handle geometric change

• Relative chain code• Direction values are relative to earlier values. Can handle

geometric change.

Relative chain code

Line segment representation

• Some cases reduced form of the outline may be sufficient

• Example replacing individual boundary (x,y) locations with line segments.

Line segment representation -generally form

• Like chain code, but segment lengths and angles can be value free

Variable line segment boundary representation• Combines adjoining line segments with similar direction

angles

Object Classification• Compare the measurements of a new object with those of

a known object or other known criteria • In recognition the objects found in image ( example letters

and numbers) gets labels.

feature extraction

classification

segmentspace

distinctobjects

featurespace

feature

classificationspace

class

Segmentation

image

Knowledge base

• In image analysis you need pre-knowledge from problem area. This is saved in knowledge base. It can contain example different kind of letter and number features, where the wanted objects may have locate

• Knowledge base also controls different module’s operation.

Object classification• Comparing measures with known features

Classification:

• pick feature measures

• pick tolerances

• create classes

Object classification - example• Recognize ring and spade from conveyor belt

Features: Hole or no holeTolerance: Hole size 5mm ± 5%

Classes: Good, if in tolerances

Otherwise reject

Other classification methods• Gray-level classification

• Image information is divided in classes by gray level; text, logo, background ...

• Fuzzy logic• If several features which depend on each other

• Neural networks• Unique ability to be “trained”

Application level• Basic description of application• Example:

1. Capture image from register plate

2. Process and interpret objects

3. Search register database, is there offenses

Object detection system - planning• Object localization• Feature choosing• Classifiers planning• Classifiers teaching• Capacity evaluating

Pattern recognition

Applications

Cell analysis

Chromosome analysis

Thermo analysis

Micro hardness analysis

DNA analysis

Motion analysis

• Counting trees QR Code

108/11

Computer vision examples

109/11

Facial recognition (=there is a face)

Facial recognition principle

Face features