Image Processing - UCL · Image segmentation Image Processing. 87. Image Processing. 94 Unassessed Assignment Use matlab to change the geometric and radiometric quantization resolution

Image Processing.

1

Image Processing

Gabriel Brostow

TA: Josias P. Elisee

(with help from Dr Wole Oyekoya)

COMP 3072 / GV12

Image Processing.

2

Image Processing.

3

Motivation and Goals

• Grounding in image processing techniques

• Concentrate on algorithms used in machine

vision, graphics, medical imaging

• Best sensors ever!

Image Processing.

4

Motivation and Goals

• Grounding in image processing techniques

• Concentrate on algorithms used in machine

vision, graphics, medical imaging

• Best sensors ever! With a few problems…

Transmission interference

5

Compression artefacts

6

Spilling

7

Scratches, Sensor noise

8

Bad contrast

9

Resolution Super resolution?

10

Super resolution

11

Removing motion blur

Original image

Cropped subwindow

After motion blur removal

[Images from Amit Agrawal]12

Removing motion blur

13

14

Removing motion blur

15

Image Processing.

16

Syllabus (1st half)

1. The digital image

2. Image segmentation*

3. Image transformations

4. Morphological operations*

5. Image Filtering

*= Homework will be assigned

GV12/3072

Image Processing.

Features and Object Recognition

GV12/3072

Image Processing.

Syllabus (2nd half)

6. Filtering Applications + Edges*

7. Corner detection

8. Feature Characterization

9. Color images*

10.Template matching

*= Homework will be assigned

Image Processing.

20

Why Now?

• Why are these sectors paying more attention?

• Medicine• Automatic or assisted diagnosis

• Image-guided surgery

• Biology / Agriculture

• Film and television

• Surveillance and police work

• Military

• http://www.cs.ubc.ca/spider/lowe/vision.html

Image Processing.

21

Course content

• Lots of material!

• Some mathematics• Calculus (light)

• Geometry and matrix algebra

• Probability and statistics (light)

• Some programming – Matlab

22

Lectures and notes• http://www.cs.ucl.ac.uk/staff/g.brostow/classes/IP2010/

• TA: Josias Elisee: J.Elisee at cs.ucl.ac.uk

• Extra help: Wole Oyekoya: w.oyekoya at cs.ucl.ac.uk

• Mondays 16:00-17:00 in Medawar Lankester LT

• Wednesdays 09:00-11:00 in MPEB 1.02

• Lab sessions

– Monday 11-13 (Malet Place Eng 4.06)

– Monday 17-18 (Malet Place Eng 4.06)

• Moodle! http://moodle.ucl.ac.uk/

Image Processing.

25

Assessment

• Exam 80%

• Four Courseworks 20%.• Implement and test algorithms in Matlab

• Honor System

Image Processing.

28

Unassessed CW Assignment

• Matlab introduction

• Start matlab:

% matlab or

% /opt/matlab7/bin/matlab

• Download any simple image

• Load it into matlab:

>> I = imread(‘foo.jpg’);

29

Unassessed CW Assignment

• Display the image in Matlab:

>> imshow(I);

• Print the image data array:

>> I

(Ha! It’s a trap! – use Ctrl-C to make it stop)

• Print the size of the image array and create a subimage:

>> size( I )

>> Isubwindow = I(72:92, 62:82);

>> imshow(Isubwindow);

Image Processing.

30

Unassessed CW Assignment

• Start the Matlab help tool (Help menu).

• In the “Contents” pane to the left of the

window. Click on MATLAB.

• Go through the “Getting Started” section.

• Continue to the “Using MATLAB” section

when you have time.

IP is Only Part of the Picture

• See Machine Vision

and/or Computational

Photography + Capture

(GV15 / M085 / 3085)

• Why?

– To work on fun projects!

31

MRI of GJB A Computational Investigation into

the Human Representation and

Processing of Visual Information

• 3D Gesture Interfaces

(Xbox 360)

• Build Your Own 3D Scanner: Optical Triangulation for

Beginners (Lanman + Taubin)

Image Processing.

32

Image Processing.

33

Developing Drosophila eye (30 hours)With Franck PichaudEpithelial Morphogenesis & Cell PolarityLMCB, Cell Biology Unit, MRC, UCLNeeded Innovations:

- Locate & track branching structures

- Propagate confidence to neighbors

Stop Motion Animation

Image Processing.

34

Super Lego Mario (Level 1)

http://www.youtube.com/watch?v=hmWWfnKVbYY

Video Annotation, Navigation, and Composition

UIST2008, http://www.danbgoldman.com/

Next Time: The Digital Image

Image Processing.

35

Image Processing.

37

The Digital Image

Image Processing.

38

Outline

• What is an image?

• What is a pixel?

• How do we store them?

Image Processing.

39

What is an image?

Image as 2D signal

• Signal: function depending on some variable with physical meaning

• Image: continuous function2 variables: xy - coordinates3 variables: xy + time (video)

• Brightness is usually the value of the function

• But can be other physical values too: temperature, pressure, depth …

40

Example 2d images

ultrasound temperature

camera image CT41

Image Processing.

46

Image?

>> t=rand(256,256);

>> imshow(t)

Image Processing.

49

Where do images come from?

• Digital cameras

• MRI scanners

• Computer graphics packages

• Body scanners

• Laser range finders

• Many more…

Image Processing.

50

Where do images come from?

• Digital cameras

• MRI scanners

• Computer graphics packages

• Body scanners

• Laser range finders

• Many more…

Image Processing.

51

The digital camera

• A Charge Coupled Device (CCD).

Lens

Sensor

array

ADC

Image array

Image Processing.

52From: Lecture Notes – EAAE

and/or Science “Nuggets” 2000

Capturing photons

Image Processing.

53

http://www.astro.virginia.edu/class/oconnell/astr121/im/CCD-fullframearc-FSU.jpg

Image Processing.

55

The sensor array

• Can be < 1cm2.

• An array of photosites.

• Each photosite is a bucket of electrical charge.

• They contain charge proportional to the incident light intensity during exposure.

Image Processing.

56

Analog to Digital Conversion

• The ADC measures the

charge and digitizes the

result.

• Conversion happens line

by line.

• The charges in each

photosite move down

through the sensor array. ADC

RAM

Image Processing.

57

ADC

RAM

Image Processing.

58

Blooming

• The buckets have finite capacity

• Photosite saturation causes blooming

Image Processing.

59

Dark Current

• CCDs produce thermally-generated charge.

• They give non-zero output even in darkness.

• Partly, this is the dark current.

• Fluctuates randomly.

• How can we reduce dark current? From: Lecture Notes - EAAE

Yohkoh satellite, 9 years apart ..

Image Processing.

60

Dark Current

• CCDs produce thermally-generated charge.

• They give non-zero output even in darkness.

• Partly, this is the dark current.

• Fluctuates randomly.

• How can we reduce dark current? From: Lecture Notes - EAAE

Image Processing.

61

What is a pix-el?

x

y

(x,y)

f(x,y)

(0,0)

63

Not a little square!

• A Pixel Is Not A Little

Square, A Pixel Is Not A

Little Square, A Pixel Is

Not A Little Square! (And

a Voxel is Not a Little

Cube),

– Alvy Ray Smith,

MS Tech Memo 6, Jul 17, 1995

Image Processing.

64

Not a little square!

Illustrations: Smith, MS Tech Memo 6, Jul 17, 1995

Image Processing.

65

Gaussian reconstruction filter

Not a little square!

Illustrations: Smith, MS Tech Memo 6, Jul 17, 1995

Image Processing.

66

Cubic reconstruction filter

Not a little square!

Graphics: Dick Lyon, 2006

Image Processing.

67

Sampling 1D

Sampling in 1D takes a function, and returns a vector whose elements are

values of that function at the sample points.

69

Sampling 2D

Sampling in 2D takes a function and

returns an array; we allow the

array to be infinite dimensional and to

have negative as well as positive indices.

70

Image Processing.

71

Greyscale digital image

Nyquist Frequency

• Half the sampling frequency of a discrete

signal processing system

Image Processing.

72

Sampling grids

73

Retina-like sensors

74

Quantization

• Real valued function will get digital values –

integer values

• Quantization is lossy!!

– After quantization, the original signal cannot be

reconstructed anymore

• This is in contrast to sampling, as a sampled but

not quantized signal can be reconstructed.

• Simple quantization uses equally spaced levels

with k intervals

bk 275

Quantization

00

01

10

11

76

Quantization

00

01

10

11

77

Usual quantization intervals

• Grayvalue image

8 bit = 2^8 = 256 grayvalues

• Color image RGB (3 channels)

8 bit/channel = 2^24 = 16.7Mio colors

• 12bit or 16bit from some sensors

• Nonlinear, for example log-scale

78

Image Processing.

79Photo: Paulo Barcellos Jr.

Winding Down

Image Processing.

80

Properties

• Image resolution

• Geometric resolution: How many pixel per area

• Radiometric resolution: How many bits per pixel

81

Image resolution

1024x1024

512x512

512x1024

82

Image Processing.

83

Geometric resolution

144x144 72x72 36x36

18x18 9x9 4x4

Image Processing.

84

Radiometric resolution

256 128 64 32

16 8 4 2

Lossless vs. Lossy

• Name some formats?

Image Processing.

85

Image Processing.

86

Aliasing and SNR

• What is the disadvantage of low sampling

resolution?

• What is the disadvantage of high sampling

resolution?

Finish

Next week:

Image segmentation

Image Processing.

87

Image Processing.

94

Unassessed Assignment

Use matlab to change the geometric and

radiometric quantization resolution in one

of your images. For each level of sampling

and quantization, plot the image function, as

in slides 67 & 68, and compare the

approximations to the true intensity function

that you get at each level.