Lecture IP1 2012

Introduction to Image Processing

Lecture_IP 1Gray Scale/Color Images, MATLAB

Image Processing Basics

0 0

image ( , ), are spatial coordinates

is image amplitude (intensity, gray level, color level)digital image

, , finite and discrete( , ) called a picture element (pel) or pixel

f x yx yf

x y ff x y

−−

−−

i

i


256 x 256

pixels

64 x 64 pixels

All images interpolated to 512 x 512

format

512 x 512

pixels

128 x 128

pixels

512 x 512 image

64 x 64 image

interpolated to 512 x 512

format

Varying Intensity Levels

Varying Intensity Levels


• Why do we care about images, image processing?– “A picture is worth 1000 words” rich information source from visual data


• Image types:– natural photographic images (jpg)– artistic and hand done drawings (pdf)– scientific images (satellite photos, medical images such as X‐rays and MRIs, cat scans, UV scans) (bmp, tiff)

– engineering drawings, technical specifications, etc. (scanned and converted to desired format, e.g., png – portable network graphics)

Why Process Images

• Enhancement and Restoration (PhotoShop)– removal of artifacts and scratches from old photos– improve contrast and lighting of existing photos– correct blurring (or redeye) due to motion (or camera flash) in photos

• Transmission and Storage (Digital Camera)– compression for reduced storage and transmission costs (e.g.,images

transmitted from outer space)– encoding for efficient transmission

• Image Understanding (Image Search)– analysis and classification of image components– face recognition and classification– object identification (e.g., people, beach, sunset)

• Security and Rights Protection (Legal)– encryption and watermarking

Why Digital Image Processing• Lossless representations of images

– no degradation and perfect reproduction possible– ability to duplicate without any loss of quality

• Sophisticated and powerful digital signal processing algorithms

‐‐ Adobe Photoshop; Photoshop Elements‐‐ easy to process and manipulate images

• Low cost of storage and transmission‐‐modern compression methods can reduce storage rate by factors of 30 or more with almost no loss of perceived quality‐‐ single CD can store thousands of photos

CD Storage for Images• Storage available on CD

– 1.4 Mbps x 60 sec x 80 minutes = 6.72 Gb = 0.84 GB = 840 MB

• Assume 1 mega‐pixel camera giving 24‐bit color images– each picture uses 1 mega‐pixel x 24 bits/pixel = 24 Mb = 3 MB

– without compress can store 840 MB / 3 MB = 280 pictures

– with jpeg compression of 30‐to‐1 can store 840 MB / 0.1 MB = 8400 pictures

Processed Images

Famous processed/spoofed image from World Trade Center 9/11/2001

Images From Outer Space

Images From Outer Space

Applications of Digital Image Processing

• Compression– standards‐based, JBIG, JPEG, JPEG2000

• Manipulation and Restoration– noise removal– noise reduction– morphing pairs of images– restoration of blurred and damaged images

• Other– visual mosaicing and virtual views– face detection– visible and invisible watermarking– error concealment and resilience in transmission

Image/Video Compression• Color image with 600x800 pixels

– without compression each pixel needs 8 bits for each of red, green and blue components

– total bit rate of 600*800*24 bits/pixel =11.52 megabits = 1.44 megabytes

– after JPEG compression can achieve almost lossless (perceptually) compression using 89 kilobytes giving a compression ratio of about 16:1

• Movie with 720x480 pixels per frame, 30 frames/sec, 24 bits/pixel (color frames)– without compression the total bit rate is 720x480x30x24=243

megbits/second or 30 megabytes per second– with MPEG compression (for DVD) can compress video to about

5 megabits/second or a compression ratio of about 48:1

Image Quantization

Original Image (8-bit gray scale)



Original Image (4-bit color scale, pseudo-color)

Image Compression

8 bit 0.5 bit

0.33 bit 0.25 bit

Image Compression

24 bit 1 bit

0.5 bit 0.25 bit

Image Compression

Examples of Image Processing

Enhancement, Denoising, Deblurring, Sampling Rate

Conversion

Image Enhancement (enhance_image.m)% read and display dark image

I=imread('pout.tif');imshow(I);

% enhance contrast and display by computing image histogram and equalizing histogram levels

figure,imhist(I);I2=histeq(I);

% display enhanced image% display enhanced image histogram

figure,imshow(I2,'border','tight');figure,imhist(I2);

Contrast Adjusted Image

Original image with poor contrast Image histogram adjusted image

Image Denoising‐Median Filtering

% Read in the image and display it.I = imread('eight.tif');imshow(I,'border','tight');

% Add noise to the image.J = imnoise(I,'salt & pepper',0.02);figure, imshow(J,'border','tight')

% Filter the noisy image with an averaging filter and display the results.

K=filter2(fspecial('average',3),J)/255;figure, imshow(K,'border','tight')

% Now use a median filter to filter the noisy image and display the % results. Notice that medfilt2 does a better job of removing noise, % with less blurring of edges.

L = medfilt2(J,[3 3]);figure, imshow(L,'border','tight')

Noise Corruption of Image

Original image, noise corrupted during digitization process

Cleaned up image using appropriate size median filter

camerman.tif image ‐‐ filtered

camerman.tif image + noise and filtered

camerman.tif image + noise and median filtered

Image Denoising‐Adaptive Filtering

(1) Original Saturn Image – Color Image

(2) Gray Scale Saturn Image

(1) (2) (3)

(4)

(3) Saturn Image with Additive Gaussian Noise

(4) Noise Removed Image Using Wiener Filter

Image Denoising

Image Lighting CorrectionTop Left: Image captured under poor lighting conditions (background brightness varying across image)Top Right: Foreground objects removed by applying a 25 x 25 grayscale dilation operationBottom Left: Original image subtracted from background imageBottom Right: Color inversion and improved contrast by adjuistingimage histogram

Frequency Domain Filtering

Top Left: Image with a corrugated effect due to fault in the acquisition processTop Right: Two-dimensional FFT of image

Bottom Left: Filtering in the frequency domain (removal of bright spots)Bottom Right: Cleaned up image without the corrugated background

Gaussian Filtering of Images

Original Lena Image Lena Image Filtered by 2D Gaussian Filter with Sigma=3

Original Lena Image Lena Image Filtered by 2D Gaussian Filter with Sigma=1

Gaussian Filtering of Images

Gaussian Filtering of Noisy Images

Lena Image with Additive Gaussian Noise (0.3*randn(512))

Noisy Lena Filtered with 2D-Gaussian Filter with Sigma=3

Gaussian Filtering of Noisy Images

Lena Image with Additive Gaussian Noise (0.3*randn(512))

Noisy Lena Filtered with 2D-Gaussian Filter with Sigma=1

Image Measurement Extraction

Top Left: Original image with objects (want to estimate distribution of the sizes of the objects)Top Right: Threshold image to separate objects from background (binary)Second Row Left: Watershed separation of image to improve object separationSecond Row Right: Logical AND of two images on left-side framesHistogram: Measure object properties (object areas) and display resulting object area histogram

Image Deblurring

Image Deblurring – Lucy‐Richardson Filter

% Read an image into the MATLAB workspace.; crop image

I = imread('board.tif');I = I(50+[1:256],2+[1:256],:);figure;imshow(I,'border','tight');title('Original Image');

% Create the PSF.PSF = fspecial('gaussian',5,5);

% Create a simulated blur in the image and add noise.Blurred =imfilter(I,PSF,'symmetric','conv');V = .002;BlurredNoisy = imnoise(Blurred,'gaussian',0,V);figure;imshow(BlurredNoisy,'border','tight');title('Blurred and Noisy Image');

% Use deconvlucy to restore the blurred and noisy image

luc1 = deconvlucy(BlurredNoisy,PSF,5);figure; imshow(luc1,'border','tight');

Image Sampling Rate Changes

(a) Original Image

(b) Original Image Down-Sampled by 2:1

(d) Original Image Up-Sampled by 2:1

(c) Original Image Down-Sampled by 2:1, then Up-Sampled by 2:1

(e) Original Image Up-Sampled by 2:1, then Down-Sampled by 2:1

Other Image Processing Effects

Visual Mosaic, Watermarking, Error Concealment, Resampling

Visual Mosaic

Watermarks• A visible watermark is a visible translucent image which is

overlaid on the primary image. • A watermark allows the primary image to be viewed, but still

marks it clearly as the property of the owning organization. • It is important to overlay the watermark in a way which makes

it difficult to remove, if the goal of indicating property rights is to be achieved.

Watermarks• An invisible watermark is an overlaid image which cannot be seen,

but which can be detected algorithmically. • Different applications of this technology call for two very different

types of invisible watermarks: – A watermark which is destroyed when the image is manipulated

digitally in any way may be useful in proving authenticity of an image. If the watermark is still intact, then the image has not been "doctored." If the watermark has been destroyed, then the image has been tampered with. Such a technology might be important, for example, in admitting digital images as evidence in court.

– An invisible watermark which is very resistant to destruction under any image manipulation might be useful in verifying ownership of an image suspected of misappropriation. Digital detection of the watermark would indicate the source of the image.

Visible Digital Watermarks

Courtesy IBM Watson web page “Vatican Digital Library

Invisible Watermark

Upper right half of image invisibly watermarked, lower left half not

watermarked.

Upper pattern is watermark detected from upper right half of image; lower pattern is watermark detected from

lower left half of image.

Error Concealment

Error Concealment

Under Sampling of Image

Proper sampling of image Under sampling of image – note the Moire patterns due to image aliasing

Image Processing Using MATLAB

Related Fields to Image Processing• Image analysis

– determine image properties (type of image: photo, drawing, medical image)

– determine image components (objects in image, scene description)

• Computer vision– analyze image to determine impediments to motion of autonomous vehicles

– identify faces in images– identify fish in coral reef (Joe Wilder vision lecture)

Image Processing Levels• Low Level Processing (input and output are images)

– noise reduction – denoising of images– contrast enhancement – improve lighting for viewing images– image enhancement – sharpen image at edges or in points of maximal

interest

• Mid Level Processing (input is an image, output is a set of image attributes)

‐‐ image segmentation (partition into regions or objects); edge detection‐‐ object description (e.g., size, color, orientation)‐‐ object classification (recognition, e.g., faces)

• High Level Processing (Input is an image, output is image understanding)

‐‐making sense of ensemble of recognized objects (e.g., path planning, alerting, alarming)

MATLAB Image Processing

MATLAB Image Processing Fundamentals

( , ) rows, columns size imageconventional indexing (0 : 1 0 : 1)MATLAB Image:f(r,c) (1: ,1: )

f x y M N M NM N

M N

→ ⇒ ×− × −

↔

Conventional image coordinates

MATLAB image coordinates

MATLAB Image Processing(0,0) (0,1) ... (0, 1)(1,0) (1,1) ... (1, 1)

( , )

( 1,1) ( 1,2) ... ( 1, 1)

f(1,1) f(1,2) ... f(1,N)f(2,1) f(2,2) ... f(2,N)

f=

f(M,1) f(M,2) ... f(M,N)

f f f Nf f f N

f x y

f M f M f M N

−⎡ ⎤⎢ ⎥−⎢ ⎥=⎢ ⎥⎢ ⎥− − − −⎣ ⎦

⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦

f(1, 1) = f(0,0)

f(M, N) = f(M-1,N-1)

MATLAB Image Indexing

Conventional Image Indexing

Equivalence of MATLAB and Conventional

Image Indexing

Image Formats for MATLAB

Image IO in MATLAB• Image read command:

– f = imread(‘filename’); % f = imread(‘lena.tif’);– [M,N] = size(f); % M=512, N=512, ; pixels‐8 bits

• Image display command:‐‐ imshow(f, [low, high]); % display as black image intensity at or below low; display as white image intensity at or above high; ‐‐ imshow(f, [ ]); % expands dynamic range so that max=white, min=black‐‐ impixelinfo; % uses cursor to read image coordinates and pixel intensity values

Example

f=imread(‘lena.tif’);whosname: fsize: 512 x 512bytes: 262144class: uint8imshow(f);

Full path read: f=imread(‘C:\data\matlab\image files\lena.tif’);

Example

f = imread(‘saturn.tif’);[M,N] = size(f); % M=328, N=448imshow(f);M=size(f,1); number of rows in fN=size(f,2); number of columns in f

imshow(f, [64,128]);

Example

Example

• f=[255 230 180; 155 130 105; 80 50 0];• imshow(f,[ ],’InitialMagnification’,’fit’);

Image Tool

f=imread(‘rose.tif’);imtool(f);

Pixel Value

Pixel Region

Cropped Image

Overview

Image Tool

imtool is extremely useful tool for gaining an understanding of image properties

Image Tool

pseudo-color with colormap set to ‘hot’; filename:’rose_hot.jpg’

Image IO in MATLAB• Image write command:

– imwrite(f, ’filename’); % writes to filename (must have valid extension)

– imwrite(f,’filename’,’tif’); % writes to filename.tif– imwrite(f, ’filename.jpg’, ’quality’, q); % 0 ≤q≤100

• File size information:– imfinfo filename; imfinfo(‘filename’);– K=imfinfo(‘filename’); % K is structure variable– K.Height, K.Width;

• Export figure images to disk:– print –fno –dfileformat –rresno filename; % no is figure number; fileformat from list; resno=dpi resolution of file– print ‐f1 –dtiff –r300 saturn_clip.tif; % save figure 1, in tiff format, using 300 dpi resolution, to filename ‘saturn_clip.tif’

Example ‐ imfinfo(‘saturn.tif’);Filename: 'saturn.tif'FileModDate: '07‐Jul‐2004 15:31:54'FileSize: 144184Format: 'tif'

FormatVersion: []Width: 438Height: 328BitDepth: 8ColorType: 'grayscale'

FormatSignature: [73 73 42 0]ByteOrder: 'little‐endian'

NewSubFileType: 0BitsPerSample: 8Compression: 'Uncompressed'

PhotometricInterpretation: 'BlackIsZero'StripOffsets: [19x1 double]

SamplesPerPixel: 1RowsPerStrip: 18

StripByteCounts: [19x1 double]XResolution: 72YResolution: 72

ResolutionUnit: 'Inch'Colormap: []

PlanarConfiguration: 'Chunky'TileWidth: []TileLength: []TileOffsets: []

TileByteCounts: []Orientation: 1FillOrder: 1

GrayResponseUnit: 0.0100MaxSampleValue: 255MinSampleValue: 0Thresholding: 1

ImageDescription: [1x168 char]

Example

Example

Image Size• f=imread(‘circuitboard.tif’);• imwrite(f,’circuitboard_r300.tif’,’compression’,’none’,’resolution’,[300 300]);

• K=imfinfo(‘circuitboard_r300.tif’);• K.XResolution=300; K.YResolution=300; 1.5 x 1.5 inches

• imwrite(f,’circuitboard_r200.tif’,’compression’,’none’,’resolution’,[200 200]);

• K=imfinfo(‘circuitboard_r200.tif’);• K.XResolution=200; K.YResolution=200; 2.25 x 2.25 inches

Image Data Classes

Data classes double through single are numeric data classes; Data class char is a character class and data class logical is a logical class.

Image Data Classes• f=imread(‘lena.tif’);• whos

– Name Size Bytes Class Attributes– f 512x512 262144 uint8

• max (f(:))=245; min(f(:))=24;• g=double(f);• whos

– Name Size Bytes Class Attributes– f 512x512 262144 uint8– g 512x512 2097152 double

• max(g(:))=245; min(g(:))=24;

Image Data Classes

• All numeric computations in MATLAB are done using double quantities

• Data class uint8 is mostly used for reading and storing images as 8‐bit images

• Data class double requires 8 bytes per number; data class uint8 requires 1 byte per number

Image Types

• Binary images ‐ logical array of 0s and 1s; obtained using function logical on a numeric array of ones and zeros (halftone)• Indexed images ‐ image with two components, namely a data matrix of integers, X, and a color map matrix, map.• RGB images ‐ image with 3 complementary color sets of intensity

• Intensity images ‐ generally gray scale images whose values represent image intensity; class can be double (floating point), uint8 (integers from 0 to 255), uint16 (integers from 0 to 65535) or scaled double in the range [0,1]

3 x 3 Binary Imagesfht{1}=[0 0 0; 0 0 0; 0 0 0];fht{2}=[0 255 0; 0 0 0; 0 0 0];fht{3}=[0 255 0; 0 0 0; 0 0 255];fht{4}=[255 255 0; 0 0 0; 0 0 255];fht{5}=[255 255 0; 0 0 0; 255 0 255];fht{6}=[255 255 255; 0 0 0; 255 0 255];fht{7}=[255 255 255; 0 0 255; 255 0 255];fht{8}=[255 255 255;0 0 255; 255 255

255];fht{9}=[255 255 255; 255 0 255; 255 255

255];fht{10}=[255 255 255; 255 255 255; 255

255 255];

lh=[ones(1,19)*128];lv=[ones(3,1)*128];

f=[fht{1} lv fht{2} lv fht{3} lv fht{4} lvfht{5};lh; fht{6} ...

lv fht{7} lv fht{8} lv fht{9} lv fht{10}];f1=uint8(f);figure;imshow(f1,'InitialMagnification','fit');

Digital Image Halftoning

Binary Pixels (0, 255)

Halftoning: simulate continuous tone images through the use of dots,

varying in size, shape or spacing.

Varying size dots; view from a distance.

Image Halftoning

lena.tif half tone image

Image Crop and Blow‐Up

Lena Eyes Region

Lighthouse Tower Blow-Up

Half‐Tone Images – Reduced in Size

Other Patterns

% Black-to-white pattern:

g=[ones(256,1)*(0:255)];

figure;imshow(g,[ ]);

Dithered Pattern

Dithered pattern:fht{1}=[0 0 0; 0 0 0; 0 0 0];fht{2}=[0 255 0; 0 0 0; 0 0 0];fht{3}=[0 255 0; 0 0 0; 0 0 255];fht{4}=[255 255 0; 0 0 0; 0 0 255];fht{5}=[255 255 0; 0 0 0; 255 0 255];fht{6}=[255 255 255; 0 0 0; 255 0 255];fht{7}=[255 255 255; 0 0 255; 255 0 255];fht{8}=[255 255 255;0 0 255; 255 255 255];fht{9}=[255 255 255; 255 0 255; 255 255

255];fht{10}=[255 255 255; 255 255 255; 255 255

255];

ght=[];for row=1:256

gloc=[];for col=1:256

gloc=[gloc fht{floor(g(row,col)/25.6)+1}];endght=[ght ; gloc];end

figure,imshow(ght,[ ])’

Image Types

• to convert from gray‐scale image to binary image:– f=imread(‘image.tif’);– g=f;– g(find(f >= 128))=1;– g(find(f < 128))=0;– b=logical(g);– islogical(b)

1

Image Types

lena.tif (512 x 512 x 8 bits) lena_binary.tif (512 x 512 x 1 bits)

Image Conversions• Images classified according to both class_image(double, uint8, …) and type_image (intensity, binary, indexed, RGB)

• Can readily convert between data classes and image types, e.g.,– Data class conversions: B = class_name(A);– A: class uint8 image– B = double(A) converts from uint8 to double– C: class double image– D = uint8(C) converts from double [0 255] to uint8 (integers in [0 255]); uint8 truncates below 0 and above 255

– g = im2uint8(f) scales f to [0 255] then converts to integer values

Image Conversions

Image Conversions

g=mat2gray(f)

g, Gray Scale Image, double [0 1] Image

Processing p=im2uint8(h)

h, Gray Scale Image, double [0 1]

p, Gray Scale Image, uint8

[0 255]

g=im2double(f)

g, Gray Scale Image, double [0 1] Image

Processingscale by 255;p=uint8(h)

h, Gray Scale Image, double [0 1]

p, Gray Scale Image, uint8

[0 255]

f, Gray Scale Image, uint8

[0 255]

f, Gray Scale Image, uint8

[0 255]

Processing Model #1

Processing Model #2

Image Conversions

• uint8: converts double to integers in range of [0 255]

• im2uint8: scale by 255 and integerize to range [0 255]; clip below 0, above 255

• mat2gray: scale to double in range [0 1]; min to 0, max to 1

• im2double: convert integers to double precision in [0 1], scale by 255; clip below 0, above 255

Processing Model #1:

Processing Model #2:

Image Conversions – im2uint8

• f = [‐0.5 0.5; 0.75 1.5];• g = im2uint8(f); What is g?

g=[0 128; 191 255]; f<0 0; % ‐0.5 goes to 0f>1 1; % 1.5 goes to 255scale by 255; 0.5 goes to 128, 0.75 goes to 191

im2uint8 scales an image in the range [0 1] to an image in the range [0 255] by scaling the image values by 255 and clipping below 0 and above 255

Image Conversions – im2double

• im2double converts input to class double– if input of class uint8, uint16 or logical, im2double converts to class double with values in range [0 1]

– if input of class single or double, im2double returns array of class double, but numerically equal to the input

– mat2gray can be used to convert an array of any class to a double array with values in range [0 1]

Image Conversions – im2double

• Example:– h = uint8([25 50; 128 200]);– g = im2double(h);– g=[0.0980 0.1961;0.5020 0.7843]; % converts integers to double precision values in range [0 1];

Image Conversions – mat2gray

• mat2gray converts image of any class to array of class double scaled to the range [0 1]

• calling sequence is:– g = mat2gray(A, [Amin, Amax]); % g has values in range 0 (black) to 1 (white) with clipping below Amin and above Amax; Amin and Amax need not be specified – routine finds min and max prior to conversion and treats them as Amin and Amax

Image Conversion Examples – mat2gray

• h = uint8([25 50; 128 200]);• g = mat2gray(h); % what is g?

g = [0 0.1429; 0.5886 1]; Amin=25; Amax=20025 0; 50 (50‐25)/175=0.1429;128 (128‐25)/175=0.5886; 200 (200‐25)/175=1

• g = im2bw(f,T); % produces a binary image, g, from intensity image, f, by thresholding such that all pixels < T go to 0, all others go to 1

Simple Example

f=imread(‘lena.tif’);figure,imshow(f);max(f(:))=245;min(f(:))=24;

f1=mat2gray(f);f2=im2uint8(f1);figure,imshow(f2);max(f2(:))=255;min(f2(:))=0;

Image Conversion Examples – im2bw

• im2bw performs conversion to class logical where nonzero elements are converted to 1s and zero elements are converted to 0s in the output

• equivalently we could do the following:‐ g = f > T; % where T is the threshold for conversion to 1s and below T inputs converted to 0s‐ g = im2bw(f, T); performs same operations as above; threshold T must be in range [0 1]; for uint8 input, threshold is T*255

Image Conversion Examples• f=[1 2; 3 4];• convert to binary such that values 1 and 2 become 0 and the

other two values become 1– first convert to range [0 1]– g = mat2gray(f) ; g= [0 0.3333; 0.6667 1.0];– next convert to binary using threshold of 0.6– gb = im2bw(g, 0.6); gb= [0 0; 1 1]

• alternatively can use– gb = f > 2; gb=[0 0; 1 1]

• convert gb to numerical array of 0s and 1s:– gbd = im2double(gb); gbd=[0 0; 1 1]

• gbd=im2double(im2bw(mat2gray(f), 0.6));• gbd = double(f > 2);

Image Conversion Examples25 50 0.5 0.75

1 (double) 1 (double)75 100 0.85 0.99

f g⎡ ⎤ ⎡ ⎤= =⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦

25 502 uint8( 1)

75 100f f ⎡ ⎤

= = ⎢ ⎥⎣ ⎦

1 12 uint8( 1) (integerize to [0 255] range)

1 1g g

⎡ ⎤= = ⎢ ⎥

⎣ ⎦

255 2553 im2uint8( 1)

255 255f f ⎡ ⎤

= = ⎢ ⎥⎣ ⎦

128 1913 im2uint8( 1) (scale by 255 and integerize)

217 252g g ⎡ ⎤

= = ⎢ ⎥⎣ ⎦

0 0.334 mat2gray( 1)

0.66 1f f

⎡ ⎤= = ⎢ ⎥

⎣ ⎦

0 0.514 mat2gray( 1) (scale to [0 1] range)

0.71 1g g ⎡ ⎤

= = ⎢ ⎥⎣ ⎦

25 505 im2double( 1)

75 100f f ⎡ ⎤

= = ⎢ ⎥⎣ ⎦

0.5 0.755 im2double( 1) (convert to double)

0.85 0.99g g

⎡ ⎤= = ⎢ ⎥

⎣ ⎦

Image Conversion Examples25 50

1 (uint8)75 100

f ⎡ ⎤= ⎢ ⎥⎣ ⎦

25 502 uint8( 1)

75 100f f ⎡ ⎤

= = ⎢ ⎥⎣ ⎦

255 2553 im2uint8( 1)

255 255f f ⎡ ⎤

= = ⎢ ⎥⎣ ⎦

0 0.334 mat2gray( 1)

0.66 1f f

⎡ ⎤= = ⎢ ⎥

⎣ ⎦

25 505 im2double( 1)

75 100f f ⎡ ⎤

= = ⎢ ⎥⎣ ⎦

Image Conversion Examples

original lena.tif image

f = imread(‘lena.tif’); % image is 512 x 512 gray scale image (8 bits/pixel)


f = imread(‘lena.tif’); % image is 512 x 512 gray scale image (8 bits/pixel)

g = mat2gray(f); % convert scale from [0 255] to [0 1]

h = dither(g); % convert to black-and-white image using dithering, 512 x 512 bits


hnd = im2bw(g); % convert to black-and-white image without dithering




MATLAB Demo: Counting Grains of Rice in an

Image

Counting Grains of Rice

• raw image shows grains of rice– the image has non‐uniform illumination; the background is brighter in the center of the image than at the bottom

– goal is to enhance the image to correct for the non‐uniform illumination

– use the enhanced image to identify individual grains

– compute the grain statistics

% Step 1 -- read in image and displayI=imread('rice.png');imshow(I);

% Step 2 -- estimate background using morphological opening operation% Morphological opening is an erosion followed by a dilation, % using the same structuring element for both operations.% The opening operation has the effect of removing objects% that cannot completely contain the structuring element.% strel('disk',15): disk-shaped structuring element with radius 15% structuring element must be bigger than a single grain of rice

background=imopen(I,strel('disk',15));

% Step 3 -- view background approximation as a surface; first convert background to double and use surf command to display every 8th pixel in each direction

figure; surf(double(background(1:8:end,1:8:end)));

% limit range to that of uint8 data to match input image (for subtraction)

zlim([0 255]);

% reverse y-axis of display to provide better view of surfaceset(gca,'ydir','reverse');

% [0 0] is origin of image; high part of curve indicates highest pixel values of background; this occurs at middle rows of image


% Step 4 -- subtract background image from original image

I2=I - background;figure; imshow(I2);

% Step 5 -- increase image contrast --background is too dark, using imadjust

I3=imadjust(I2);figure; imshow(I3);


Counting Grains of Rice% Step 6 -- threshold image to create binary version of image in order to count the number of grains of rice% graythresh automatically computes threshold for conversion from gray% scale to binary image

level=graythresh(I3);

% im2bw converts grayscale image to binary image using thresholdingbw=im2bw(I3,level);

% bwareaopen removes background noisebw=bwareaopen(bw,50);figure; imshow(bw);

% Step 7 -- identify objects in image% bwconncomp finds all connected components (objects) in binary image

cc=bwconncomp(bw,4);cc.NumObjects

ans =95

% Step 8 -- examine a single detected object; each distinct object is labeled—show 50th connected component

grain=false(size(bw));grain(cc.PixelIdxList{50})=true;figure; imshow(grain);

Counting Grains of Rice% Step 9 -- view all objects% create a label matrix and display it as a pseudo-color indexed image% labelmatrix creates a label matrix from output of bwconncomp

labeled=labelmatrix(cc);whos labeled

Name Size Bytes Class Attributes

labeled 256x256 65536 uint8

% label2rgb creates colormap, choose background color, and maps objects in% labelmatrix map to colors in colormap

RGB_label=label2rgb(labeled,@spring,'c','shuffle');figure; imshow(RGB_label);

% Step 10 -- compute area of each object% regionprops operates on cc to compute object areas

graindata=regionprops(cc,'basic')

% find area of 50th componentgraindata(50).Area

% Step 11 -- compute area-based statisticsgrain_areas=[graindata.Area];

% find grain with smallest area[min_area,idx]=min(grain_areas);grain=false(size(bw));grain(cc.PixelIdxList{idx})=true;figure; imshow(grain);min_area

graindata = 95x1 struct array with fields:

AreaCentroidBoundingBox

ans =194min_area = 61

% Step 12 -- create area histogramnbins=20;figure; hist(grain_areas,nbins);title('Histogram of Rice Grain Area');


Array Indexing

• V = [1 3 5 7 9]; % 1 row, 5 columns• V’ = V(:) = [1; 3; 5; 7; 9]; % 5 rows, 1 column

• V(3:end) = [5 7 9]; % 1 row, 3 columns• V(end:‐2:1) = [9 5 1]; % 1 row, 3 columns• V([1 4 5]) = [1 7 9]; % 1 row, 3 columns• V = linspace(a,b,n); generates row vector beginning at a, ending at b, with n equi‐spaced elements• V = linspace(1,9,5) = [1 3 5 7 9]; % 1 row, 5 columns

• V(1:3) = [1 3 5]; % 1 row, 3 columns

Matrix Indexing• A = [1 2 3; 4 5 6; 7 8 9]; % 3 rows, 3 columns• A(2, 3) = 6; % second row, third column entry• A(:, 3) = [3; 6; 9]; % third column, all entries• A(2, :) = [4 5 6]; % second row, all entries• A(1:2, 1:3) = [1 2 3; 4 5 6]; % first and second rows, all three columns• A(end, end) = 9; % last entry in matrix• A(end, end – 2)=7; % entry in end row, end column ‐2• A(2:end, end:‐2:1) = [6 4; 9 7]; rows 2‐3, columns 3 and 1 (backwards)• A([1 3], [2 3]) = [2 3;8 9]; % rows 1 and 3, columns 2 and 3• A(:)=[1; 4; 7; 2; 5; 8; 3; 6; 9]; % columns concatenated

Matrix Operations – matrix sums

• matrix sums (3 possible ways of computing)– A = [1 2 3; 4 5 6; 7 8 9]; % 3 rows, 3 columns

– col_sums = sum(A); % col_sums=[12 15 18]

– total_sum = sum(col_sums); % total_sum=45

– total_sum1 = sum(A(:)); % total_sum1=45;

– total_sum2 = sum(sum(A)); % total_sum2=45

Max Operator

• C = max(A); % C = row vector with max of each column

• C = max(A, B); % C = array of largest elements of A or B

• C = max(A, [ ], dim); % C has the largest elements along dimension of A specified by scalar dim• [C, I] = max(A); % C is a row vector with max of each column, and I is a row vector with row index for each column max

Max Operator Examples• A = [1 4 7;2 5 8; 3 6 9]; B = [9 6 3;8 5 2; 7 4 1];

• C = max(A) = [3 6 9]; % maximum of each column

• C = max(A, B) = [9 6 7; 8 5 8; 7 6 9]; % largest elements of A or B at each row and column index• C = max(A, [ ], 2) = [7; 8; 9]; % largest elements along dimension of A specified by last argument (2=rows, 1=columns)• C= max(A, [ ], 1) = [3 6 9];• [C, I] = max(A)=max(A, [ ], 1); C = [3 6 9]; I = [3 3 3]; % I = indices of maximums• [C, I]=max(A, [ ], 2); C=[7; 8; 9], I=[3 3 3];

Matrix Operations – linear indexing• use a single subscript to index a matrix• example:

1 0.5 0.3333 0.250.5 0.3333 0.25 0.2

0.3333 0.25 0.2 0.16670.25 0.2 0.1667 0.1429

⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎣ ⎦

H =

• H([2 11]); % H=[0.5 0.2] – extracts 2nd and 11thelements of H; elements counted by columns

• extract values of H at (1,3), (2,4) and (4,3) – we do this by extracting row and column indices of the three pixels, i.e., (1,3) maps to 9; (2,4) maps to 14; (4,3) maps to 12

Matrix Operations – linear indexing

• MATLAB functions sub2ind and ind2sub convert back and forth between row‐column subscripts and linear indices‐ linear_indices = sub2ind(size(H), r, c); % r is the row index (or array of indices), c is the column index (or array of indices)- linear_indices = sub2ind([4 4],[1 2 4], [3 4 3]); % linear_indices = [9 14 12];‐ [r, c] = ind2sub(size(H), linear_indices);

- [r, c]=ind2sub([4 4], [9 14 12]); % r=[1 2 4], c=[3 4 3]

Matrix Indexing Examplef = imread(‘rose.tif’);imshow(f);

fp=f(end:‐1:1,:);flip vertically

(c) - fc=f(257:768,257:768);% image section(d) – fs=f(1:2:end,1:2:end); % subsampled image(e) = plot(f(512,:)); horizontal % scan line

Standard MATLAB Arrays• zeros(M, N) – generates an M row, N column array of 0s of class double

• ones(M, N) – generates an M row, N column array of 1s of class double• rand(M, N) – generates an M row, N column array whose entries are uniformly distributed random numbers in the interval [0 1]• randn(M, N) – generates an M row, N column array whose numbers are normally distributed (i.e., Gaussian) random numbers with mean 0 and variance 1

Standard MATLAB Arrays

• A = 5*ones(3, 3); % A=[5 5 5;5 5 5; 5 5 5]• B = randn(1, 5000); hist(B, 100);

Array and Matrix Arithmetic Operators

Array and Matrix Arithmetic Operators

1 2 1 2,

3 4 3 4A = B =

a a b ba a b b⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦

The array product of and is:1 1 2 23 3 4 4

A B

A.*B =a b a ba b a b⎡ ⎤⎢ ⎥⎣ ⎦

i

whereas the matrix product yields the familiar result:1 1 2 3 1 2 2 43 1 4 3 3 2 4 4

A *B =a b a b a b a ba b a b a b a b

+ +⎡ ⎤⎢ ⎥+ +⎣ ⎦

i

MATLAB m function• write MATLAB function that blends two input images in equal

weightfunction [w, wmax, wmin]=imblend(f, g)% w is a blend of f and g with wmax and wmin the maximum and

minimum values of the blendw1 = 0.5 * f;w2 = 0.5 * g;w = w1 + w2;wmax = max (w(:));wmin = min(w(:));% could use w = 0.5 * (f + g); % however when (f + g) exceeds

255 (for a uint8 array) the sum saturates and we get a wrong result

f = [1 2; 3 4]; g = [1 2; 2 1];[w, wmax, wmin] = imblend(f, g);w=[1 2; 2.5 2.5];wmax=2.5;wmin=1;

MATLAB m function• to alleviate the saturation problem use MATLAB function

g = imlincomb(k1, f1, k2, f2)k1 = weight for f1;f1 = image 1;k2 = weight for f2;f2 = image 2;w = imlincomb(0.5, f, 0.5 g); % this no

longer saturates on uint8 arrays

Array and Matrix Arithmetic Operators for Images (uint8 arrays)

MATLAB Relational and Logical Operators

A == B; 1 when A(i,j) = B(i,j), 0 otherwiseA ~= B; 1 when A(i,j) ≠ B(i,j), 0 otherwiseA >= B; 1 when A(i,j) ≥ B(i,j), 0 otherwiseA <= B; 1 when A(i,j) ≤ B(i,j), 0 otherwise

A = [1 2 0;0 4 5]; B = [1 ‐2 3;0 1 1];A&B = [1 1 0;0 1 1];A|B = [1 1 1;0 1 1];~A = [0 0 1;1 0 0];~B = [0 0 0;1 0 0];A == B = [1 0 0;1 0 0];A ~= B = [0 1 1;0 1 1];A >= B = [1 1 0;1 1 1];A <= B = [1 0 1;1 0 0];

MATLAB Relational Examples• A = [1 2 3; 4 5 6; 7 8 9];• B = [0 2 4; 3 5 6; 3 4 9];• A == B; [0 1 0; 0 1 1; 0 0 1];

• A >= B; [1 1 0; 1 1 1; 1 1 1];• C = [1 2 0; 0 4 5];• D = [1 ‐2 3; 0 1 1];• C & D; [1 1 0; 0 1 1];• C | D; [1 1 1; 0 1 1];

Image Addition

Image Subtraction

Image 1 – Image 2; tendency to blacks (intensities near 0)

Image 1 – Image 2; histogram equalized; improved brightness and contrast (still dark image)

MATLAB Function, fgprod

Write MATLAB function, fgprod, that multiplies two input images and outputs the product, max and min of the product, and the normalized product whose values are in the [0 1] range.function [p, pmax, pmin, pn] = fgprod(f,g)% FGPROD computes the product of two images.% [P,PMAX,PMIN,PN] = FGPROD(F,G) outputs the element‐by‐element product% of two input images, F and G, the product maximum and minimum values,% and a normalized product array with values in the range [0, 1]. The input images% must be of the same size. They can be of class uint8, uint16, or double. The% outputs are of class double.fd = double(f); % use double rather than im2double to retain original intensity rangegd = double(g);p = fd.*gd;pmax = max(p(:));pmin = min(p(:));pn = mat2gray(p); % obtain a normalized array in the range [0, 1]

MATLAB Function, fgprod

Product of two images; again there is a tendency to black as the product of two numbers (both <1) is smaller than either number (i.e., it tends to 0 which is black.

Product of two images – histogram equalized. We see a huge improvement in brightness and contrast as the histogram fills in white shades from dark shades.

MATLAB Flow Control

MATLAB Flow Control ‐ Switchswitch switch_expression

case case_expressionstatement(s)

case {case_expression1, case_expression2, …}statement(s)

otherwisestatement(s)

end

switch newclasscase ‘uint8’

g = im2uint8(f);case ‘uint16’

g = im2uint16(f);case ‘double’

g = im2double(f);otherwise

error(‘Unknown or improper image class.’);end

MATLAB Timing

• tic; % sets variable tic to current clock time• run MATLAB code to be timed• toc; – Elapsed time is 0.xxx seconds• for timing function calls can use special function s=timeit(f) where f is the function handle for the function to be timed, and s is the measurement time, in seconds (timeit must be retrieved from course website)

Vectorizing Loops – Example 1

f(x) = Asin(x / 2π), x = 0,1,2,...,M-1For loop implementation : A =10; for x =1:M % Array indices in MATLAB cannot be 0 f(x) = A * sin((x -1) / (2 *pi)); endVectorized MATLAB code : A =10; tpi = 2*pi; x = 0 :M-1; f = A * sin(x / tpi);

Ratio of time for loop versus vectorized MATLAB code: 1.8


0 0( , ) sin( )f x y A u x v y= +

Create MATLAB function to create a synthetic image of the form : First version uses nested "for" loops function f = twodsin1(A, u0, v0, M, N) f = zeros(M, N); % pre -allocation of memory for c = 1:N v0y = v0 * (c -1); for r = 1:M u0x = u0 * (r - 1); f(r, c) = A * sin(u0x + v0y); end endTime this version : f = twodsin1(1, 1/ (4 *pi), 1/ (4 *pi), 512, 512); imshow(f, [ ]);

Time: 0.0215 seconds


ii

i

Vectorize function using meshgrid [C, R] = meshgrid(c, r); c, r vectors of horizontal (column) and vertical (row) coordinates meshgrid transforms coordinate vectors into arrays

z x y

xy

= +

C and R that can be used to compute function of two variablesExample : want to evaluate function forinteger values of ranging from 1 to 3, and forinteger values of ranging from 10 to 14 [X, Y] = meshgrid(1: 3, 10 :14); X = [1 2 3; 1 2 3; 1 2 3; 1 2 3; 1 2 3]; Y = [10 10 10; 11 11 11; 12 12 12; 13 13 13; 14 14 14]; Z = X + Y = [11 12 13; 12 13 14; 13 14 15; 14 15 16; 15 16 17];

Vectorizing Loops – Example 2Rewrite 2 -D sine function without loops function f = twodsin2(A, u0, v0, M, N) r = 0 :M - 1; % row coordinates c = 0 :N - 1; % column coordinates [C, R] = meshgrid(c, r); f = A * sin(u0 *R + v0 * C);

Time: 0.0157 secondsRatio of times: 1.369

Interactive IO• disp(argument) – display on screen• A = [1 2;3 4]; disp(A);

1 23 4

• sc = ‘text string’; disp(sc);‘text string’

• t = input(‘message:’, ’s’) – accept screen inputn = input(‘specify value for n:’);t = input(‘specify value for n:’,’s’); n=str2num(t);

• [a,b,c,…] = strread(cstr, ’format’, ’param’, ’value’);• format = %f (floating point), %q(character strings), %d (integer)

t = ’12.6, x2y, z’; [a,b,c] = strread(t, ’%f%q%q’, ’delimiter’, ’,’);a = 12.6; b = ‘x2y’, c = ‘z’ (b and c are cells); convert cells to character arrayd = char(b) = x2y, e = char(c) = z

String Compare• We assume that parameter param can have two forms, namely ‘norm1’ ([0 1]) or

‘norm255’ ([0 255])• Use strcmp to decide which option to use

f is image inputf = double(f); % floating point valuesf = f‐min(f(:)); % min(f) set to 0f = f./max(f(:)); % max(f) set to 1if (strcmp(param, ’norm1’)

g = f;elseif strcmp(param, ’norm255’)

g = 255*f;else

error(‘Unknown value of param.’);end

Cell Arrays and Structures• Cell array – multidimensional array whose elements are

copies of other arrays – signaled by { and } bracketsc = {‘gauss’, [1 0; 0 1], 3};c{1} = gauss (a character string) c{2} = [1 0; 0 1] ( a 2 x 2 matrix) c{3} = 3 ( a scalar)

• Structure – grouping of a collection of dissimilar data into a single variable – elements of structures addressed by fieldsS=structure ‐‐ enter elements of S via simple commands, e.g.,

S.char_string = ‘gauss’;S.matrix = [1 0; 0 1];S.scalar = 3;

MATLAB Image Processing Commands

• f = imread(‘filename’); % reads in file to array

• imshow(f); % displays array as image, [0 255]‐ imshow(f, [low high]); % f(i,j) ≤ low => low (black)

‐ % f(i,j) ≥ high => high (white)‐ imshow(f, [ ]); % low = min(min(f)); high = max(max(f));

• pixval; % displays intensity values of pixels

• imwrite(f, ’filename’); % writes image to disk‐ imwrite(f, ‘filename’, ‘tif’);‐ imwrite(f, ‘filename.jpg’, ‘quality’, q); % q in [0 100] range

MATLAB Image Processing Commands

• K = imfinfo filename; % get file information in structure K– image_bytes = K.Width*K.Heights*K.Depth/8;

• g = im2uint8(f); % convert from image to uint8‐ f ≤ 0 => 0; f ≥ 1 => 1; scale by 255 and round

• g = mat2gray(A, [Amin Amax]); % converts double [ ] to double [0 1]

• g = im2double(f); % converts image of uint8 or double to double

‐ g = im2double(mat2gray(f)); % converts to [0 1] range

Images on Websiteblood1.tif; 265 rows, 272 columns, min=46, max=255

breast.tif; 570 rows, 482 columns, min=21, max=255

building.tif; 240 rows, 320 columns, min=21, max=255

chalk.tif; 1040 rows, 1040 columns, min=0, max=255

checker.tif; 512 rows, 512 columns, min=0, max=255

circuitboard.tif; 450 rows, 450 columns, min=0, max=255

FTspectrum.tif; 257 rows, 257 columns, min=0, max=255

hardware.tif; 240 rows, 320 columns, min=4, max=248

Images on Websiteiris.tif; 600 rows, 600 columns, min=0, max=255

iris_gray.tif; 600 rows, 600 columns, min=27, max=255

iris_luminence.tif; 600 rows, 600 columns, min=22, max=255

lena.gif; 512 rows, 512 columns, min=19, max=191

lena.tif; 512 rows, 512 columns, min=24, max=245

lighthou.tif; 240 rows, 320 columns, min=21, max=255

moon.tif; 540 rows, 466 columns, min=0, max=254

parrots.tif; 256 rows, 384 columns, min=0, max=254

Images on Websitepeppers.jpg; 512 rows, 512 columns, min=0, max=245

rose.tif; 1024 rows, 1024 columns, min=0, max=255

saturn.tif; 256 rows, 256 columns, min=0, max=255

shuttle.tif; 240 rows, 320 columns, min=32, max=255

tooth1.jpg; 512 rows, 512 columns, min=0, max=229

xray.tif; 240 rows, 320 columns, min=42, max=255

Lecture IP1 2012

Documents

conventional image indexing

additive gaussian noise

2d gaussian filter

noisy lena filtered

lena image filtered

bit gray scale

original lena image

structuring element