Computational Biology, Part 23 Biological Imaging II Robert F. Murphy Copyright 1996, 1999, 2000-2006. All rights reserved.

Computational Biology, Part 23

Biological Imaging II

Computational Biology, Part 23

Biological Imaging II

Robert F. MurphyRobert F. Murphy

Copyright Copyright 1996, 1999, 1996, 1999, 2000-2006.2000-2006.

All rights reserved.All rights reserved.

OutlineOutline

Image DisplayImage Display Image ProcessingImage Processing Image AnalysisImage Analysis Image InterpretationImage Interpretation

From Images to Knowledge From Images to Knowledge

Image ProcessingImage Image

Image AnalysisImage

Image InterpretationImage

Numbers

Knowledge

Image DisplayImage Display

Operations that change Operations that change display without changing display without changing imageimage LUT - grayscale or colorLUT - grayscale or color Contrast stretchingContrast stretching

Operations that change imageOperations that change image reversiblereversible non-reversible (majority)non-reversible (majority)

Image Display - LUT changeImage Display - LUT change

Image Display - LUT changeImage Display - LUT change

Image Display - Enhance contrastImage Display - Enhance contrast

Image DisplayImage Display After enhancement uses full range

Original (before contrast enhancement)

ThresholdingThresholding

Thresholding refers to the division Thresholding refers to the division of the pixels of an image into two of the pixels of an image into two classes: those below a certain classes: those below a certain value (the value (the thresholdthreshold) and those at ) and those at or above it. The two classes are or above it. The two classes are often shown in white and black, often shown in white and black, respectively.respectively.

Thresholding serves as a means to Thresholding serves as a means to consider only a consider only a subsetsubset of the of the pixels of an images.pixels of an images.

ThresholdingThresholding

The choice of threshold must be The choice of threshold must be made empirically by considering the made empirically by considering the nature of the sample, the type and nature of the sample, the type and number of objects expected in the number of objects expected in the image, and/or a histogram of pixel image, and/or a histogram of pixel valuesvalues

The threshold can be specified as a The threshold can be specified as a multiple of the background value multiple of the background value (normally the most common value) (normally the most common value) for partial automationfor partial automation

ThresholdingThresholding

ThresholdingThresholding

Once a threshold has been Once a threshold has been applied, the resulting image applied, the resulting image may bemay be displayeddisplayed in black and white in black and white displayeddisplayed with above threshold with above threshold pixels at their original pixels at their original intensities and below threshold intensities and below threshold pixels in blackpixels in black

ThresholdingThresholding

Once a threshold has been Once a threshold has been applied, the resulting image may applied, the resulting image may bebe savedsaved as a new image with only as a new image with only pixels above threshold being pixels above threshold being retained (others set to 0)retained (others set to 0)

savedsaved as or as or convertedconverted to a binary to a binary image (above threshold pixels set to image (above threshold pixels set to 1, below threshold pixels set to 0)1, below threshold pixels set to 0)

Binary image operationsBinary image operations ErosionErosion

Remove pixels from edges of objectsRemove pixels from edges of objects Set “on” pixel to “off” if four or Set “on” pixel to “off” if four or more of its eight neighbors are more of its eight neighbors are whitewhite

DilationDilation Add pixels to edges of objectsAdd pixels to edges of objects Set “off” pixel to “on” if four or Set “off” pixel to “on” if four or more of its neighbors are blackmore of its neighbors are black

Binary image operationsBinary image operations Process/Binary/

Threshold does auto threshold and applies it to make binary image

Binary image operationsBinary image operations

Binary image operationsBinary image operations

This image shows just the pixels that were turned off by the erode operation

Binary image operationsBinary image operations OpenOpen

Smooth objects and fill in small Smooth objects and fill in small holesholes

Erosion followed by dilationErosion followed by dilation CloseClose

Smooth objects and fill in small Smooth objects and fill in small holesholes

Dilation followed by erosionDilation followed by erosion

Binary image operationsBinary image operations OutlineOutline

Find “on” pixel, trace around Find “on” pixel, trace around outside until return to first outside until return to first “on” pixel“on” pixel

SkeletonizeSkeletonize Remove pixels from the edges of Remove pixels from the edges of objects until the objects are objects until the objects are one pixel wideone pixel wide

Binary image operationsBinary image operations

Binary image operations - outlineBinary image operations - outline

Basic Image Processing OperationsBasic Image Processing Operations

Image MathImage Math Kernel/Filter OperationsKernel/Filter Operations Image CalculatorImage Calculator

Arithmetic OperationsArithmetic Operations

Two cases:Two cases: Perform a Perform a singlesingle operand operation operand operation (e.g., logarithm, square root) on each (e.g., logarithm, square root) on each pixel of an imagepixel of an image

Perform a Perform a dualdual operand operation operand operation (e.g., add, multiply) on each pixel of (e.g., add, multiply) on each pixel of an image using a an image using a constantconstant as the as the second operandsecond operand

In both cases, the result is In both cases, the result is usually stored in the same pixel usually stored in the same pixel location (“storing in place”)location (“storing in place”)

Arithmetic OperationsArithmetic Operations

Kernel/Filter OperationsKernel/Filter Operations Basic idea: Use a matrix (usually Basic idea: Use a matrix (usually square and of odd dimension, e.g., square and of odd dimension, e.g., 3x3) in combination with an image to 3x3) in combination with an image to generate a new imagegenerate a new image

Algorithm:Algorithm: For each pixel in the image (the For each pixel in the image (the currentcurrent pixelpixel))

Align the matrix to center it on that Align the matrix to center it on that pixelpixel

For each position in the matrix, For each position in the matrix, multiply the corresponding pixel value multiply the corresponding pixel value in the image by the value in the matrix in the image by the value in the matrix and sum the resultsand sum the results

Store the result in the Store the result in the currentcurrent pixelpixel

Kernel/Filter OperationsKernel/Filter Operations A matrix used in this fashion is A matrix used in this fashion is called a called a kernel kernel oror filter filter

Note that the operation is Note that the operation is different from matrix different from matrix multiplication of the kernel by multiplication of the kernel by the image becausethe image because the dimensions don’t match, andthe dimensions don’t match, and all elements of the matrix are all elements of the matrix are combined to give one resultcombined to give one result

Common Kernel Operations used in Image Processing

Common Kernel Operations used in Image Processing SmoothingSmoothing SharpeningSharpening Edge FindingEdge Finding

OrigiOriginal nal imageimage

Examples of Kernel Operations using NIH Image

Examples of Kernel Operations using NIH Image

SmoothSmooth

ResulResults of ts of one one SmootSmooth h

ResulResults of ts of a a seconsecond d SmootSmooth h

Examples of Kernel Operations using NIH Image

Examples of Kernel Operations using NIH Image

Close smoothed image, reopen original Close smoothed image, reopen original image, then Sharpenimage, then Sharpen

OrigiOriginal nal imageimage

Image Image after after one one SharpSharpenen

Image Image after after a a seconsecond d SharpSharpenen

Examples of Kernel Operations using NIH Image

Examples of Kernel Operations using NIH Image

Close sharpened image, reopen original Close sharpened image, reopen original image, then Find Edgesimage, then Find Edges

Image Image after after Find Find EdgesEdges

Example kernelsExample kernels

SmoothingSmoothing

1 1 11 4 11 1 1

Example kernelsExample kernels

SharpenSharpen

-1 -1 -1-1 12 -1-1 -1 -1

Example kernelsExample kernels

Edge detection (Sobel)Edge detection (Sobel)

1 2 10 0 0-1 -2 -1

Image MathImage Math

Basic idea: Combine two images Basic idea: Combine two images using an dual operand operator to using an dual operand operator to generate a new imagegenerate a new image

Algorithm:Algorithm: For each pixel in the first image, For each pixel in the first image, operate on it using the operate on it using the corresponding pixel in the second corresponding pixel in the second image and store the result in the image and store the result in the corresponding pixel in a new corresponding pixel in a new (output) image(output) image

Image MathImage Math

Any operator can be usedAny operator can be used Most common operators:Most common operators:

division: generate ratio imagedivision: generate ratio image logical AND: logical AND: maskmask one image with one image with another (usually binary) imageanother (usually binary) image

Examples of Image Math using NIH ImageExamples of Image Math using NIH Image

Open original image and sharpen once (save as Open original image and sharpen once (save as Abdomen.sharpen1), reopen original imageAbdomen.sharpen1), reopen original image

Ratio Ratio of of sharp sharp to to originaoriginal image l image (shows (shows regions regions affecteaffected by d by sharpensharpen))

Image Math vs. Arithmetic OperationsImage Math vs. Arithmetic Operations Note difference between Image Math Note difference between Image Math which does an operation on two which does an operation on two images and Arithmetic which does images and Arithmetic which does an operation on a single image and an operation on a single image and a constanta constant

Summary: Basic Image Processing OperationsSummary: Basic Image Processing Operations Arithmetic OperationsArithmetic Operations

Inputs: Image, Constant Inputs: Image, Constant (optional)(optional)

Common use: Subtract backgroundCommon use: Subtract background Kernel OperationsKernel Operations

Inputs: Image, KernelInputs: Image, Kernel Common use: SmoothingCommon use: Smoothing

Image MathImage Math Inputs: Two imagesInputs: Two images Common use: Generate ratio imageCommon use: Generate ratio image

Image ProcessingImage Processing

Basic example of image Basic example of image processing: find objects in processing: find objects in an image and describe them an image and describe them numericallynumerically

Object finding (Particle analysis)Object finding (Particle analysis) Principle: Identify a contiguous Principle: Identify a contiguous set of pixels that are all above set of pixels that are all above some thresholdsome threshold

Implementation:Implementation: Start with a binary (thresholded) imageStart with a binary (thresholded) image Find a pixel that is “on” and start a Find a pixel that is “on” and start a list or maplist or map

Recursively search all nearest Recursively search all nearest neighbors for additional pixels that neighbors for additional pixels that are on and add them to the list or mapare on and add them to the list or map

Object finding (Particle analysis)Object finding (Particle analysis)

Start with a thresholded image (Image/Adjust/Threshold)

Object finding (Particle analysis)Object finding (Particle analysis)

Object finding (Particle analysis)Object finding (Particle analysis)

Save as Excel file using Save As...

Object finding (Particle analysis)Object finding (Particle analysis) Uses:Uses:

Counting objectsCounting objects Obtaining area measurements for Obtaining area measurements for objectsobjects

Obtaining integrated intensityObtaining integrated intensity Isolating objects for other Isolating objects for other processingprocessing