22 Image Processing

8/13/2019 22 Image Processing

1/51

Image Processing

Point Processing

FiltersDitheringImage CompositingImage Compression


2/51

Images

Image stored in memory as 2D pixel array Value of each pixel controls color Depth of image is information per pixel

1 bit: black and white display 8 bit: 256 colors at any given time via colormap 16 bit: 5, 6, 5 bits (R,G,B), 2 16 = 65,536 colors 24 bit : 8, 8, 8 bits (R,G,B), 2 24 = 16,777,216 colors


3/51

Fewer Bits: Colormaps

Colormaps typical for 8 bit framebuffer depth With screen 1024 * 768 = 786432 = 0.75 MB Each pixel value is index into colormap Colormap is array of RGB values, 8 bits each

Only 2 8 = 256 at a time Poor approximation of full color

i

R G B

0

1

2

255

R G B

R G B

255 0 0


4/51

Image Processing

2D generalization of signal processing Image as a two-dimensional signal Point processing : modify pixels independently

Filtering : modify based on neighborhood Compositing : combine several images Image compression: space-efficient formats Related topics (not in this lecture or this course)

Image enhancement and restoration Computer vision


5/51

Outline

Point Processing Filters Dithering Image Compositing Image Compression


6/51

Point Processing Input: a[x,y], Output b[x,y] = f(a[x,y])

f transforms each pixel value separately Useful for contrast adjustment

Suppose our picture is grayscale (a.k.a. monochrome).Let v denote pixel value, suppose its in the range [0,1].

f(v) = v identity ; no change

f(v) = 1-v negate an image(black to white, white to black)

f(v) = v p , p1 darken

v

f(v)


7/51

Point Processing

f(v) = v identity; no change

f(v) = 1-v negate an image(black to white, white to black)

f(v) = v p

, p1 darken

v

f(v)


8/51

Gamma correction compensates fordifferent monitors

G = 1.0; f(v) = v G = 2.5; f(v) = v 1/2.5 = v 0.4

Monitors have a intensity to voltage response curve which is roughly a 2.5 power function Send v actually display a pixel which has intensity equal to v 2.5


9/51

Outline

Point Processing Filters Dithering Image Compositing Image Compression


10/51

Signals and Filtering

Audio recording is 1D signal: amplitude(t) Image is a 2D signal: color(x,y) Signals can be continuous or discrete

Raster images are discrete In space: sampled in x, y In color: quantized in value

Filtering: a mapping from signal to signal


11/51

Convolution

Used for filtering, sampling and reconstruction Convolution in 1D

Chalkboard


12/51

Convolve box and step


13/51

Convolution filters

box

tent

gaussian


14/51

Convolution in 1D

a(t) is input signal b(s) is output signal

h(u) is filter

Convolution in 2D

Convolution filters


15/51

Filters with Finite Support

Filter h(u,v) is 0 except in given region Represent h in form of a matrix Example: 3 x 3 blurring filter

As function

In matrix form


16/51

Blurring Filters

A simple blurring effect can be achieved with a 3x3filter centered around a pixel,

More blurring is achieved with a wider n n filter:

Original Image Blur 3x3 mask Blur 7x7 mask


17/51

Image Filtering: Blurring

original, 64x64 pixels 3x3 blur 5x5 blur


18/51

Blurring Filters

Average values of surrounding pixels Can be used for anti-aliasing What do we do at the edges and corners?

For noise reduction , use median, not average Eliminates intensity spikes Non-linear filter


19/51

Example: Noise Reduction

Image with noise Median filter (5x5)


20/51

Example: Noise Reduction

Original image Image with noise Median filter (5x5)


21/51

Edge Filters

Discover edges in image Characterized by large gradient Approximate square root

Approximate partial derivatives, e.g.

Filter =000

101

000


22/51

Sobel Filter

Edge detection filter, with some smoothing Approximate

Sobel filter is non-linear Square and square root (more exact computation) Absolute value (faster computation)


23/51

Sample Filter Computation

Part of Sobel filter, detects vertical edges

-1

-1-2

0

00

1

12

4

1252525

25252525

2525

25

252525

25252525

2525

25

252525

25252525

2525

25

252525

25252525

2525

25

252525

25252525

2525

25

000

0000

00

0

000

0000

00

0

000

0000

00

0

000

0000

00

0

000

0000

00

0

a b

252525

25252525

2525

25

000

0000

00

0

000

0000

00

000

0000

00

0

000

0000

00

0

252525

25252525

2525

25

000

0000

00

0

000

0000

00

0

000

0000

00

0

000

0000

00

0 0

h


24/51

Example of Edge Filter

Original image Edge filter, then brightened


25/51

Image Filtering: Edge Detection


26/51

Outline

Display Color Models Filters Dithering

Image Compositing Image Compression


27/51

Dithering

Compensates for lack of color resolution Eye does spatial averaging Black/white dithering to achieve gray scale

Each pixel is black or white

From far away, color determined by fraction of white For 3x3 block, 10 levels of gray scale

h


28/51

Dithering

Dithering takes advantage of the human eye's tendency to "mix"

two colors in close proximity to one another.

Di h i


29/51

Dithering

Dithering takes advantage of the human eye's tendency to "mix"

two colors in close proximity to one another.

original no dithering with dithering

Colors = 2 24 Colors = 2 8 Colors = 2 8

O d d Di h i


30/51

Ordered Dithering

How do we select a good set of patterns? Regular patterns create some artifacts Example of good 3x3 dithering matrix

6 8

1 0 3

5 2 7

Fl d St i b E Diff i


31/51

Floyd-Steinberg Error Diffusion Diffuse the quantization error of a pixel to its neighboring pixels

Scan in raster order At each pixel, draw least error output value Add the error fractions into adjacent, unwritten pixels

If a number of pixels have been rounded downwards, it becomesmore likely that the next pixel is rounded upwards

7/16

3/16 5/16 1/16

Fl d St i b g E Diff i


32/51

Floyd-Steinberg Error Diffusion

Floyd Steinberg Error Diffusion


33/51

Floyd-Steinberg Error Diffusion

From http://www.cs.rit.edu/~pga/pics2000/node1.html

Enhances edges

Retains high frequencySome checkerboarding

Color Dithering
http://www.cs.rit.edu/~pga/pics2000/node1.htmlhttp://www.cs.rit.edu/~pga/pics2000/node1.html


34/51

Color Dithering

Example: 8 bit framebuffer Set color map by dividing 8 bits into 3,3,2 for RGB Blue is deemphasized because we see it less well

Dither RGB separately Works well with Floyd-Steinberg

Generally looks good

Outline


35/51

Outline



Image Compositing


36/51

Image Compositing

Represent an image as layers that are composited(matted) together

Image Compositing


37/51

Image Compositing

To support this, give image an extra alpha channel inaddition to R, G, B

Alpha is opacity: 0 if totally transparent, 1 if totallyopaque

Alpha is often stored as an 8 bit quantity; usually notdisplayed.

Mathematically, to composite a 2 over a 1 according tomatte

b(x,y) = (1- (x,y))a 1(x,y)+ (x,y)a 2 (x,y) = 0 or 1 -- a hard matte, = between 0 and 1 -- a soft matte

Compositing is useful for photo retouching and specialeffects.

Special Effects: Compositing


38/51

Special Effects: Compositing

Lighting match Proper layering Contact with the real world Realism (perhaps)

ApplicationsCel animationBlue-screen matting


39/51

Roger Rabbit

http://members.tripod.com/~Willy_Wonka/Theatr.jpg

Special Effects: Green Screen
http://members.tripod.com/~Willy_Wonka/Theatr.jpghttp://members.tripod.com/~Willy_Wonka/Theatr.jpg


40/51


Green screenSecond green screen shotCompositing of everything

Digital Domain (from http://www.vfxhq.com/1997/titanic-picssink.html )

http://www.vfxhq.com/1997/titanic-picssink.htmlhttp://www.vfxhq.com/1997/titanic-picssink.htmlhttp://www.vfxhq.com/1997/titanic-picssink.htmlhttp://www.vfxhq.com/1997/titanic-picssink.html


41/51


Green screenCompositing of people with

ship model, sky and digital water

Digital Domain (from http://www.vfxhq.com/1997/titanic-picssink.html )

Outline
http://www.vfxhq.com/1997/titanic-picssink.htmlhttp://www.vfxhq.com/1997/titanic-picssink.htmlhttp://www.vfxhq.com/1997/titanic-picssink.htmlhttp://www.vfxhq.com/1997/titanic-picssink.html


42/51

Outline



Image Compression


43/51

age Co p ess o

Exploit redundancy Coding : some pixel values more common Interpixel: adjacent pixels often similar Psychovisual: some color differences imperceptible

Distinguish lossy and lossless methods

Image Sizes


44/51

g

1024*1024 at 24 bits uses 3 MB

Encyclopedia Britannica at 300 pixels/inch and 1bit/pixes requires 25 gigabytes (25K pages)

90 minute movie at 640x480, 24 bits per pixels,24 frames per second requires 120 gigabytes

Applications: HDTV, DVD, satellite imagetransmission, medial image processing, fax, ...

Exploiting Coding Redundancy


45/51

p g g y

Not limited to images (text, other digital info) Exploit nonuniform probabilities of symbols Entropy as measure of information content

H = -S i Prob(s i) log 2 (Prob(s i))

Low entropy non uniform probability High entropy uniform probability

If source is independent random variable need H bits

Exploiting Coding Redundancy


46/51

p g g y

Idea: More frequent symbols get shorter code strings Best with high redundancy (= low entropy)

Common algorithms Huffman coding LZW coding (gzip)

Huffman Coding


47/51

g

Codebook is precomputed and static Use probability of each symbol to assign code Map symbol to code Store codebook and code sequence

Precomputation is expensive

What is symbol for image compression?

lossless

Exploiting Interpixel Redundancy


48/51

Neighboring pixels are correlated Spatial methods for low-noise image

Run-length coding: Alternate values and run-length Good if horizontal neighbors are same

Can be 1D or 2D (e.g. used in fax standard) WWWWWWWWWWWWBWWWWWWWWWWWWBBBWWWWWWWW

WWWWWWWWWWWWWWWWBWWWWWWWWWWWWWW 12W 1B 12W 3B 24W 1B 14W

Quadtrees: Recursively subdivide until cells are constant color

Region encoding : Represent boundary curves of color-constant regions

lossless

Improving Noise Tolerance


49/51

Predictive coding: Predict next pixel based on prior ones Output difference to actual

Transform coding Exploit frequency domain Example: discrete cosine transform (DCT) Used in JPEG

lossy compression

Discrete Cosine Transform


50/51

Used for lossy compression (as in JPEG)

Subdivide image into n x n blocks (n = 8)

Apply discrete cosine transform for each block Each tile is converted to frequency space

Discrete Cosine Transform


51/51

Quantize Human eye good at seeing variations over large area

Not good at seeing the exact strength of a high frequency Greatly reducing the amount of information in the high frequency components

Use variable length coding (e g Huffman)

22 Image Processing

Documents