631S09_lec1intro

M. Wu: ENEE631 Digital Image Processing (Spring'09)

Digital Image and Video Processing –Digital Image and Video Processing –

An IntroductionAn Introduction

Spring ’09 Instructor: Min Wu

Electrical and Computer Engineering Department

University of Maryland, College Park

bb.eng.umd.edu (select ENEE631 S’09) [email protected]

ENEE631 Spring’09ENEE631 Spring’09Lecture-1 (1/26/2009)Lecture-1 (1/26/2009)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec1 – Introduction [2]

ENEE631 Logistics – Spring 2009ENEE631 Logistics – Spring 2009

Lectures– Monday and Wednesday 11am-12:15pm, CSI 2120

Assignments and Projects– Matlab will be used for many assignments; C/C++ may also be involved

in some. – Kim Lab #2107 ~ image/video related software installed for EE408G

students are encouraged to make use of them in public lab hours.

Office Hours– Dr. Min Wu ([email protected])

Wednesday 12:30 – 2:30pm @ Kim 2142, or by appointment

Regularly check the course web pagebb.eng.umd.edu


Scope of ENEE631Scope of ENEE631

First graduate course on image/video processing

Prerequisites: ENEE620 and 630, or by permission

– Not assume you have much exposure on image processing at undergraduate level

– Require and build on background in random process and DSP

Emphasis on fundamental concepts– Provide theoretical foundations on multi-dimensional signal

processing built upon pre-requisites

– Coupled with assignments and projects for hands-on experience and reinforcement of the concepts

– Follow-up courses image analysis, computer vision, pattern recognition multimedia communications and security


Textbooks and ReferencesTextbooks and References

R.C. Gonzalez and R.E. Woods: Digital Image Processing, Prentice Hall, 3rd Edition, 2008. (yellow cover)

Related technical publications (will be announced in class)

Other related textbooks

– Y. Wang, J. Ostermann, Y-Q. Zhang: Digital Video Processing and Communications, Prentice Hall, 2001.

– A.K. Jain: Fundamentals of Digital Image Processing, Prentice Hall, 1989.

– John W. Woods: Multidimensional Signal, Image, and Video Processing and Coding, Academic Press, 2006.

– A.Bovik: Handbook Of Image & Video Processing, 2nd Edition, Academic Press, 2005.


ENEE631 Course Organization ENEE631 Course Organization

Grading– Assignments and class participation 20%– Projects 45%– Exams 35%

Assignments: theoretical problems + computer components – Involves Matlab or C/C++ programming and tasks with image/video tools

to reinforce concepts– Grading is based mainly on completeness; encourage further explorations

and discussions

Projects– Put theories and principles in use and learn from doing; critical thinking

Exams– In-class mid-term exam: on basic concepts, theories, and approaches– Final exam: apply theories and principles to image/video proc tasks


ENEE631 Course PoliciesENEE631 Course Policies

No late submission will be accepted– Start early! Plan wisely and prepare for unforeseen hurdles– Inform instructor of special circumstances with documentation

Independent work vs. discussions– Write up your solutions INDIVIDUALLY– Discussions with classmates on assignments and projects are encouraged

(unless otherwise noted)

Computer codes– You should write your own codes unless otherwise stated– DO NOT COPY other students’ codes– Clearly state the code modules obtained elsewhere and consult instructor

for permission to use in your project

Academic integrity: cheating, plagiarism, fabrication of results, …


Image and Video Processing: Image and Video Processing:

An Introduction and Overview An Introduction and Overview

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


A picture is worth 1000 words.A picture is worth 1000 words.

Rich info. from visual data

Examples of images around usnatural photographic images; artistic and engineering drawingsscientific images (satellite, medical, etc.)

“Motion pictures” => videomovie, TV program; family video; surveillance and highway/ferry camera

A video is worth 1000 sentences?A video is worth 1000 sentences?

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)

http://marsrovers.jpl.nasa.gov/gallery/press/opportunity/20040125a.htmlJPL Mars’ Panorama captured by the Opportunity


Increasing Use of Images – Increasing Use of Images – A Glimpse from Encyclopedia BritannicaA Glimpse from Encyclopedia Britannica

First Edition (1768-1771)

“A dictionary of arts and sciences” by“a Society of Gentlemen in Scotland”– 3 volumes, ~ 2600 pages– illustrated with 160 copperplates

11th Edition (1911)– “last time to encapsulate ALL human

knowledge”– one picture every 4 pages

1999 Edition – 32 volumes; in CD and DVD– 73,000 articles; 30,000 photos and

illustrations Now online: http://www.britannica.com/

(From B. Liu EE488 F’06 at Princeton)


Why Do We Process Images?Why Do We Process Images?

Enhancement and restoration– Remove artifacts and scratches from an old photo/movie– Improve contrast and correct blurred images

Composition (for magazines and movies), Display, Printing …

Transmission and storage– images from oversea via Internet, or from a remote planet

Information analysis and automated recognition– Providing “human vision” to machines

Medical imaging for diagnosis and exploration

Security, forensics and rights protection– Encryption, hashing, digital watermarking, digital fingerprinting …

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


Why Digital?Why Digital?

“Exactness”

– Perfect reproduction without degradation– Perfect duplication of processing result

Convenient & powerful computer-aided processing

– Can perform sophisticated processing through computer hardware or software

– Even kindergartners can do some!

Easy storage and transmission

– 1 CD can store hundreds of family photos!– Paperless transmission of high quality photos through network

within seconds

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


Examples of Digital Image & Video ProcessingExamples of Digital Image & Video Processing

Compression

Manipulation and Restoration

– Restoration of blurred and damaged images– Noise removal and reduction– Morphing

Applications

– Visual mosaicing and virtual views– Face detection– Visible and invisible watermarking– Error concealment and resilience in video transmission

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


CompressionCompression

Color image of 600x800 pixels– Without compression

600*800 * 24 bits/pixel = 11.52K bits = 1.44M bytes

– After JPEG compression (popularly used on web)

only 89K bytes compression ratio ~ 16:1

Movie ~ Image Sequence– 720x480 per frame, 30 frames/sec,

24 bits/pixel– Raw video ~ 243M bits/sec– DVD ~ about 5M bits/sec– Compression ratio ~ 48:1 “Library of Congress” by M.Wu (600x800)

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


DenoisingDenoising

From X.Li http://www.ee.princeton.edu/~lixin/denoising.htm

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


DeblurringDeblurring

http://www.mathworks.com/access/helpdesk/help/toolbox/images/deblurr7.shtml

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


Special Effects: MorphingSpecial Effects: Morphing

Princeton CS426 face morphing exampleshttp://www.cs.princeton.edu/courses/archive/fall98/cs426/assignments/morph/morph_results.html

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


Visual MosaicingVisual Mosaicing– Stitch photos together without thread or scotch tape

R. Radke – Princeton thesis 5/2001

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


– Image enhancement, feature extractions, and statistical modeling are often important steps in computer vision tasks

See more image understanding examples by Prof. Chellappa’s research group (http://www.cfar.umd.edu/~rama/research.html)

Face DetectionFace Detection

Face detection in ’98 @ CMU CS, http://www.cs.cmu.edu/afs/cs/Web/People/har/faces.html

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


““General Illumination Correction and IGeneral Illumination Correction and Its Application to Face Normalization”, Jts Application to Face Normalization”, J. Zhu . Zhu et al, ICASSP 2003et al, ICASSP 2003


Visible Digital WatermarksVisible Digital Watermarks

from IBM Watson web page“Vatican Digital Library”

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

; 2

00

9)

Invisible Watermark Invisible Watermark

– Original, marked, and their amplified luminance difference– human visual model for imperceptibility: protect smooth areas and sharp edges


Data Hiding for Annotating Binary Line DrawingsData Hiding for Annotating Binary Line Drawings

originaloriginal marked w/ marked w/ “01/01/2000”“01/01/2000”

pixel-wise pixel-wise differencedifference

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)

Error ConcealmentError Concealment

25% blocks in a checkerboard pattern are corrupted

corrupted blocks are concealed via edge-directed interpolation

(a) original lenna image (c) concealed lenna image

(b) corrupted lenna image

Examples were generated using the source codes provided by W.Zeng.


2009 International Conf. on Image Processing (ICIP)2009 International Conf. on Image Processing (ICIP)

According to the Call-for-Paper (Cairo, Egypt, Nov. 2009)

16th in the series (since 1994) http://icip2009.org/

Research frontiers ranging from traditional image processing applications to evolving multimedia and video technologies

Areas of interest include but are not limited to:– Image/Video Coding and Transmission: Still image coding, video coding,

stereoscopic and 3-D coding, distributed source coding, . . .– Image/Video Processing: filtering, restoration, enhancement, segmentation,

video segmentation and tracking, morphological processing, stereoscopic and 3-D processing, feature extraction and analysis, interpolation and super-resolution, motion detection and estimation, . . .

– Image Formation: Biomedical imaging, remote sensing, geophysical and seismic imaging, optimal imaging, synthetic-natural hybrid image systems

– Image Scanning, Display, and Printing: Scanning, sampling, quantization and halftoning, color reproduction, image representation and rendering, …

– Image/Video Storage, Retrieval, and Authentication: Image/video databases, image/video indexing and retrieval, multimodality image/video indexing and retrieval, authentication and watermarking

– Applications: biomedical sciences, geosciences and remote sensing, . . .


So What’s a Digital Image After All?So What’s a Digital Image After All?

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

& R

.Liu

© 2

00

2)


What is an Image? What is an Image?

What we perceive as a grayscale image is a pattern of light intensity over a 2-D plane (aka “image plane”)

– Described by a nonnegative real-valued function I(x,y) of two continuous spatial coordinates on an image plane.

– I(x,y) is the intensity of the image at the point (x,y).– An image is usually defined on a bounded rectangle for processing

I: [0, a] [0, b] [0, inf )

Color image– Can be represented by three functions:

R(x,y) for red, G(x,y) for green, B(x,y) for blue.

x

y

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

07

)


Different Ways to View an Image Different Ways to View an Image

(More generally, to view a 2-D real-valued function)

Example from B. Liu – EE488 F’06 Princeton

In 3-D (x,y, z) plot with z=I(x,y);red color for high value and blue for low

Equal value contour in (x,y) plane

Intensity visualization over 2-D (x,y) plane


Sampling and QuantizationSampling and Quantization

Computer handles “discrete” data.

Sampling– Sample the value of the image at the nodes of a

regular grid on the image plane.

– A pixel (picture element) at (i, j) is the image intensity value at grid point indexed by the integer coordinate (i, j).

Quantization– Is a process of transforming a real valued sampled

image to one taking only a finite number of distinct values.

– Each sampled value in a 256-level grayscale image is represented by 8 bits.

=> Stay tuned for the theories on these in future weeks.

0 (black)

255 (white)

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


Recall: 1-D Sampling TheoremRecall: 1-D Sampling Theorem

1-D Sampling Theorem

– A 1-D signal x(t) bandlimited within [-B,B] can be uniquely determined by its samples x(nT) if s > 2B (i.e. sample fast enough).

– Using the samples x(nT), we can reconstruct x(t) by filtering the impulse version of x(nT) by an ideal low pass filter

Sampling below Nyquist rate (2B) cause Aliasing

=> Will extend sampling theorem to 2-D later in the course

Xs() with s < 2B Aliasing

s=2/T

B

Xs() with s > 2B

Perfect Reconstructable

s=2/T

B-s

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

02

)


UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)

Examples of SamplingExamples of Sampling

256x256

64x64

16x16


UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)

Examples of QuantizaionExamples of Quantizaion

8 bits / pixel

4 bits / pixel

2 bits / pixel


An Ancient Example of Digital ImageAn Ancient Example of Digital Image

An Old “Digital” Picture (from a small church in Crete Island, Greece)

=> Colored tiles as “pixels”

Slide from B. Liu – EE488 F’06 Princeton


Summary of Today’s LectureSummary of Today’s Lecture

Course organization and policies

Background and examples of digital image processing

Sampling and quantization concepts for digital image

Next time– Color and Human Visual System

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


Readings and AssignmentReadings and Assignment

Introductory sections in Matlab Image Processing Toolbox– http://www.mathworks.com/access/helpdesk/help/toolbox/images/images.shtml

Gonzalez-Wood book, Chapter 1

Bovik’s Handbook – Section 1 Introduction (see course web)

Go over mathematical preliminaries– Linear system and basics of 1-D signal processing– FT and ZT– Matrix and linear algebra– ProbabilityU

MC

P E

NE

E6

31

Slid

es

(cre

ate

d b

y M

.Wu

© 2

00

1,2

00

4,2

00

9)



Color of LightColor of Light

Perceived color depends on spectral content (wavelength composition)

– e.g., 700nm ~ red.– “spectral color”

A light with very narrow bandwidth

A light with equal energy in all visible bands appears white

“Spectrum” from http://www.physics.sfasu.edu/astro/color.html

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

& R

.Liu

© 2

00

2)


Perceptual Attributes of Color Perceptual Attributes of Color

Value of Brightness (perceived luminance)

Chrominance– Hue

specify color tone (redness, greenness, etc.)

depend on peak wavelength

– Saturation describe how pure the color is depend on the spread

(bandwidth) of light spectrum reflect how much white light is

added

RGB HSV Conversion ~ nonlinear

HSV circular cone is from online documentation of Matlab image processing toolbox

http://www.mathworks.com/access/helpdesk/help/toolbox/images/color10.shtml

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

& R

.Liu

© 2

00

2)


Questions for Today (QFT)Questions for Today (QFT)

Why “seeing yellow without yellow”?

– mix green and red light to obtain the perception of yellow, without shining a single yellow light

520nm 630nm570nm

=

“Seeing yellow” figure is from B.Liu ELE330 S’01 lecture notes @ Princeton; primary color figure is from Chapter 6 slides at Gonzalez/ Woods DIP book website

631S09_lec1intro

Documents

umcp enee631

digital image

video processing

image processing

digital image

wu amp

color image

prentice hall