Multimedia Programming 13: Review and term project Departments of Digital Contents Sang Il Park.

Multimedia Programming 13:

Review and term projectDepartments of Digital

ContentsSang Il Park

중간고사 !• 10 월 15 일 월요일 오후 7 시 ~9 시• 충무관 210 호• 필기고사

Outline

• Review• Topics for the term project

What you’ve learned so far:• OpenCV• Image Processing

– Image filtering• Brightness/Contrast• Gamma• Histogram Equalization• Blur Filtering• Unsharp Filter

– Image Warping• Scaling/Rotation/Shearing• Translation• Recovering Transformation

• Image Morphing

HelloCV

#include "stdafx.h"#include <cv.h>#include <cxcore.h>#include <highgui.h>

int _tmain(int argc, _TCHAR* argv[]){

IplImage * img;img = cvLoadImage("d:\\test.jpg");

cvNamedWindow("HelloCV");cvShowImage("HelloCV", img);

cvWaitKey();

cvDestroyWindow("HelloCV");cvReleaseImage(&img);

return 0;}

• IplImage

• cvLoadImage• cvReleaseImage

• cvNamedWindow

• cvShowImage• cvDestroyWindo

w

• cvWaitKey

Image structure

• IplImage (Image Processing Library)• typedef struct _IplImage {

int nSize; /* size of iplImage struct */int ID; /* image header version */int nChannels;int alphaChannel;int depth; /* pixel depth in bits */char colorModel[4];char channelSeq[4];int dataOrder;int origin;int align; /* 4- or 8-byte align */int width;int height;struct _IplROI *roi; /* pointer to ROI if any */struct _IplImage *maskROI; /*pointer to mask ROI if any */void *imageId; /* use of the application */struct _IplTileInfo *tileInfo; /* contains information on tiling*/int imageSize; /* useful size in bytes */char *imageData; /* pointer to aligned image */int widthStep; /* size of aligned line in bytes */int BorderMode[4]; /* the top, bottom, left, and right border mode */int BorderConst[4]; /* constants for the top, bottom,left, and right border */char *imageDataOrigin; /* ptr to full, nonaligned image */

} IplImage;•

Image I/O IplImage* cvLoadImage(image_path, colorness_flag);

loads image from file, converts to color or grayscle, if need, and returns it (or returns NULL).

image format is determined by the file contents.#define CV_LOAD_IMAGE_COLOR 1 #define CV_LOAD_IMAGE_GRAYSCALE 0 #define CV_LOAD_IMAGE_UNCHANGED -1 DEFAULT

cvSaveImage(image_path, image);saves image to file, image format is determined from extension.

cvReleaseImage(image_path, image);releases memory

BMP, JPEG, PNG, TIFF, PPM/PGM formats are supported.

IplImage* img = cvLoadImage(“picture.jpeg”,-1);

if( img ) cvSaveImage( “picture.bmp”, img );

Windows

• cvNamedWindow(window_name, fixed_size_flag);creates window accessed by its name. Window handles repaint, resize events. Its position is remembered in registry:cvNamedWindow(“ViewA”);cvMoveWindow(“ViewA”,300,100);cvDestroyWindow(“ViewA”);…

• cvShowImage(window_name, image);copies the image to window buffer, then repaints it when necessary. {8u|16s|32s|32f}{C1|3|4} are supported.

only the whole window contents can be modified. Dynamic updates of parts of the window are done using operations on images, drawing functions etc.

• cvDestroyWindow(window_name);deletes the window with the given name

User Input

• int cvWaitKey( int delay=0 )waits for a pressed key. After waiting for the given delay, it proceeds. Zero delay means waiting forever until user input. – Delay in milliseconds.

Good for animating something

• Example) swapping between two images

Mouse Callback• Two things you have to do:

– Implementation• Define what you are going to do when events happen

– Setting (registration)• Let the OpenCV know which one is the callback function

void cvSetMouseCallback(window_name, yourFunction)

void yourFunction (int event, int x, int y, int flags, void *param);

Draw Line

void cvLine( IplImage, CvPoint pt1, CvPoint pt2, CvScalar color, int thickness=1);

void cvLine( IplImage, CvPoint pt1, CvPoint pt2, CvScalar color, int thickness=1);

• Draw Line from pt1 to pt2 with the given thickness

• CvPoint: a structure for storing 2D position

– Use cvPoint(x,y) for a quick usage.

struct CvPoint{

int x; // x-coordinateint y; // y-coordinate

}

struct CvPoint{

int x; // x-coordinateint y; // y-coordinate

}

Draw Rectanglevoid cvRectangle( IplImage, CvPoint pt1, CvPoint pt2, CvScalar color, int thickness=1 )

void cvRectangle( IplImage, CvPoint pt1, CvPoint pt2, CvScalar color, int thickness=1 )

• Draw a rectangle with two opposite corners pt1 and pt2

pt1

pt2

What happens if thickness is -1?

Draw Circlevoid cvCircle( IplImage, CvPoint center, int radius, CvScalar color, int thickness=1 )

void cvCircle( IplImage, CvPoint center, int radius, CvScalar color, int thickness=1 )

• Draw a circle with given center and radius

centerradius

Image Processing

• image filtering: change range of image• g(x) = h(f(x))

• image warping: change domain of

image• g(x) = f(h(x))

hf g

hf g

Alexei Efros

Image Filtering 1

• Brightness/contrast

input

ou

tpu

t

g = Af + B

g = input color valuef = output color value

A = contrast value ( 초기값 = 1)B = brightness value ( 초기값 = 0)

• Gamma

Image Filtering 2

• Histogram Equalization:Modify the image to have a well-distributed histogram

Image Filtering 3

Image Filtering 3

• Histogram Equalization Procedure:For each color channel (R, G, B)– 1. Compute the histogram

– 2. Compute the cumulative histogram

– 3. Set the maximum value as 255

– 4. Using the cumulative histogram as a mapping function

255

192

128

64

0

Image Filtering 4:

Cross-correlation filtering:

or it can be written as:

H is called the “filter,” “kernel,” or “mask.”

Image Filtering 4-1

• Blur filter: 3x3 Mean kernel

1/9 1/9 1/9

1/9 1/9 1/9

1/9 1/9 1/9

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

u -1 0 1v

-1

0

1

Image Filtering 4-2

• Blur filter: 3x3 Gaussian kernel

1 2 1

2 4 2

1 2 1

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

Image Filtering 4-3

• Median Filter: selecting the median intensity in the window.

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0

0 90 0

0 0 0

1 2 3 4 5 6 7 8 90 0 0 0 0 0 0 0 90

Median( 중간값 )

Image Filtering 5:

• Unsharp Masking

=+

blurred difference original

200 400 600 800

100

200

300

400

500

- =

original blurred difference

Image Processing

• image filtering: change range of image• g(x) = h(f(x))

• image warping: change domain of

image• g(x) = f(h(x))

hf g

hf g

Alexei Efros

Image Warping

•Homogeneous coordinates– represent coordinates in 2 dimensions with a 3-

vector

1

coords shomogeneou y

x

y

x

( 동차좌표 )

• Basic 2D transformations as 3x3 matrices

1100

0cossin

0sincos

1

'

'

y

x

y

x

1100

10

01

1

'

'

y

x

t

t

y

x

y

x

Translate

Rotate

1100

00

00

1

'

'

y

x

s

s

y

x

y

x

Scale

Image Warping

• Affine transformations: – Combinations of …

• Linear transformations, and• Translations

– Properties of affine transformations:• Origin does not necessarily map to origin• Lines map to lines• Ratios are preserved• Parallel lines remain parallel• Closed under composition• Models change of basis

wyx

fedcba

wyx

100''

( 유사변환 )

Image Warping

• Projective transformations: – Combinations of..

• Affine transformations, and• Projective warps

– Properties of projective transformations:• Origin does not necessarily map to origin• Lines map to lines• Ratios are not preserved• Parallel lines do not necessarily remain parallel• Closed under composition• Models change of basis

( 사영변환 )

Image Warping

wyx

ihgfedcba

wyx

'''

• Matrix Composition

wyx

sysx

tytx

wyx

1000000

1000cossin0sincos

1001001

'''

p’ = T(tx,ty) R() S(sx,sy) p

Sequence of composition1. First, Scaling2. Next, Rotation3. Finally, Translation

Image Warping

• Keyframe animation:Interpolation of state vector:

)1( 10 vvv

1-α α

0v1v

α

Image Warping

Image Morphing: Cross-Dissolve

Interpolate whole images:Imagehalfway = (1-t)*Image1 + t*image2

This is called cross-dissolve in film industry

But what is the images are not aligned?

Image Morphing : cross-disolve2

Align first, then cross-dissolve– Alignment using global warp – picture still valid

General Image MorphingProcedure

1. Create an intermediate shape (by interpolation)2. Warp both images towards it3. Cross-dissolve the colors in the newly warped

images