Vedio Matlab

1

Computer Vision and Video

Processing Using MATLAB

이웅재 차장,

Senior Application Engineer

2

Agenda

Introduction

Video and Image Processing Blockset

Demo: optical flow

Demo: stereo vision

Questions

3

Example Computer Vision and Video

Processing Tasks

Detect and track objects

Count objects in a scene

Stabilize camera motion

Deinterlace video frames

Create mosaicks and panoramas

Generate depth maps from stereo

image pairs

4

Common Challenges

Accessing and analyzing video data

Exploring algorithms and what-if

scenarios

Understanding the system-wide context

Rebuilding standard algorithms

Visualizing intermediate results

Testing and validating under real-world

conditions

4

5

Video and Image Processing Blockset

5

Design and simulate video and image processing

systems

Multimedia file I/O

Video display

Text and graphic overlays

Pre- and post-processing

Motion-based processing

Object detection and tracking

Feature recognition

Computer vision

6

Demo: Video Processing in MATLAB

6

Use optical flow on a video sequence

7

System Objects

7

Definition

MATLAB objects that represent time-based and data-driven

algorithms, sources, and sinks

Instantiate and configure hReader = video.MultimediaFileReader(‘viptraffic.avi’)

Execute within a loop step(hReader);

Other methods available reset

isDone

close

For related information, see the following webinar:

“Object-Oriented Programming in MATLAB”

88

Batch Processing

8

Load the entire video file and process it all at once

Source

MATLAB

Batch

Video

Algorithm

Memory

9

MATLAB Memory

Stream

Source

Stream Processing

9

Load a video frame and process it before moving on to the

next frame

Stream

Processing

10

Stream Processing in MATLAB is Hard

Need to maintain buffer

Explicit indexing

Explicit state management

myVid = mmreader(‘myvideofile.avi’);

numFrames = myVid.NumberOfFrames;

currentFrame = read(myVid,1);

numIter = 10;

opticalFlowOutput = zeros([size(currentFrame) numFrames]);

i = 1;

while i <= numFrames

prevFrame = currentFrame;

currentFrame = read(myVid,i);

flow = opticalFlow(currentFrame,prevFrame,‘horn-schunck’,...

numIter,‘magitude-squared’);

opticalFlowOutput(:,:,2:end) = opticalFlowOutput(:,:,1:end-1);

opticalFlowOutput(:,:,1) = flow;

i = i+1;

end

implay(opticalFlowOutput)

11

System Objects Make it Easier

Initialize objects

“In-the-loop” code is much simpler

Implicit states, buffering, and indexing

Video player runs in-the-loop

reader = videoMultimediaFileReader

reader.Filename = ‘myvideofile.avi’;

viewer = video.VideoPlayer

optical = video.OpticalFlow

optical.Method = ‘horn-schunck’;

optical.OutputValue = ‘Magitude-squared’;

optical.ReferenceFrameDelay = 3;

optical.MaximumIterationCount = 10;

while ~isDone(reader)

currentFrame = step(reader);

OF = step(optical, currentFrame);

step(viewer, OF);

end

12

Demo: Video Processing in MATLAB

12

Use optical flow to detect and counting moving

vehicles on a road

13

Video and Image Processing System

ObjectsAlphaBlender Deinterlacer IFFT2D MorphologicalErode

Autocorrelator2D DemosaicInterpolator ImageComplementer MorphologicalOpen

Autothresholder DeployableVideoPlayer ImageDataTypeConverter MorphologicalTopHat

BinaryFileReader EdgeDetector ImageFilter MultimediaFileReader

BinaryFileWriter FFT2D ImagePadder MultimediaFileWriter

BlobAnalysis GammaCorrector LocalMaximaFinder OpticalFlow

BlockMatcher GeometricRotator MarkerInserter PSNR

BoundaryTracer GeometricScaler Maximum Pyramid

ChromaResampler GeometricTransformer Mean ShapeInserter

ColorSpaceConverter GeometricTransformEstimator Median StandardDeviation

ConnectedComponentLabeler GeometricTranslator MedianFilter2D TemplateMatcher

ContrastAdjuster Histogram2D MedianFilter2D TextInserter

Convolver2D HistogramEqualizer Minimum Variance

CornerDetector HoughLines MorphologicalBottomHat VideoPlayer

Crosscorrelator2D HoughTransform MorphologicalClose

DCT2D IDCT2D MorphologicalDilate

14

Computer Vision Example 1:

Mosaicking and Stabilization

14

Register neighboring frames to create a large view of

the scene or stabilize camera motion

Applications:

Security

License plate recognition

Aerial surveying

Medical imaging

15

Computer Vision Example 2:

Object Detection and Tracking

15

Detect, classify, and track objects

in a scene

Applications:

Traffic monitoring

Cell counting

Lane departure

warning system

16

Computer Vision Example 3:

Stereo Vision

16

Extract 3D information from a pair of stereo images

Applications:

Obstacle avoidance

Automotive safety

Face recognition

Scene reconstruction

Part pickers

17

Stereo Vision Basics

17

Left Right

18

Stereo Vision Algorithm

18

Stereo rectification

Align images horizontally

Block matching

Find disparity map

Backprojection

Calculate 3D points

19

Video and Image Processing Blockset

Advanced Capabilities

Graphical design in Simulink

Fixed point modeling

Code generation

Targeting and verification

DSPs

FPGAs

For more information, see the following webinar:

“Image and Video Processing with DSPs and FPGAs”

20

Image Acquisition Toolbox

Live video and image

acquisition directly into

MATLAB and Simulink

Device property configuration

Live video previewing

GUI or functional interface

Support for multiple hardware

vendors

21

Why Use MATLAB for Computer Vision

and Video Processing?

System objects simplify coding and reduce errors

Algorithms for computer vision and video processing

Flexible environment enables algorithm exploration

Support for reading and writing many video file formats

Text and graphics annotations on video data

Support for cameras and frame grabbers

Embedded hardware design considerations

22

For More Informationmathworks.com/products/viprocessing/demos.html

Other examples:

People tracking

Video mosaicking

Video stabilization

Lane departure warning system

Abandoned object detection

23

For More Information

Experiment with product by downloading a trial

http://www.mathworks.com/products/viprocessing

Related demos and webinars

More detailed product information

Links to product documentation

User stories

Contact us

Sales representative

24

Questions?