Top Banner
Chapter 1 Introduction 1 | Page
96

Project Report Audio Video Processor

Aug 27, 2014

Download

Documents

Rahul Dhiman
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Project Report Audio Video Processor

Chapter 1

Introduction

1 | P a g e

Page 2: Project Report Audio Video Processor

Chapter: 1

Introduction

1.1 Introductions

In our project we are taking input from the hex keypad connecting it to microcontroller we are

serially transmitting this data to the computer and this data is used for the video processing in

different ways. In video processing we will perform following actions:

1. Motion Detection

2. Color detection

3. Motion tracking

4. Shape detection

5. Ghost mouse

6. Object counting

7. Audio playing

8. Optical character recognition

The input is taken from hex key pad. The data from hex key pad goes parallel to the

microcontroller (ATmega16a). This data is transmitted serially to the computer. This serially

transmitted data is taken by the MATLAB. In the MATLAB this data is used for the different

type of the video processing. The type of the video processing is decided by this data. The value

of the data will tell the MATLAB which program to run.

2 | P a g e

Page 3: Project Report Audio Video Processor

For the value:

1. It will be detecting the motion in front of the camera. This will show us the output on the

screen when there is some movement in front of the camera. If there is no movement then

it will not give us the output. The data will come on the screen if and only if there is some

movement. This can be used for the smart switching and movement detection.

2. It will be detecting the color from the video input. Red color will be detected and a

bounding box will be used to point its location. Centroid of the box will show the center

point of that color region. This can easily detect the color and tell us how much color

component is present. This processing can easily be used in the industrial automation

jobs like picking the defective component from the bulk manufacturing and in the process

like detection of the faulty place in the manufactured component.

3. It will track the motion of the video input. In this processing we will take a reference

node that can be anything like nib of a pen, head of any person, the tip of the finger etc.

here we are taking the tip of the finger as the reference node. It is detected and its motion

is traced. The motion of the tip will be displayed on the screen. This traced path can be

used for the detecting the path followed by the node. This processing will help us in

keeping eyes over the fix node for example any naughty student moving in the library or

using the application for painting.

4. It will detect the shape of the video input. In this processing any reference shape can be

taken. It can be circular, rectangular etc. here we are taking the circular shape as the

reference shape. The circular shape is detected from the video input. The circular shape in

the input data will get highlighted by its boundary and its centroid will be displayed on

the screen. Through this processing the circle is detected in the video input and its

position is also displayed. This processing is used for the detection of the particular shape

3 | P a g e

Page 4: Project Report Audio Video Processor

from the input data. It can also be used for the industrial purposes in order to find the

particular component in the bulk of the raw material or in the manufactured materials.

5. In this processing the pointer control like we do in the mouse is controlled. As in the

mouse the pointer moves when we move our mouse on the mouse pad similarly the

pointer will move on the screen which we control through the moving any reference

object it can be any object like our figure, any laser beam, any object of any shape, here

we are using the red color object. Optical Flow is used for the motion detection and it is

calculated from the video input data. The pointer position is calculated and then set

accordingly. This processing can be used for the interactive interface between the

machine and the human beings. This type of processing makes it possible to have the

computers placed like TV in our houses and we are operating them without any wired

device.

6. It will be counting the objects of the video data. In this processing the objects of any

reference shape will be counted. The reference shape can any shape like circle, triangle,

squares, rectangle or can be any shape. In this processing we are taking the rectangle as

the reference shape. In this processing the shape can be filtered out from the image

captured from the video input and these shapes are counted. The images will be captured

at the rate of 30 frames per second so that we cannot recognized that it is an image

processing. The no. of the object will be shown on the screen and all the objects will be

highlighted. This processing can be used for the counting objects in the industries, for

counting the people in the seminar hall etc.

7. It is the audio processing. This is the processing the audio format data. In this processing

the audio data file is played like it happen in the widow media player. In this processing

4 | P a g e

Page 5: Project Report Audio Video Processor

the actions like media player can be performed through the direct interfacing through the

pointer.

8. It will be finding the character in the video input. In this processing the alphabet like a, b,

c, d, 1, 4, A, F etc. are recognized form the video input. The character that is recognized

is displayed on the text reader. In this processing the frames from the video input is

captured as the image. The mage which we captured is convolved to the some reference

pattern which is predefined in the programming. The convolution to the particular pattern

will produce the character corresponding to that character. The recognized character is

printed in the text editor. This processing can be used for the detecting the data written on

the moving components.

1.2 Digital Image Processing

Images are produced by a variety of physical devices, including still and video cameras,

scanners, X-ray devices, electron microscopes, radar, and ultrasound, and are used for a

variety of purposes, including entertainment, medical, business, industrial, military, civil,

security, and scientific. The interests in digital image processing stem from the

improvement of pictorial information for human interpretation and the processing of

scene data for autonomous machine perception.

Digital image analysis is a process that converts a digital image into something other than

a digital image, such as a set of measurement data or a decision. Image digitization is a

process that converts a pictorial form to numerical data. A digital image is an image f(x,

5 | P a g e

Page 6: Project Report Audio Video Processor

y) that has been discredited in both spatial coordinates and brightness (intensity). The

image is divided into small regions called picture elements or pixels.

Image digitization includes image sampling (i.e., digitization of spatial coordinates (x, y))

and gray-level quantization (i.e., brightness amplitude digitization). An image is

represented by a rectangular array of integers. The image sizes and the number of gray

levels are usually integer powers of 2. The number at each pixel represents the brightness

or darkness (generally called the intensity) of the image at that point.

In general, image processing operations can be categorized into four types:

o Pixel operations: The output at a pixel depends only on the input at that pixel,

independent of all other pixels in that image. Thresholding, a process of making

the corresponding input pixels above a certain threshold level white and others

black, is simply a pixel operation. Other examples include brightness

addition/subtraction, contrast stretching, image inverting, log, and power law.

o Local (neighborhood) operations: The output at a pixel depends on the input

values in a neighborhood of that pixel. Some examples are edge detection,

smoothing filters (e.g., the averaging filter and the median filter), and sharpening

filters (e.g., the Laplacian filter and the gradient filter). This operation can be

adaptive because results depend on the particular pixel values encountered in each

image region.

o Geometric operations: The output at a pixel depends only on the input levels at

some other pixels defined by geometric transformations. Geometric operations are

different from global operations, such that the input is only from some specific

6 | P a g e

Page 7: Project Report Audio Video Processor

pixels based on geometric transformation. They do not require the input from all

the pixels to make its transformation.

o Global operations: The output at a pixel depends on all the pixels in an image. It

may be independent of the pixel values in an image, or it may reflect statistics

calculated for all the pixels, but not a local subset of pixels. A popular distance

transformation of an image, which assigns to each object pixel the minimum

distance from it to all the background pixels, belongs to a global operation.

Other examples include histogram equalization/specification, image warping, Hough

transform, and connected components.

Nowadays, there is almost no area that is not impacted in some way by digital image

processing. Its applications include

o Remote sensing: Images acquired by satellites and other spacecrafts are useful in

tracking Earth’s resources, solar features, geographical mapping, and space image

applications.

o Image transmission and storage for business: Its applications include broadcast

television, teleconferencing, and transmission of facsimile images for office

automation, communication over computer networks, security monitoring

systems, and military communications.

o Medical processing: Its applications include X-ray, cineangiogram, transaxial

tomography, and nuclear magnetic resonance. These images may be Remote

sensing images for tracking Earth’s climate and resources used for patient

screening and monitoring or for detection of tumors or other diseases in patients.

7 | P a g e

Page 8: Project Report Audio Video Processor

o Radar, sonar, and acoustic image processing: For example, the detection and

recognition of various types of targets and the maneuvering of aircraft.

o Robot/machine vision: Its applications include the identification or description of

objects or industrial parts in 3D scenes.

1.3 ELEMENTS OF AN IMAGE PROCESSING SYSTEM

Elements of an image processing system include

o Image acquisition: A physical device that is sensitive to a band in the

electromagnetic energy spectrum can produce an electrical signal output. A

digitizer is used for converting the electrical signal output into a digital form.

Digital images can be obtained by conversion of the analog images (such as

35mmprints, slides, transparencies, or reflective art) into digital images with a

scanner, or else by directly capturing the object or scene into digital forms by

means of a digital camera or video-capturing device.

o Storage:

o Short-term storage for use during processes. One of the means of

providing short-term storage is computer memory. Another is a

specialized board, called a frame buffer.

o Online storage for relatively fast recall.

o Archival storage characterized by infrequent access. The term “archival

quality” is used to designate materials that are permanent and durable, and

therefore can be safely used for preservation purposes. The objective of

archival storage is the protection against tampering, deletion, viruses, and

disaster.

8 | P a g e

Page 9: Project Report Audio Video Processor

o Processing: Most image processing functions can be implemented in software,

running on a host computer.

o Communication: Cable modem Internet services on average promise higher levels

of bandwidth than DSL Internet services, and this bandwidth roughly translates to

communication speed. Cable Internet theoretically runs faster than DSL. Cable

technology can support approximately 30 Mbps (megabits per second) of

bandwidth, whereas most forms of DSL cannot reach 10 Mbps.

o Display: An image display device may take the form of an illuminating picture

lamp providing means by which images may be illuminated by a light source on a

selectable and removably attachable basis. Monochrome and color monitors are

the principal display devices used. Other display media include random access

cathode ray tubes (CRTs) and printing devices.

To illustrate the systematic procedures of an image processing system, we give an

example of human face identification (Adler and Schuckers, 2007). The problem domain

is the faces of people. The objective is to associate the face with the identity of the

person. The output is a person’s unique identifier (e.g., social security number).

The necessary procedures to achieve the goal of face identification could include

o Image acquisition: The face image could be acquired through a high-resolution

still digital camera and compressed to an image file.

o Preprocessing: The acquired image may be enhanced by improving contrast,

sharpness, color, and so on.

9 | P a g e

Page 10: Project Report Audio Video Processor

o Segmentation: The image may first be cropped to only the facial area. Then, the

face may be segmented into eyes, mouth, nose, chin, and so on.

o Representation and description: In this step, each of the segmented areas may be

characterized by statistical data, for example, principal components analysis,

texture, aspect ratios of eyes and nose, or the color of eyes.

o Matching recognition and interpretation: This step may involve using the

characteristics derived in the previous step to match each individually segmented

area based on specific recognition algorithms. For example, eyes may be

processed to determine, based on its features, what class of eye it is. Then, all of

these interpretations are used to create a composite description of the “ensemble,”

perhaps in the form of a feature vector for the subject.

o Knowledge base: Finally, the feature vector above may be fed to a knowledge

base of all known subjects to associate it with one of the subjects in the database,

thus returning perhaps the individual’s social security number or perhaps a

confidence score of the match.

10 | P a g e

Page 11: Project Report Audio Video Processor

Chapter 2

Image Acquisition Toolbox

11 | P a g e

Page 12: Project Report Audio Video Processor

Chapter 2

Image Acquisition Toolbox

2.1 IntroductionImage Acquisition Toolbox enables you to acquire images and video from cameras and

frame grabbers directly into MATLAB and Simulink. You can detect hardware

automatically and configure hardware properties. Advanced workflows let you trigger

acquisition while processing in-the-loop, perform background acquisition, and

synchronize sampling across several multimodal devices. With support for multiple

hardware vendors and industry standards, you can use imaging devices ranging from

inexpensive Web cameras to high-end scientific and industrial devices that meet low-

light, high-speed, and other challenging requirements.

Together, MATLAB, Image Acquisition Toolbox, and Image Processing Toolbox™

(and, optionally, Video and Image Processing Blockset) provide a complete environment

for developing customized imaging solutions. You can acquire images and video,

visualize data, develop processing algorithms and analysis techniques, and create GUIs.

The image acquisition engine enables you to acquire frames as fast as your camera and

PC can support for high speed imaging. In addition, you can use Image Acquisition

Toolbox with Simulink and Video and Image Processing Blockset to model and simulate

real-time embedded imaging systems.

Image Acquisition Toolbox simplifies the acquisition process by providing a consistent

interface across operating systems, hardware devices, and vendors. The toolbox provides

multiple ways to access hardware devices from MATLAB and Simulink: the Image

Acquisition Tool, a programmatic interface in MATLAB, and a block for Simulink. Each

workflow provides access to camera properties and controls while enabling you to solve

different types of problems with the strengths of each environment.

12 | P a g e

Page 13: Project Report Audio Video Processor

2.2 Key Featureso Support for industry standards, including DCAM, Camera Link, and GigE Vision

o Support for common OS interfaces for webcams, including Direct Show,

QuickTime, and video4linux2

o Support for a range of industrial and scientific hardware vendors

o Multiple acquisition modes and buffer management options

o Synchronization of multimodal acquisition devices with hardware triggering

o Interactive tool for rapid hardware configuration, image acquisition, and live

video previewing

o Support for C code generation in Simulink

2.3 Graphical Interface for AcquisitionThe Image Acquisition Tool is a graphical interface for working with image and video

acquisition devices and is well suited for interactive configuration of cameras. You can

browse all hardware devices available on your computer, change device settings, select a

region of interest (ROI), preview an acquisition, acquire images and video, and record

data. A preview window helps verify and optimize your acquisition parameters by

automatically reflecting any adjustments made to camera properties in the video stream.

The Image Acquisition Tool serves as a starting point in the development of automated

and custom image acquisition and processing systems.

2.3.1 Session Logging

Session logging lets you track actions performed in the tool with a history of

command-line equivalent functions.

In situations that require the same configuration and control process for multiple

trials and experiments, the session log provides the ability to load settings to a

common point and export the code to a programmatic interface in MATLAB for

further automation.

13 | P a g e

Page 14: Project Report Audio Video Processor

2.3.2 Data Logging and Export

You can log data to disk, memory, or both simultaneously with the Image

Acquisition Tool or programmaticallyat the MATLAB command line. You can

set a limit on memory usage to prevent overuse of resources inmemory-intensive

applications. Data acquired with the tool can also be exported directly to the

Image ProcessingTool in Image Processing Toolbox for greater control over

visualization. In addition, you can:

o Log each image frame or log frames at specified intervals

o Log data to disk as compressed or uncompressed AVI streams and MAT-

files

o Specify frame rate, compression techniques, and key frame rate for AVI

streams

o Extract single images from a video stream and store them in standard

formats, including JPEG 2000, BMP,JPEG, and TIFF

2.4 Image Acquisition in MATLABImage Acquisition Toolbox provides graphical tools and a programmatic interface to help

you work with image acquisition hardware in MATLAB. You can automate repetitive

tasks, create workflows combined with tasks

such as image processing, and create standalone

executable that acquire images and video with

MATLAB Compiler .The toolbox enables you

to customize the acquisition process to include

integrating image processing functionality to

identify objects, enhance imaginary.

fig 2.1 shows an image acquired by a camera.

2.4.1Connecting to Hardware

Image Acquisition Toolbox automatically detects compatible image and video

acquisition devices. Each device connection is encapsulated as an object,

14 | P a g e

Page 15: Project Report Audio Video Processor

providing an interface for configuration and acquisition. You can create multiple

connection objects for simultaneous acquisition from as many devices as your PC

and imaging hardwaresupport. Image Acquisition Toolbox can be used on

Windows®, Linux®, and Macintosh® systems, enabling you toreuse code when

connecting to the same camera in different operating systems.

2.4.2 Acquiring Image Data

Image Acquisition Toolbox supports several modes, including background

acquisition and continuous acquisition, while processing the acquired data. The

toolbox automatically buffers data into memory, handles memory and buffer

management, and enables acquisition from an ROI. The image acquisition engine

is designed to acquire imagery as fast as your camera and computer can support,

enabling analysis and processing of high-speed imaging applications.

Data can be acquired in a wide range of data types, including signed or unsigned

8-, 16-, and 32-bit integers and single- or double-precision floating point. The

toolbox supports any color space provided by the image acquisition device

including RGB, YUV, or grayscale. Raw sensor data in a Bayer pattern can be

automatically converted into RGB data.

2.4.3 Triggering and Acquisition Customizations

Image Acquisition Toolbox supports multiple trigger types to customize the

acquisition process: immediate, manual, and hardware. Initiating an acquisition

with a manual trigger

gives extensive control

over when to begin

acquisition. In some

applications, you may

want to record at a high

frame rate, perform

acquisition only when fig 2.2 shows binary image taken from the video

15 | P a g e

Page 16: Project Report Audio Video Processor

an object is detected, or when a proximity warning is issued as a trigger. In these

situations, you can write MATLAB code to create the logic to activate a manual

trigger that can be controlled by the toolbox. Hardware triggers, which are device-

specific, let you synchronize your acquisition among several multimodal devices

to an external signal. This enables you to record imagery along with other analog

and digital input sources.

2.4.4 Callbacks and Events

You can create callback functions or customized code that automatically execute

when specific events occur, suchas when acquisition starts or stops, a trigger

occurs, or a set number of frames is acquired. Callback events can beused to

process your data as it is acquired by the image acquisition engine or to automate

configuration settings as acquisition starts and stops. For example, you can

measure statistics of frames within a video stream and activate downstream

processing when a threshold is surpassed. In addition, buffer and memory options

let you control the acquisition process further and flush data when needed.

fig 2.3 shows the block diagram of image processing set up.

16 | P a g e

Page 17: Project Report Audio Video Processor

2.4.5 Image Acquisition Thread

The acquisition thread is designed to run separately from the MATLAB thread in

order to provide maximum performance. Using a toolbox function, you can

initiate an acquisition thread for each connected device, enabling high-speed

frame capture, asynchronous acquisition, and multi-device synchronization. You

can capture as high a frame rate and resolution as your camera and computer will

support. The image acquisition engine manages the data buffer and provides the

ability to access data from the buffer whenever needed.

17 | P a g e

Page 18: Project Report Audio Video Processor

Chapter 3

Image Processing Toolbox

18 | P a g e

Page 19: Project Report Audio Video Processor

Chapter 3

Image Processing Toolbox

Image Processing Toolbox provides a comprehensive set of reference-standard

algorithms and graphical tools for image processing, analysis, visualization, and

algorithm development. You can perform image enhancement, image deblurring, feature

detection, noise reduction, image segmentation, spatial transformations, and image

registration. Many functions in the toolbox are multithreaded to take advantage of multi

core and multiprocessor computers.

Image Processing Toolbox supports a diverse set of image types, including high dynamic

range, giga pixel resolution, ICC-compliant color, and tomographic images. Graphical

tools let you explore an image, examine a region of pixels, adjust the contrast, create

contours or histograms, and manipulate regions of interest (ROIs).

With the toolbox algorithms you can restore degraded images, detect and measure

features, analyze shapes and textures, and adjust the color balance of images.

3.1 Key Featureso Image enhancement, filtering, and deblurring

o Image analysis, including segmentation, morphology, feature extraction, and

measurement

o Spatial transformations and image registration

o Image transforms, including FFT, DCT, Radon, and fan-beam projection

o Workflows for processing, displaying, and navigating arbitrarily large images

o Modular interactive tools, including ROI selections, histograms, and distance

measurements

o ICC color management

o Multidimensional image processing

o Image-sequence and video display

19 | P a g e

Page 20: Project Report Audio Video Processor

o DICOM import and export

3.2 Importing and Exporting ImagesImage Processing Toolbox supports images generated by a wide range of devices,

including digital cameras, satellite and airborne sensors, medical imaging devices,

microscopes, telescopes, and other scientific instruments.

You can visualize, analyze, and process these images in many data types, including

single- and double-precision floating-point and signed and unsigned 8-, 16-, and 32-bit

integers.

There are several ways to import and export images into and out of the MATLAB®

environment for processing.

You can use Image Acquisition Toolbox™ to acquire live images from Web cameras,

frame grabbers, DCAM-compatible cameras, and other devices. Using Database

Toolbox™, you can access images stored in ODBC/JDBC-compliant databases.

MATLAB supports standard data and image formats, including JPEG, JPEG-2000, TIFF,

PNG, HDF, HDF-EOS, FITS,

Microsoft Excel, ASCII, and binary

files. It also supports the multiband

image formats BIP and BIL, as used

by LANDSAT for example. Low-

level I/O and memory mapping

functions enable you to develop

custom routines for working with

any data format.

Image Processing Toolbox supports

a number of specialized image file

formats. For medical images, it

supports the DICOM file format, fig 3.1 shows pixel values of a region of an image

including associated metadata,

20 | P a g e

Page 21: Project Report Audio Video Processor

as well as the Analyze 7.5 and Interfile formats. The toolbox can also read geospatial

images in the NITF format and high dynamic range images in the HDR format.

3.3 Displaying and Exploring ImagesImage Processing Toolbox extends MATLAB graphics to provide image display

capabilities that are highly customizable. You can create displays with multiple images in

a single window, annotate displays with text and graphics, and create specialized displays

such as histograms, profiles, and contour plots.

In addition to display functions, the toolbox provides a suite of interactive tools for

exploring images and building GUIs. You can view image information, zoom and pan

around the image, and closely examine a region of pixels.

You can interactively place and manipulate ROIs, including points, lines, rectangles,

polygons, ellipses, and freehand shapes. You can also interactively crop, adjust the

contrast, and measure distances. The suite of tools is available within Image Tool or from

individual functions that can be used to create customized GUIs.

The toolbox includes tools for displaying video and sequences in either a time-lapsed

video viewer or an image montage. Volume visualization tools in MATLAB let you

create iso-surface displays of multidimensional image data sets.

3.4 Preprocessing and Post processing ImagesImage Processing Toolbox provides reference-standard algorithms for preprocessing and

post processing tasks that solve frequent system problems, such as interfering noise, low

dynamic range, out-of-focus optics, and the difference in color representation between

input and output devices.

3.4.1 Image enhancement techniques

Image Processing Toolbox enables you to increase the signal-to-noise ratio and

accentuate image features by modifying the colors or intensities of an image. You

can:

o Perform histogram equalization

o Perform decorrelation stretching

o Remap the dynamic range

21 | P a g e

Page 22: Project Report Audio Video Processor

o Adjust the gamma value

o Perform linear, median, or adaptive filtering

The toolbox includes specialized filtering routines and a generalized

multidimensional filtering function that handles integer image types, offers

multiple boundary-padding options, and performs convolution and correlation.

Predefined filters and functions for designing and implementing your own linear

filters are also provided.

Fig 3.2 shows different parameters of a image.

3.4.2 Image deblurring algorithms

Image Processing Toolbox includes blind, Lucy-Richardson, Wiener, and

regularized filter deconvolution, as well as conversions between point spread and

22 | P a g e

Page 23: Project Report Audio Video Processor

optical transfer functions. These functions help correct blurring caused by out-of-

focus optics, movement by the camera or the subject during image capture,

atmospheric conditions, short exposure time, and other factors. All deblurring

functions work with multidimensional images.

3.4.3 Device-independent color management

Image Processing Toolbox enables you to accurately represent color

independently from input and output devices. This is useful when analyzing the

characteristics of a device, quantitatively measuring color accuracy, or developing

algorithms for several different devices. With specialized functions in the toolbox,

you can convert images between device-independent color spaces, such as RGB,

XYZ,xyY, L*a*b*, uvL, and L*ch. For more flexibility and control, the toolbox

supports profile-based color space conversions using a color management system

based on ICC version 4. For example, you can import n-dimensional ICC color

profiles, create new or modify existing ICC color profiles for specific input and

output devices, specify the rendering intent, and find all compliant profiles on

your machine.

3.4.4 Image transforms

Transforms such as FFT and DCT play a critical role in many image processing

tasks, including image enhancement, analysis, restoration, and compression.

Image Processing Toolbox provides several image transforms, including Radon

and fan-beam projections. You can reconstruct images from parallel-beam and

fan-beam projection data (common in tomography applications). Image

transforms are also available in MATLAB and Wavelet Toolbox™.

3.4.5 Image conversions

Conversion between data classes and image types are a common requirement for

imaging applications.

Image Processing Toolbox provides a variety of utilities for conversion between

data classes, including single- and double-precision floating-point and signed or

23 | P a g e

Page 24: Project Report Audio Video Processor

unsigned 8-, 16-, and 32-bit integers. The toolbox includes algorithms for

conversion between image types, including binary, gray scale, indexed color, and

true color.

Specifically for color images, the toolbox supports a variety of color spaces (such

as YIQ, HSV, and YCrCb) as well as Bayer pattern encoded and high dynamic

range images.

3.4.6 Analyzing Images

Image Processing Toolbox provides a comprehensive suite of reference-standard

algorithms and graphical tools for image analysis tasks such as statistical analysis,

feature extraction, and property measurement.

Statistical functions let you analyze the general characteristics of an image by:

o computing the mean or standard deviation

o determining the intensity values along a line segment

o displaying an image histogram

o plotting a profile of intensity values

3.4.7 Edge-detection algorithms

It lets you identify object boundaries in an image. These algorithms include the

Sobel, Prewitt, Roberts, Canny, and Laplacian of Gaussian methods. The

powerful Canny method can detect true weak edges without being "fooled" by

noise.

3.4.8 Image segmentation algorithms

It determines region boundaries in an image. You can explore many different

approaches to image segmentation, including automatic thresholding, edge-based

methods, and morphology-based methods such as the watershed transform, often

used to segment touching objects.

24 | P a g e

Page 25: Project Report Audio Video Processor

Fig.3.3 show segmented image of an aero plane.

3.4.9 Morphological operators It enables you to detect edges, enhance contrast, remove noise, segmental image

into regions, thin regions, or perform skeletonization on regions. Morphological

functions in Image Processing Toolbox include:

o Erosion and dilation

o Opening and closing

o Labeling of connected components

o Watershed segmentation

o Reconstruction

o Distance transform

Image Processing Toolbox also contains advanced image analysis functions that

let you:

o Measure the properties of a specified image region, such as the area,

center of mass, and bounding box

o Detect lines and extract line segments from an image using the Hough

transform

o Measure properties, such as surface roughness or color variation, using

texture analysis functions

25 | P a g e

Page 26: Project Report Audio Video Processor

3.4.10 Spatial Transformations

Spatial transformations modify the spatial relationships between pixels in an

image and are useful for tasks such as rotating an image, creating thumbnails,

correcting geometric distortions, and performing image registration.

Image Processing Toolbox supports common transformational operations, such as

resizing, rotating, and interactive cropping of images, as well as generalized

transformations for arbitrary-dimensional arrays.

Image registration is important in remote sensing, medical imaging, and other

applications where images must be aligned to enable quantitative analysis or

qualitative comparison. Using Image Processing Toolbox, you can interactively

select points in a pair of images and align the two images by performing a spatial

transformation, such as linear conformal, affine, projective, polynomial,

piecewise linear or local weighted mean. You can also perform image registration

using normalized 2D cross-correlation.

26 | P a g e

Page 27: Project Report Audio Video Processor

Chapter 4

ATMega16A

27 | P a g e

Page 28: Project Report Audio Video Processor

Chapter 4

ATMega16A

4.1 Features:

o High-performance, Low-power AVR® 8-bit Microcontroller

o Advanced RISC Architecture

o 131 Powerful Instructions – Most Single-clock Cycle Execution

o 32 x 8 General Purpose Working Registers

o Fully Static Operation

o Up to 16 MIPS Throughput at 16 MHz

o On-chip 2-cycle Multiplier

o High Endurance Non-volatile Memory segments

o 16K Bytes of In-System Self-programmable Flash program memory

o 512 Bytes EEPROM

o 1K Byte Internal SRAM

o Write/Erase Cycles: 10,000 Flash/100,000 EEPROM

o Data retention: 20 years at 85°C/100 years at 25°C

o Optional Boot Code Section with Independent Lock Bits

In-System Programming by On-chip Boot Program

True Read-While-Write Operation

o Programming Lock for Software Security

o JTAG (IEEE std. 1149.1 Compliant) Interface

o Boundary-scan Capabilities According to the JTAG Standard

o Extensive On-chip Debug Support

28 | P a g e

Page 29: Project Report Audio Video Processor

o Programming of Flash, EEPROM, Fuses, and Lock Bits through the JTAG

Interface

o Peripheral Features

o Two 8-bit Timer/Counters with Separate Prescalers and Compare Modes

o One 16-bit Timer/Counter with Separate Prescaler, Compare Mode, and Capture

Mode

o Real Time Counter with Separate Oscillator

o Four PWM Channels

o 8-channel, 10-bit ADC

8 Single-ended Channels

7 Differential Channels in TQFP Package Only

2 Differential Channels with Programmable Gain at 1x, 10x, or 200x

o Byte-oriented Two-wire Serial Interface

o Programmable Serial USART

o Master/Slave SPI Serial Interface

o Programmable Watchdog Timer with Separate On-chip Oscillator

o On-chip Analog Comparator

o Special Microcontroller Features

o Power-on Reset and Programmable Brown-out Detection

o Internal Calibrated RC Oscillator

o External and Internal Interrupt Sources

o Six Sleep Modes: Idle, ADC Noise Reduction, Power-save, Power-down,

Standby and Extended Standby

o I/O and Packages

29 | P a g e

Page 30: Project Report Audio Video Processor

o 32 Programmable I/O Lines

o 40-pin PDIP, 44-lead TQFP, and 44-pad QFN/MLF

o Operating Voltages

o 2.7 - 5.5V for ATmega16A

o Speed Grades

o 0 - 16 MHz for ATmega16A

o Power Consumption @ 1 MHz, 3V, and 25°C for ATmega16A

o Active: 0.6 mA

o Idle Mode: 0.2 mA

o Power-down Mode: < 1μA

4.2 Pin Diagram and Description:

30 | P a g e

Page 31: Project Report Audio Video Processor

Fig 4.1 shows pin diagram of the atmega16a.

Pins Description

VCC Digital supply voltage.

GND Ground.

Port A (PA7:PA0) Port A serves as the analog inputs to the A/D Converter. It also serves as an

8-bit bi-directional I/O port, if the A/D Converter is not used. Port pins can

provide internal pull-up resistors (selected for each bit). The Port A output

buffers have symmetrical drive characteristics with both high sink and

source capability. When pins PA0 to PA7 are used as inputs and are

externally pulled low, they will source current if the internal pull-up

resistors are activated. The Port A pins are tri-stated when a reset condition

31 | P a g e

Page 32: Project Report Audio Video Processor

becomes active, even if the clock is not running.

Port B (PB7:PB0)

Port B is an 8-bit bi-directional I/O port with internal pull-up resistors

(selected for each bit). The Port B output buffers have symmetrical drive

characteristics with both high sink and source capability. As inputs, Port B

pins that are externally pulled low will source current if the pull-up resistors

are activated. The Port B pins are tri-stated when a reset condition becomes

active, even if the clock is not running. It also serves the functions of

various special features of the ATmega16A as listed as

Port B Pins Alternate Functions

PB7 SCK (SPI Bus Serial Clock)

PB6 MISO (SPI Bus Master Input/Slave Output)

PB5 MOSI (SPI Bus Master Output/Slave Input)

PB4 SS (SPI Slave Select Input)

PB3 AIN1 (Analog Comparator Negative Input)

OC0 (Timer/Counter0 Output Compare Match Output)

PB2 AIN0 (Analog Comparator Positive Input)

INT2 (External Interrupt 2 Input)

PB1 T1 (Timer/Counter1 External Counter Input)

PB0 T0 (Timer/Counter0 External Counter Input)

XCK (USART External Clock I/O)

Port C (PC7:PC0) Port C is an 8-bit bi-directional I/O port with internal pull-up resistors

(selected for each bit). The Port C output buffers have symmetrical drive

characteristics with both high sink and source capability. As inputs, Port C

pins that are externally pulled low will source current if the pull-up resistors

are activated. The Port C pins are tri-stated when a reset condition becomes

32 | P a g e

Page 33: Project Report Audio Video Processor

active, even if the clock is not running. If the JTAG interface is enabled, the

pull-up resistors on pins PC5 (TDI), PC3 (TMS) and PC2 (TCK) will be

activated even if a reset occurs. It also serves the functions of the JTAG

interface and other special features are listed as

Port C Pins Alternate Functions

PC7 TOSC2 (Timer Oscillator Pin 2)

PC6 TOSC1 (Timer Oscillator Pin 1)

PC5 TDI (JTAG Test Data In)

PC4 TDO (JTAG Test Data Out)

PC3 TMS (JTAG Test Mode Select)

PC2 TCK (JTAG Test Clock)

PC1 SDA (Two-wire Serial Bus Data I/O Line)

PC0 SCL (Two-wire Serial Bus Clock Line)

Port D (PD7:PD0) Port D is an 8-bit bi-directional I/O port with internal pull-up resistors

(selected for each bit). The Port D output buffers have symmetrical drive

characteristics with both high sink and source capability. As inputs, Port D

pins that are externally pulled low will source current if the pull-up resistors

are activated. The Port D pins are tri-stated when a reset condition becomes

active, even if the clock is not running. It also serves the functions of

various special features of the ATmega16A are listed as

Port D Pins Alternate Functions

Port Pin Alternate Function

PD7 OC2 (Timer/Counter2 Output Compare Match Output)

PD6 ICP1 (Timer/Counter1 Input Capture Pin)

PD5 OC1A (Timer/Counter1 Output Compare A Match

33 | P a g e

Page 34: Project Report Audio Video Processor

Output)

PD4 OC1B (Timer/Counter1 Output Compare B Match Output)

PD3 INT1 (External Interrupt 1 Input)

PD2 INT0 (External Interrupt 0 Input)

PD1 TXD (USART Output Pin)

PD0 RXD (USART Input Pin)

RESET

Reset Input. A low level on this pin for longer than the minimum pulse

length will generate a reset, even if the clock is not running. The minimum

pulse length is 1.5us. Shorter pulses are not guaranteed to generate a reset.

XTAL1Input to the inverting Oscillator amplifier and input to the internal clock

operating circuit.

XTAL2 Output from the inverting Oscillator amplifier.

AVCC

AVCC is the supply voltage pin for Port A and the A/D Converter. It should

be externally connected to VCC, even if the ADC is not used. If the ADC is

used, it should be connected to VCC through a low-pass filter.

AREF AREF is the analog reference pin for the A/D Converter.

34 | P a g e

Page 35: Project Report Audio Video Processor

Chapter 5

Serial Communication

35 | P a g e

Page 36: Project Report Audio Video Processor

Chapter 5

Serial communication

Serial communication is the process of sending data one bit at a time, sequentially, over

a communication channel or computer bus. This is in contrast to parallel communication, where

several bits are sent as a whole, on a link with several parallel channels. Serial communication is

used for all long-haul communication and most computer networks, where the cost

of cable and synchronization difficulties makes parallel communication impractical

Serial data communication uses two methods:

1. Synchronous method transfers a block of data at a time

2. Asynchronous method transfers a single byte at a time

36 | P a g e

Page 37: Project Report Audio Video Processor

5.1 serial communications in AVR:

The Serial Peripheral Interface (SPI) allows high-speed synchronous data transfer between

ATmega16A and peripheral devices, or between several AVR devices. The interconnection

between Master and Slave CPUs with SPI system consists of two Shift Registers, and a Master

clock generator. The SPI Master initiates the communication cycle when pulling low the Slave

Select SS pin of the desired Slave. Master and Slave prepare the data to be sent in their respective

Shift Registers, and the Master generates the required clock pulses on the SCK line to interchange

data. Data is always shifted from Master to Slave on the Master Out – Slave In, MOSI, line, and

from Slave to Master on the Master In– Slave Out, MISO, line. After each data packet, the Master

will synchronize the Slave by pulling high the Slave Select, SS, line.

SPCR – SPI Control Register:

• Bit 7 – SPIE: SPI Interrupt Enable

This bit causes the SPI interrupt to be executed if SPIF bit in the SPSR Register is set and the

global interrupt enable bit in SREG is set.

• Bit 6 – SPE: SPI Enable

When the SPE bit is written to one, the SPI is enabled. This bit must be set to enable any SPI

operations.

• Bit 5 – DORD: Data Order

When the DORD bit is written to one, the LSB of the data word is transmitted first.

37 | P a g e

Page 38: Project Report Audio Video Processor

When the DORD bit is written to zero, the MSB of the data word is transmitted first.

• Bit 4 – MSTR: Master/Slave Select

This bit selects Master SPI mode when written to one, and Slave SPI mode when written logic

zero. If SS is configured as an input and is driven low while MSTR is set, MSTR will be cleared,

and SPIF in SPSR will become set. The user will then have to set MSTR to re-enable SPI Master

mode.

• Bit 3 – CPOL: Clock Polarity

When this bit is written to one, SCK is high when idle. When CPOL is written to zero, SCK is

low when idle.

Bit 2 – CPHA: Clock Phase

The settings of the Clock Phase bit (CPHA) determine if data is sampled on the leading (first) or

trailing (last) edge of SCK.

Bits 1, 0 – SPR1, SPR0: SPI Clock Rate Select 1 and 0

These two bits control the SCK rate of the device configured as a Master. SPR1 and SPR0 have

no effect on the Slave.

SPSR – SPI Status Register:

• Bit 7 – SPIF: SPI Interrupt Flag

When a serial transfer is complete, the SPIF Flag is set. An interrupt is generated if SPIE in

SPCR is set and global interrupts are enabled. If SS is an input and is driven low when the SPI is

in Master mode, this will also set the SPIF Flag. SPIF is cleared by hardware when executing the

corresponding interrupt handling vector. Alternatively, the SPIF bit is cleared by first reading the

SPI Status Register with SPIF set, then accessing the SPI Data Register (SPDR).

• Bit 6 – WCOL: Write Collision Flag

38 | P a g e

Page 39: Project Report Audio Video Processor

The WCOL bit is set if the SPI Data Register (SPDR) is written during a data transfer. The

WCOL bit (and the SPIF bit) are cleared by first reading the SPI Status Register with WCOL set.

• Bit 5:1 – Res: Reserved Bits

These bits are reserved bits in the ATmega16A and will always read as zero.

• Bit 0 – SPI2X: Double SPI Speed Bit

When this bit is written logic one the SPI speed (SCK Frequency) will be doubled when the SPI

is in Master mode

Chapter 6

Universal Serial Bus39 | P a g e

Page 40: Project Report Audio Video Processor

Chapter 6

Universal Serial Bus

USB is a likely solution any time you want to use a computer to communicate with device outside

the computer. The interface is suitable for one of kind and small scale design as well as mass

production standard peripheral type.

To be successful an interfacing has to please two audiences the user who want to use the

peripherals and the developer who design the hardware and write the code that communicate with

the device. USB has the feature to please both.

6.1 Benefits of USB:

1. Ease of use

2. One interface for many devices

3. Automatic configuration

40 | P a g e

Page 41: Project Report Audio Video Processor

4. No user setting

5. Free hardware source for other device.

6. Easy to connect

7. Simple cables.

8. Plug and Play

9. No power supply required.

10. Speed: USB supports three bus speeds, high speed at 480 mbps. Full speed at 12 mbps

and low speed at 1.5 mbps. Every USB capable pc supports low and full speed. High

speed was added in version 2.0 of the specification, and requires USB 2.0 capable

hardware on mother board or an expansion card.

11. Reliability: the reliability of USB results from both the hardware design and the data

transfer protocols. The hardware specifications for the USB drivers, receiver cable

eliminate most noise that could otherwise cause data error.

12. Operating system support.

6.2 USB version:

USB 1.0

o USB1.0: It was released in January 1996. Specified data rates of 1.5 Mbps (Low-

Bandwidth) and 12 Mbps (Full-Bandwidth). Does not allow for extension cables or pass-

through monitors (due to timing and power limitations). Few such devices actually made

it to market.

o USB1.1: It was release in August 1991. It fixed problems identified in 1.0, mostly

relating to hubs. Earliest revision to be widely adopted

USB 2.0:

41 | P a g e

Page 42: Project Report Audio Video Processor

USB evolved version 2.0 whose main added feature is support for faster transfer. the original

hope when the new speed was 20 times increase in speed but the studied and tests showed that his

estimation was low , in the end a 40 times increment was found to be feasible for a bus speed of

480 megabits per second. USB 2.0 was backward compatible with USB 1.1 version 2.0

peripherals can use the same connector and cable as 1.x peripheral.

6.3 Developing a Device

Designing a USB product for PCs involves both getting the peripheral up and running and

developing or obtaining PC software needed to communicate with the peripheral.

6.3.1 Elements in the Link

A USB peripheral needs all of the following:

o A controller chip with a USB interface.

o Code in the peripheral to carry out the USB communications.

o Whatever hardware and code the peripheral needs to carry out its other functions

(processing data, reading inputs, writing to outputs).

o Device-driver software on the host to enable applications to communicate with the

peripheral.

o If the peripheral isn’t a standard type supported by the operating system, the host must

have application software to enable users to access the peripheral. For standard peripheral

types such as a mouse, keyboard, or disk drive, you don’t need custom application

software, though you may want to write a test application.

6.3.2 Tools for Developing

To develop a USB peripheral, you need the following tools:

o An assembler or compiler to create the device firmware (the code that runs inside the

device’s controller chip).

42 | P a g e

Page 43: Project Report Audio Video Processor

o A device programmer or development kit that enables you to store the assembled or

compiled code in the controller’s program memory.

o A programming language and development environment on the host for writing and

debugging the host software. The host software may include a device driver or filter

driver and/or application code. To write a device driver, you’ll need Microsoft’s

Windows Driver Development Kit (DDK).

6.4 Handshaking

USB transfers use handshaking and error-checking to help ensure that data gets to its destination

as quickly as possible and without errors. Like other interfaces, USB uses status and control, or

handshaking, information to help to manage the flow of data. In hardware handshaking, dedicated

lines carry the handshaking information.

ACK

ACK (acknowledge) indicates that a host or device has received data without error. Devices must

return ACK in the handshake packets of Setup transactions when the token and data packets were

received without error.

NAK

NAK (negative acknowledge) means the device is busy or has no data to return. If the host sends

data at a time when the device is too busy to accept the data, the device returns a NAK in the

handshake packet.

STALL

The STALL handshake can have any of three meanings: unsupported control request, control

request failed, or endpoint failed.

NYET

43 | P a g e

Page 44: Project Report Audio Video Processor

Only high-speed devices use NYET, which stands for not yet. High-speed bulk and control

transfers have a protocol that enables the host to find out before sending data if a device is ready

to receive the data.

ERR

The ERR handshake is used only by high-speed hubs in complete-split transactions. ERR

indicates the device didn’t return an expected handshake in the transaction the hub is completing

with the host.

44 | P a g e

Page 45: Project Report Audio Video Processor

Chapter 7

Project

45 | P a g e

Page 46: Project Report Audio Video Processor

Chapter 7

Project

The minor project includes studying various aspects of image processing and the

applying them to process real time video input. Various MATLAB programs have been

made corresponding to various applications of video processing on a minor scale.

It includes programs for:

o Tracking an object on the basis of color

o Simple motion detection in a live video input

o Using Motion detection for a simple application like, VIRTUAL MOUSE…

The MATLAB programs have been listed for reference and their corresponding outputs

have also been shown as captured images.

46 | P a g e

Page 47: Project Report Audio Video Processor

Program 1: Color Tracking

This program acquires video input from webcam and tracks red color objects in the video

feed. It also marks the object in a box and points out the centroid of the object.

Flow chart 7.1 shows the algorithm for color tracking.

clc;

clear all;

close all;

%%

% Create video object and define its acquisition properties

imvideo = videoinput('winvideo', 1, 'YUY2_320x240');

set(imvideo, 'FramesPerTrigger', Inf);

set(imvideo, 'ReturnedColorSpace', 'rgb'); % Returning RGB colorspace

imvideo.FrameGrabInterval = 2;

47 | P a g e

Page 48: Project Report Audio Video Processor

% Start the video acquisition

start(imvideo);

%%

while (imvideo.FramesAcquired <=200) % Loop until 200 frames are grabbed

im = getsnapshot(imvideo); % Get the snapshot of the current frame

im(:,:,1) = fliplr(im(:,:,1)); % Flip the red component

im(:,:,2) = fliplr(im(:,:,2)); % Flip the green component

im(:,:,3) = fliplr(im(:,:,3)); % Flip the blue component

imred = imsubtract(im(:,:,1), rgb2gray(im)); % Extract the red pixels

imred = medfilt2(imred, [3 3]); % Use a median filter to filter out noise

imred = im2bw(imred, 0.15); % Convert the resulting gray scale image into a binary

image.

imred = bwareaopen(imred, 500); % Remove all those pixels less than 500px

bw = bwlabel(imred, 8); % Label all the connected components in the image.

stats = regionprops(bw, 'BoundingBox', 'Centroid'); % Get the properties of different

labels

imshow(im); % Show the image

hold on; % Hold the image for overwrite

for object = 1:length(stats) % This is a loop to bound the red objects in a rectangular

box.

bb = stats(object).BoundingBox;

bc = stats(object).Centroid;

rectangle('Position',bb,'EdgeColor','r','LineWidth',2)

plot(bc(1),bc(2), '-m+') % Plot the rectangle

a=text(bc(1)+15,bc(2), strcat('X: ', num2str(round(bc(1))), ' Y: ',

num2str(round(bc(2)))));

set(a, 'FontName', 'Arial', 'FontWeight', 'bold', 'FontSize', 12, 'Color', 'yellow');

end

hold off;

end% Both the loops end here.

%%

48 | P a g e

Page 49: Project Report Audio Video Processor

stop(imvideo); % Stop the video acquisition.

clear all; % Clear all variables

close all; % Close all open images

disp('That was all about Image tracking, Guess that was pretty easy :) ')

Output:

Fig7.1 shows output of the color tracking program.

49 | P a g e

Page 50: Project Report Audio Video Processor

Program 2: Motion Detection

This program takes the video input from the webcam and shows the relative motion in the

video feed. It shows only those pixels where motion occurred otherwise showing black

background.

Flow chart 7.2 shows algorithm for motion detection.

50 | P a g e

Page 51: Project Report Audio Video Processor

function testmotion()

clc;

clear all;

close all;

%%

% Define the video object

imvideo = videoinput('winvideo',1);

set(imvideo, 'FramesPerTrigger', Inf);

set(imvideo, 'Tag', 'Image Acquisition Motion Detector');

set(imvideo, 'ReturnedColorSpace', 'gray scale');

%%

imdata = get(imvideo, 'UserData');

imdata.background = getsnapshot(imvideo);

% Creating the figure

imfigure = figure('NumberTitle', 'off', 'MenuBar', 'None', 'Name',...

'Image Acquisition Motion Detector', 'Renderer',...

'OpenGL');

nbands = get(imvideo, 'NumberOfBands');

rpos = get(imvideo, 'ROIPosition');

himage = imagesc(rand(rpos(4), rpos(3), nbands));

imdata.hfigure = imfigure;

imdata.himage = himage;

ax = get(himage, 'Parent');

set(ax, 'XTick', [], 'XTickLabel', [], 'YTick', [], 'YTickLabel', []);

% Set the timer function

set(imvideo, 'TimerFcn', @chkmotion, 'TimerPeriod', 0.05);

start(imvideo);

warning offimaq:peekdata:tooManyFramesRequested

set(imvideo, 'UserData', imdata);

end

function chkmotion(imv, ~)

51 | P a g e

Page 52: Project Report Audio Video Processor

%%

% Check for if video is running

if ~isvalid(imv) || ~isrunning(imv)

return

end

imdata = get(imv, 'UserData');

imbg = imdata.background;

frame = peekdata(imv,1);

flushdata(imv);

if isempty(frame)

return

end

if ~ishandle(imdata.hfigure)

stop(imv);

disp('Motion Tracking...It''s Awesome')

return

end

imd = imabsdiff(frame, imbg);

set(imdata.himage, 'CData', imd);

imdata.background = frame;

set(imv, 'UserData', imdata);

figure(imdata.hfigure);

end

52 | P a g e

Page 53: Project Report Audio Video Processor

Output:

Fig7.2 Shows output of motion detection program.

53 | P a g e

Page 54: Project Report Audio Video Processor

Program 3 Motion Tracing

This program has been used as a display of virtual mouse using video processing.

It takes real-time video input from webcam and checks the motion of the pointing device.

As an application of this program a slideshow has been shown in the example which is

being controlled by motion detection and tracing.

Flow chart 7.3 shows the algorithm of the motion tracking program.

54 | P a g e

Page 55: Project Report Audio Video Processor

clc;

clear all;

close all;

pause on;

%%

% Define the video object

imvideo = videoinput('winvideo',1, 'YUY2_320x240');

set(imvideo, 'FramesPerTrigger', Inf);

set(imvideo, 'ReturnedColorSpace', 'gray scale');

%%

imdata = get(imvideo, 'UserData');

imdata.background = fliplr(getsnapshot(imvideo));

imdata.himage(:,:,:,1) = imread('Slide1.jpg');

imdata.himage(:,:,:,2) = imread('Slide2.jpg');

imdata.himage(:,:,:,3) = imread('Slide3.jpg');

imdata.himage(:,:,:,4) = imread('Slide4.jpg');

imdata.himage(:,:,:,5) = imread('Slide5.jpg');

imdata.himage(:,:,:,6) = imread('Slide6.jpg');

imdata.himage(:,:,:,7) = imread('Slide7.jpg');

imdata.himage(:,:,:,8) = imread('Slide8.jpg');

imdata.himage(:,:,:,9) = imread('Slide9.jpg');

imdata.himage(:,:,:,10) = imread('Slide10.jpg');

imdata.himage(:,:,:,11) = imread('Slide11.jpg');

imdata.himage(:,:,:,12) = imread('Slide12.jpg');

imdata.flag = 1;

% Set the timer function

set(imvideo, 'TimerFcn', @chkmotion, 'TimerPeriod', 0.05);

start(imvideo);

warning offimaq:peekdata:tooManyFramesRequested

55 | P a g e

Page 56: Project Report Audio Video Processor

set(imvideo, 'UserData', imdata);

function chkmotion(imv, ~)

%%

% Check for if video is running

if ~isvalid(imv) || ~isrunning(imv)

return

end

imdata = get(imv, 'UserData');

imbg = imdata.background;

imt = imdata.himage(:,:,:,imdata.flag);

frame = fliplr(getsnapshot(imv));

flushdata(imv);

if isempty(frame)

return

end

imd = imabsdiff(frame, imbg);

imdata.background = frame;

imd = im2bw(imd, 0.05);

imd = bwareaopen(imd, 200);

r = [];

c = 0;

while isempty(r) && c<(size(imd,1)-4);

c = c+1;

r = find(imd(c,:), 1, 'first');

end

r = min(find(imd(c,:), 1, 'first'), (size(imd,2)-4));

if ~isempty(r)

imt(c:c+4,r(1):r(1)+4,:) = 255;

end

56 | P a g e

Page 57: Project Report Audio Video Processor

if (c>11 && c<23 && r>267 && r<303)

imdata.flag = min(imdata.flag + 1,12);

stop(imv);

pause(1);

start(imv);

imt = imdata.himage(:,:,:,imdata.flag);

end

if (c>11 && c<23 && r>150 && r<170)

stop(imv);

close all;

disp('Motion tracing...Really cool...Awesome!!!');

return;

end

if (c>11 && c<23 && r>27 && r<63)

imdata.flag = max(imdata.flag - 1, 1);

stop(imv);

pause(1);

start(imv);

imt = imdata.himage(:,:,:,imdata.flag);

end

imshow(imt);

set(imv, 'UserData', imdata);

end

57 | P a g e

Page 58: Project Report Audio Video Processor

Output:

Fig 7.3 shows output of the motion tracking program.

58 | P a g e

Page 59: Project Report Audio Video Processor

Program 4 Virtual Mouse

Flow chart 7.4 shows the algorithm of the virtual mouse program.

59 | P a g e

Page 60: Project Report Audio Video Processor

clc;

clear all;

close all;

%% start taking the input from the camera

imvideo = videoinput('winvideo',1, 'YUY2_160x120');

set(imvideo, 'FramesPerTrigger', Inf);

set(imvideo, 'ReturnedColorSpace', 'ycbcr');

start(imvideo);

%%

hof = video.OpticalFlow('OutputValue', 'Horizontal and vertical components in complex form', ...

'ReferenceFrameDelay', 1);

r2 = 0;

c2 = 0;

while (imvideo.FramesAcquired <=500) % Loop until 500 frames are grabbed

frame = getsnapshot(imvideo);

flushdata(imvideo);

frame = fliplr(im2bw(frame(:,:,3),0.7));

% frame = im2bw(frame(:,:,3),0.7);

se = strel('disk',5);

frame = double(imopen(frame,se));

single(bwareaopen(frame,20));

imshow(frame);

of = step(hof, frame);

% compute optical flow

r1 = [];

c1 = 0;

while isempty(r1) && c1<(size(frame,1));

c1 = c1+1;

r1 = find(frame(c1,:), 1, 'first');

end

60 | P a g e

Page 61: Project Report Audio Video Processor

r1 = min(find(frame(c1,:), 1, 'first'), size(frame,2));

x = get(0, 'PointerLocation');

if ~isempty(r1)

a = (r1-r2);

b = (c2-c1);

x(1) = x(1) + a;

x(2) = x(2) + b;

set(0,'PointerLocation', x);

r2 = r1;

c2 = c1;

else

r2 = 0;

c2 = c1;

end

end

stop(imvideo);

close figure 1;

61 | P a g e

Page 62: Project Report Audio Video Processor

Program 5 Audio Player

In this program there is Graphics to the user interface. In this program a window will be displayed on the screen and according to our command the action will be perform. As it is an audio player so when we click on the play button the song will get loaded and played. Other options like volume managing option speed managing options are also there in that window.Algorithm for the program:For a graphics interface following algorithm will be required.

Flow chart 7.5 shows the algorithm for object formation in the GUI.

62 | P a g e

Page 63: Project Report Audio Video Processor

function main

N=500;

M=300;

global Fs;

xscale = 1;

xpos = 0;

Fs=44100;

% create a figure

handles.fig = figure('Units', 'Pixels', ...

'Position', [100 100 N-200 M], ...

'Menubar', 'none', ...

'Color', [1 1 1]);

handles.button1 = uicontrol('Units', 'pixels', ...

'Position', [140 100 50 50], ...

'String', 'Play', ...

'Style', 'togglebutton');

%first text box

uicontrol( 'style', 'text', 'string', 'Speed', ...

'position', [25 100 100 20]);

uicontrol( 'style', 'text', 'string', 'Title of the Song:''Beats''', ...

'position', [22 180 210 20]);

uicontrol( 'style', 'text', 'string', 'Volume', ...

'position', [240 230 50 20]);

% create two slider controls

handles.slider1 = uicontrol( 'style', 'slider', ...

'position', [10 125 N-370 20], ...

'min', 1, 'max', 99, ...

'sliderstep', [0.5 1], ...

'value', xscale, ...

'backgroundcolor', [xscale/100 xscale/100 xscale/100]);

handles.slider2 = uicontrol( 'style', 'slider', ...

63 | P a g e

Page 64: Project Report Audio Video Processor

'position', [10 230 N-280 20], ...

'min', 0, 'max', 100, ...

'sliderstep', [0.5 1], ...

'value', xpos, ...

'backgroundcolor', [xpos/100 xpos/100 xpos/100]);

%%

set(handles.slider1, 'Callback', {@slider1_callback, handles});

set(handles.slider2, 'Callback', {@slider2_callback, handles});

set(handles.button1, 'callback', {@hide_panel_callback, handles});

%%

function hide_panel_callback(hObject, event_data, handles)

if get(handles.button1, 'value')

a=wavread('Movie_0002222.wav');

x=0.5;

b=a;

sound(b,Fs);

clear a;

end

function slider1_callback(hObject, event_data, handles)

value = get(handles.slider1, 'value');

if value == 1

Fs=Fs*2;

else

Fs=Fs/2;

end

64 | P a g e

Page 65: Project Report Audio Video Processor

function slider2_callback(hObject, event_data, handles)

if value==1

b=a*2;

else

b=a/2;

set(handles.slider1, 'backgroundcolor', [xscale/100 xscale/100 xscale/100]);

end

set(handles.slider2, 'backgroundcolor', [xpos/100 xpos/100 xpos/100]);

output:

Fig. 7.4 the window of the audio player.

65 | P a g e

Page 66: Project Report Audio Video Processor

Chapter 8

Application and Further Improvements

66 | P a g e

Page 67: Project Report Audio Video Processor

Chapter 8

Application and Further Improvements

The project undertaken has large applications in today’s mechatronic world. A few could

be listed as under:

o CCTV application for security purpose

Abandoned object recognition

Number plate recognition

Tracking a person

Face recognition

o Industrial Robotics

Flaw detection

Pattern Recognition

Bar-code reader

Standardization

o Portability of operating systems

Virtual Mouse

Virtual Keyboard

Gesture Recognition

Future improvements to be made in this project could be:

o Efficient GUI

o Motion tracking using a more efficient algorithm

o Pattern Recognition

o Gesture Recognition

o Parallel Computing

67 | P a g e

Page 68: Project Report Audio Video Processor

References

68 | P a g e

Page 69: Project Report Audio Video Processor

References

Books:o RC Gonzalez, RE Woods, “Digital Image Processing” Addison-Wesely

Publishing Company 1992

o RC Gonzalez, RE Woods, SL Eddins, “Digital Image Processing Using

MATLAB”, Pearson Education Inc. 2004

o Frank Y Shih, “Image Processing & Pattern Recognition, fundamentals &

technique”, Wiley IEEE press, 2010

o Uvais Qidwai, CH Chen, “Digital Image Processing an Algorithmic approach

with MATLAB”, CRC Press, 2010

o John C. Russ , “The Image Processing Handbook”, 6th ed, CRC Press, 2011

o Muhammad Ali Mazidi, Sarmad Maimi, Sepehr Naimi, “The AVR

Microcontroller and Embedded Systems using assembly and C”, Pearson

Education Inc. 2011

o Jan Axelson, “USB Complete”,3rd ed, Penram International Publishing Pvt. Ltd,

20

Web:o http://www.mathworks.in/products/datasheets/pdf/image-processing-toolbox.pdf

o http://www.mathworks.in/products/datasheets/pdf/image-acquisition-toolbox.pdf

o www.mikroe.com/eng/downloads/get/300/mikroc-pro-for-avr-manual-v100.pdf

o http://www.mathworks.com/pl_fileexchange

o http://www.atmel.com/images/doc8154.pdf

o http://www.atmel.com/images/doc8163.pdf

o http://www.ti.com/lit/ds/symlink/max232.pdf

o http://datasheets.maxim-ic.com/en/ds/MAX220-MAX247.pdf

Software Tools:

69 | P a g e

Page 70: Project Report Audio Video Processor

o winavr.sourceforge.net

o www.mikroe.com/eng/products/view/228/mikroc-pro-for-avr

o www.atmel.com/images/AVRStudio2Setup.exe

o www.hpinfotech.ro/cvavre.zip

o www.speedy-bl.de/avr-prog-e.htm

o www.obdev.at/ftp/pub/Products/vusb/vusb-20120109.zip

MATLAB documentation:o Image Processing Toolbox

o Image Acquisition Toolbox

o Video and Image Processing Blockset

70 | P a g e