Audio Measurements Su, Amit, David, Muthu. Outline Microphone Introduction to Win CE Audio data collecting with iPAQ Audio data analysis.

Audio Measurements

Su, Amit, David, Muthu

Outline

Microphone Introduction to Win CE Audio data collecting with iPAQ Audio data analysis

Introduction to microphones What is microphone?

Microphone is a transducer -- an energy converter.

It senses acoustic energy (sound) and translates it into equivalent electrical energy.

How it works? Dynamic Microphones

Good reliability, need little

maintenance fairly good signal-to-

noise ratio Bad

no "tailored" response

How it works?

Condenser Microphones Good: high-quality perform

ance Ability to respond to transi

ent sounds extended high-frequency r

esponse weigh less smaller

Bad sensitive to mechanical no

ise Other Types of Micropho

nes Ribbon microphone Phantom Power

How to choose microphone

Microphone specifications Decibel (dB) scale

Measures how sensitive the microphone is. Frequency Response

“Bandwidth“ of microphone Multiple frequency response

“Bandwidths“ for sound coming from different directions On-axis response

Response to sound coming directly to the microphone Off-axis responses

Response to sound coming from all angles

Microphone specifications

Diffuse field responseResponse to sound coming from reflections

Polar Responsehow certain frequencies are reproduced when they enter the

microphone from a circle Equivalent noise level

noise from microphone itself (good if <15db) Sensitivity

what voltage a microphone will produce at a certain sound pressure level

SPL handling capability (Sound pressure level)Where a certain Total Harmonic Distortion (THD) occurs.Where the signal from the microphone will clip, that is the

waveforms will become squares.

Outline

Microphone Introduction to Win CE Audio data collecting with iPAQ Audio data analysis

Windows CE Architecture

Windows CE Design Principles Small Memory Modular Approach Processor Portability Win32 Compatibility Comprehensive Development Tool Support Connectivity Real Time Processing

Win32 Programming Model Utilises a large subset of the Win32 API (No Win16

support) Supports MFC, VC and VB (eMbedded)

Windows CE Architecture

OEMOEM HardwareHardwareOEMOEM HardwareHardware

Embedded ShellEmbedded ShellEmbedded ShellEmbedded Shell

ApplicationsApplicationsApplicationsApplications

WIN32 APIsWIN32 APIsCOREDLL, WINSOCK, OLE, COMMCTRL, COMMDLG, WININET, TAPICOREDLL, WINSOCK, OLE, COMMCTRL, COMMDLG, WININET, TAPI

WIN32 APIsWIN32 APIsCOREDLL, WINSOCK, OLE, COMMCTRL, COMMDLG, WININET, TAPICOREDLL, WINSOCK, OLE, COMMCTRL, COMMDLG, WININET, TAPI

Windows CE Shell ServicesWindows CE Shell ServicesWindows CE Shell ServicesWindows CE Shell Services

Remote Remote ConnectivityConnectivityRemote Remote ConnectivityConnectivity

KernelKernelLibraryLibraryKernelKernelLibraryLibrary

IrDAIrDAIrDAIrDA

GWESGWESGWESGWES Device Device ManagerManagerDevice Device

ManagerManagerFileFile

ManagerManagerFileFile

ManagerManagerTCP/IPTCP/IPTCP/IPTCP/IP

OALOALBootloaderBootloader

OALOALBootloaderBootloader DriversDriversDriversDrivers DeviceDevice

driversdriversDeviceDevice driversdrivers File driversFile driversFile driversFile drivers

MicrosoftMicrosoftMicrosoftMicrosoft OEMOEMOEMOEM ISV, OEMISV, OEMISV, OEMISV, OEM

Developer Issues

Windows CE Memory Model Protected Address Space Virtual Memory

Memory Allocation Stack Heap Virtual Memory (VirtualAlloc) Memory mapped files

Processes and Threads No process priority classes Threads with the same priority

run in a round-robin fashion Number of threads only limite

d by available memory

Reserved for system

64KB Guard

Memory Mapped Files

Process Slot 1(32MB)

Process Slot 0 (32MB)

Process slot 32 (32MB)

4GB

3GB

2GB

1GB

0GB

.

.

.Process Slot 2 (32MB)

.

.

.

Developer Issues

File System No Concept of Current Directory No Support for Overlapped I/O Support for Installable and Remote File Systems

Power Issues Porting Win32 Applications

Unicode GDI differences User interface issues – e.g. no mouse

Tool Support eMbedded Visual C++ eMbedded VB Visual Studio .NET

Outline

Microphone Introduction to Win CE Audio data collecting with iPAQ Audio data analysis

My own experience

Life cycle on data analysis Background Difficulties Achievement Demo

Clean Data

Analysis Data

Collect Data

Build Application

Audio data

What is audio data To human: something

you can hear To computer: digital

signals What is audio data

features Energy zero-crossing Spectrum ……

Where audio data being used?

Engineering Acoustics Acoustic signal processing musical sounds synthesis and composition

Physical Acoustics Ultrasonics and infrasonics Propagation of sound through the atmosphere, fluids, and fluid-f

illed materials Psychological and Physiological Acoustics

Speech Recognition and Generation Physiology and biophysics of the ear, the auditory nerve, and hi

gher neural centers Others

Acoustical Oceanography Architectural Acoustics

Data collecting procedure

Tools used in our data collecting iPAQ build in mirophone Microsoft embedded C++

How? On iPAQ

Record, compress, send

On serverReceive, unzip, concat

Difficulties

Which recorder is better? Windows build in recorder control vs. self-developed

wav recorder Why choosing self-developed wav recorder?

Guessing …

Measure accuracy Channels – one or two data stream

Mono Stereo

Bit per sample – how good each sample is 8 bit 16 bit

Sample rate – how many samples are taken each second? 8.0 kHz (telephone quality) 11.025 kHz 22.05 kHz (FM radio quality) 44.1 kHz (CD quality)

File size Channel * Bit/sample * Sample rate * sample time

Procedure

Record Prepare

Open a connection with the device using this handleAllocate a buffer for incoming data

Reading data Write to wave file

Compress/Uncompress Standard zip/unzip

Send/Receive Sockets similar to ftp

Achievements

Let us do the demo now…

Future Improvements

Better headset Better Compression

More efficient algorithm? Online zipping

Make data streaming Weakness

Each file length is limited by iPAQ memory

Total recording depends on wireless link

Your own file format What if wireless link is brok

en?

Outline

Microphone Introduction to Win CE Audio data collecting with iPAQ Audio data analysis

Data Cleaning and Analysis

What is noise? From textbook

Sound - the occurrence of an audible event Noise – nonperiodic sound

To us Sound – signal data we are interested in Noise – signal data that is useless to us

How to remove noise? Example 1- data are mixed. pick up certain people’s voice while he is talking with a group. Example 2 – data are sparse. Is there any cell phone rings during a 3 hours meeting?

Data Processing

Given voice samples, what can we get from it? Volumn Picth Spectrum …

What can we do with it?

Audio data from Telcordia

What is pattern inside?

Future Work

On-line processing Server side Client side - Fat

sensor?

Fusing network

Future Work

Data Annotation?

Applications

Location detector Scenario 1: Prof. Muthu is sleeping on the trai

n, but he does not worry about missing New Brunswick…

Smart filterScenario 2: Prof. Muthu preparing his lecture

notes on the train. And David will call him around that time. He does not want to be interrupted except that call…

A little test for fun

Given voice samples from David, Amit and Su. Can you tell them apart?

A little hint

Result

1- Amit 2 – David 3 - Su