R.V. COLLEGE OF ENGINEERING, Bangalore- 560059 (Autonomous Institution Affiliated to VTU, Belgaum) “SOUND SOURCE LOCALIZATION USING LabVIEW” PROJECT REPORT 2011-12 Submitted by 1. JAGRITI R 1RV08IT058 2. SHREE VARDHAN SARAF 1RV08IT061 Under the Guidance of Mr. HARSHA HERLE Assistant professor Department of Instrumentation Technology, RVCE In partial fulfillment for the award of degree of
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
R.V. COLLEGE OF ENGINEERING, Bangalore-560059
(Autonomous Institution Affiliated to VTU, Belgaum)
“SOUND SOURCE LOCALIZATION USING
LabVIEW”
PROJECT REPORT2011-12
Submitted by
1. JAGRITI R 1RV08IT058
2. SHREE VARDHAN SARAF 1RV08IT061
Under the Guidance of
Mr. HARSHA HERLE Assistant professor
Department of Instrumentation Technology, RVCE
In partial fulfillment for the award of degree
of
Bachelor of Engineering
in
INSTRUMENTATION TECHNOLOGY
R.V. COLLEGE OF ENGINEERING, BANGALORE – 560059(Autonomous Institution Affiliated to VTU, Belgaum)
DEPARTMENT OF INSTRUMENTATION TECHNOLOGY
CERTIFICATE
Certified that the project work titled ‘Sound source localization using LabVIEW’ is
carried out by Jagriti R (1RV08IT058) and Shree Vardhan (1RV08IT061) who are
bonafide students of R.V College of Engineering, Bangalore, in partial fulfillment for
the award of degree of Bachelor of Engineering in Instrumentation Technology of
the Visvesvaraya Technological University, Belgaum during the year 2011-2012. It is
certified that all corrections/suggestions indicated for the internal Assessment have
been incorporated in the report deposited in the departmental library. The project
report has been approved as it satisfies the academic requirements in respect of
project work prescribed by the institution for the said degree.
Signature of Guide: Signature of Head of Department: Signature of Principal
External Viva
Name of Examiners Signature with date
1
2
R.VCOLLEGE OF ENGINEERING, BANGALORE - 560059(Autonomous Institution Affiliated to VTU, Belgaum)
DEPARTMENT OF INSTRUMENTATION TECHNOLOGY
DECLARATION
We, Jagriti R (1RV08IT058) and Shree Vardhan Saraf (1RV08IT061) the
students of eighth semester B.E., Instrumentation Technology, hereby declare that
the project titled “Sound source localization using LabVIEW” has been carried out
by us and submitted in partial fulfillment for the award of degree of Bachelor of
Engineering in Instrumentation Technology. We do declare that this work is not
carried out by any other students for the award of degree in any other branch.
Place: Bangalore Names Signature
Date: 1. Jagriti R
2. Shree Vardhan
ACKNOWLEDGEMENT
The satisfaction that accompanies the successful completion of any endeavour would
be incomplete without the mention of people who made it possible and whose
constant support, encouragement and guidance has been a source of inspiration
throughout the course of this project.
We thank our internal guide Mr. Harsha Herle, Assistant Professor, Instrumentation
Technology, R.V College of Engineering for his endearing help and guidance.
We express our heart-felt gratitude to Mr. Rohit Pannikar, Manager, Applications
engineering division and Mr. Rajshekhar, Staff Applications engineer, National
Instruments India for providing a very congenial work environment and for their
expert supervision that enabled us to complete this project successfully in the given
duration.
We would like to thank Dr. Prasanna Kumar S.C., Professor and Head of
Department of Instrumentation Technology, R.V College of Engineering, Bangalore
for his encouragement and support.
We would like to thank Prof. B.S. Satyanarayana, Principal, R.V College of
Engineering for his constant support.
Finally, we thank one and all involved directly or indirectly in successful completion
of the project.
ABSTRACT
Problem of locating a sound source in space has received a growing interest. The
human auditory system uses several cues for sound source localization, including
time- and level-differences between ears, spectral information, timing analysis,
correlation analysis, and pattern matching. Similarly, a biologically inspired sound
localization system can be built by making use of an array of microphones, which are
hooked up to a computer.
Methods for determining the direction of incidence based on sound intensity, the
phase of cross-spectral functions, cross-correlation functions, and Frequency
Selection algorithm are available. Sound source localizations finds applications in
military, camera pointing in video-conferencing environments, beam former steering
for robust speech recognition systems etc.
There is no universal solution for accurate sound source localization. Depending on
the object under study and the noise problem, the most appropriate technique has to
be selected. In this project we attempt to localize a single sound source by using four
microphones. A most practical acoustic source localization scheme is based on time
delay of arrival estimation (TDOA). We implement generalized cross correlation to
find time delay of arrival between microphone pairs. TDOA estimation using
microphone arrays is to use the phase information present in signals from
microphones that are spatially separated. Phase difference between the Fourier
transformed signals to estimate the TDOA and is implemented using a 4 element
tetrahedron shaped microphone array.
Once TDOA estimation is performed, it is possible to find the position of the source
through geometrical calculations therefore deriving the source location by solving the
set of non-linear least squares equations. The experimental results showed that the
direction of the two sources was estimated with high accuracy while the range of the
sources was estimated with moderate accuracy.
\i
CONTENTS
Abstract iList of Figures iiList of Tables iii List of symbols, Acronyms and Nomenclature i
1. Chapter 1: Introduction 1 1.1 Sound localization in Biology 2 1.2 Sound localization: a signal processing view 31.3 Problem statement 41.4 Objective 41.5 Overview of the Project 5 1.6 Organization of Report 51.7 Block Diagram and Description 6
2.2.1 Types of microphone 112.3 Microphone array 132.4 Various Coherence Measures 14
3. Chapter 3: Design and methodology 153.1 Scenario 163.2 Direction of Arrival Estimation 16
3.2.1 The Geometry of the Problem 163.2.2 Microphone array structure 173.2.3 Time Delay of Arrival (TDOA) 183.2.4 Algorithm to find Time delay of Arrival 19
3.3 Distance Estimation 213.3.1 Source Localization in 2-Dimensional Space 213.3.2 Hyperbolic position location 21
3.3.2.1 General Model 223.3.2.2 Position Estimation 23
3.4 Hardware Design 243.4.1 cDAQ-9172 243.4.2 Analog input module - NI 9234 243.4.3 Digital output module – NI 9472 24
3.5 Assumptions and Limitations 25
4. Chapter 4: Implementation Overview 26 4.1 Hardware and interfacing 274.2 Overview of LabVIEW 27
4.2.1 Front panel 284.2.2 Block diagram 28
4.3 Programming using LabVIEW 11 294.3.1 Microphone signal interface using NI DAQ Assistant. 294.3.2 Threshold detection of each signal 324.3.3 Finding time delay of arrival 344.3.4 Direction and distance estimation 364.3.5 Servo control 37
4.4 System Hardware 374.4.1 Microphone 384.4.2 The microphone array 384.4.3 Data Acquisition 39
4.4.3.1 Modules 404.5 Flow Chart 42
5. Chapter 5: Results and Discussion 435.1 Experimental Setup 445.2 Experiment 1: Time delay of arrival 455.3 Experiment 2: Direction of arrival 465.4 Experiment 3: Distance estimation 47
6. Chapter 6: Conclusion and future work 496.1 Conclusion 506.2 Future work 50
7. Chapter 6: Appendix 527.1 Bibliography 537.2 Snapshots of working model 557.3 Datasheets 57
Given below (Fig. 4.9) is a snap shot of the cross correlation done. The signal
received at each microphone is cross correlated with the reference microphone
(Microphone 1 at coordinate (0,0,0))
Fig. 4.9 Generalized cross correlation
Department of Instrumentation Technology Page 36
R V College of Engineering
4.3.4 Direction and distance estimation
Using the estimated time delay of arrival, specific algorithms are implemented to
estimate the position of sound source. Fig shows the direction estimation in 2D and
3D.
Fig. 4.10 Direction estimation
In figure 4.10, 1 indicates the coordinates of the microphones 2, 3 and 4. The
coordinates along with the difference in distance in solved as a linear equation for the
unknown matrix which is the direction vector indicating the direction of the sound
source. In 3, all the three components of the direction vector are extracted to indicate
direction in 3-D. The same is plotted on a 3-D graph. In 4, only two components of
direction vector are extracted to indicate direction in 2-D. The value obtained in
radians is converted to degrees.
Department of Instrumentation Technology Page 37
1. Coordinates of all five the microphones
4. Extracting first two elements of the direction vector and finding direction in 2D
3. Extracting all the three elements of direction vector and indicating the same on 3D graph
2. Solving linear equation
R V College of Engineering
In figure Fig 4.11, using the algorithm mentioned in the design, the distance to the
sound source can also be estimated using hyperbolic position estimation. It again
employs the time delay of estimation to formulate the equations.
Fig. 4.11 Distance estimation
4.3.5 Servo control
Once the direction is found, a servo motor is used to indicate the same. The duty cycle
of the servo motor and the direction values are interpolated to specify the rotation of
the servo motor for every degree. (Fig 4.12)
Fig. 4.12 Servo control
4.4 System Hardware
Figure 4.13 shows the major components in the physical set up of our system. The
microphones are mounted on the array structure to collect the sound signals. These
signals are sent to the PC via the CompactDAQ. In the PC, the program is run on
LabVIEW which does the processing and computation to obtain of the direction and
Department of Instrumentation Technology Page 38
R V College of Engineering
distance to the sound source. We will now describe each of the components in greater
detail.
Fig. 4.13 Set up
4.4.1 Microphone
The Panasonic RPVK21 Microphone (Fig 4.3) is a dynamic type, uni-directional
microphone. The microphone features an 80 Hz- 12 kHz frequency response and 55
dB/mW sensitivity which ensures that the sound is clear. It comes with a built-in
on/off switch that is easy to operate and an O.F.C output cable that measures 3 meters
in length.
Fig 4.14 Dynamic microphone
4.4.2 The microphone array
A stand for the microphones (figure 4.4), was constructed as per specifications, and
enabled the height of the entire array be adjusted from 1.5 -2 meter. For the purposes
of this project, a baseline of 1.5 meter was used. The servo was mounted below the
microphones on the central axis.
Department of Instrumentation Technology Page 39
R V College of Engineering
The purpose of the servo motor was to indicate the direction of the sound source on
one half of the 2-D plane. i.e 0-180 degrees.
Fig. 4.15 Microphone array
The coordinates of each microphone were fixed as shown in the figure below( Fig
4.5)
Fig. 4.16 Microphone coordinates
4.4.3 Data Acquisition
The cDAQ-9172 is an eight-slot NI CompactDAQ chassis that can hold up to eight C
Series I/O modules. It is connected to the Windows host computer connected over
USB. NI CompactDAQ serves as a flexible, expandable platform to meet the needs of
any electrical or sensor measurement system.
Department of Instrumentation Technology Page 40
(50,-35,0) (100,-35,0)
(-50,-35,0)
(0,-30,35)
(0,0,0)
R V College of Engineering
By placing instrumentation close to the test subject, electrical noise can be minimized
from the surroundings. This is because digital signals, used by USB are significantly
less susceptible to electromagnetic interference. Since the NI CompactDAQ is a small
rugged package, it can be easily placed close to the unit under test.
4.4.3.1 Modules
Analog input module - In our project, of the 8 slots, we have utilized 3 slots (slot1,
slot2, slot5) as shown in fig. Slot 1 and slot 2 are occupied by two NI 9234 modules.
NI 9234 are analog input modules capable of simultaneous acquisition. The five
microphones were connected to five channels respectively use BNC connections. The
required signal conditioning is done within the modules itself. Maximum allowable
sampling rate is 51.2kHz per channel. We have set our sampling rate as half of that i.e
25.6kHz per channel as during testing, our maximum frequency component does not
exceed 1000 hz. So 25.6kHz was found to be more than sufficient as over sampling
was resulting is excess data and hence slower processing. After the signals are
received by the analog input modules, they are sent for further processing to
LabVIEW. Here the algorithms which have already been discussed are implemented.
And once the direction and distance have been found, it is displayed on the front panel
as shown in fig 4.7.
Fig. 4.17 Front panel
Department of Instrumentation Technology Page 41
R V College of Engineering
Fig. 4.18 Hardware set up
Digital output module - Once the direction of sound source is found, the same is
indicated visually using a pointer which is mounted on a servo motor (fig. 4.8)
As shown in fig. the digital module NI 9274 is placed in slot 5 of the cDAQ chassis
(as slot 5 and 6 are the counter slots). The direction of sound source is given to the
digital module which in turn sends it to a servo motor in the form of a duty cycle
input.
Fig 4.19 Indicator
Department of Instrumentation Technology Page 42
Microphones 1,2,3,4 and 5 connected to channels 0 to 3 in the first module and channel 0 in the second using standard BNC connectors.
NI 9234 is connected to the counter 0 of the cDAQ in slot 5. PWM output is given from channel 3 of the DO module to the servo motor. Channels 8 and 9 are used for giving a Vsup of +5v.
NI cDAQ 9172 NI 9234 NI 9274
Servo motor
R V College of Engineering
4.5 Flow Chart
Department of Instrumentation Technology Page 43
Signals from five microphone array
Generalized cross correlation
Pick a peak
Estimate TDOA
Calculate path differences
Estimated path differences
Position estimation
Microphone locations
Estimated source location
Dimensions of coordinate system
R V College of Engineering
Chapter - 5
RESULTS AND DISCUSSION
Chapter - 5
Department of Instrumentation Technology Page 44
R V College of Engineering
RESULTS AND DISCUSSION
Experiments were done, using the algorithms described in the previous chapter, in
order to be able to gain insight into the operation of the system. A localization error
for each scenario was measured as the difference between the true angle, calculated
from the center of the array to the primary source, and the estimated angle as
predicted by the time delays. For this, it was assumed that the source was far away,
compared to the size of the array, and that the source could therefore fall on a straight
line from the array. This assumption was made and the errors calculated for both the
azimuthal and altitudinal angles of incidence and for each time-delay estimation
routine implemented. By its definition, the altitudinal angle may vary from +90 to -
90. The azimuthal angle may vary from 0 to 180.
5.1 Experimental set up
The source localization routine was tested by sound recording experiments done in a
laboratory. We setup a fixed coordinate system in the laboratory. Four microphones
were placed at the tips of an imaginary tetrahedral, whose sides are about 40 cm long.
A fifth microphone was placed as an extended arm of one of the microphones
(Fig.5.1). The microphones were hooked up to a computer, which ran a LabVIEW
program. The program saved five of signals from the microphones. Several sound
recording experiments were done by placing a source of sound at various locations in
the laboratory.
Fig. 5.1 Array structure
Department of Instrumentation Technology Page 45
Mic 4
Mic 1
Mic 2 Mic 5Mic 3
R V College of Engineering
We take into account both correlated noise and reverberation into account when
generating our test data. By setting a threshold, we eliminate the inherent noise and
pick up the most dominant sound in the room. The setup corresponds to a 6m-7m-
2.5m room, with five microphones placed at a distance from each other, 1m from the
floor and 1m from the 6m wall (in relation to which they are centered). The sound
source is generated from different positions.
The sampling frequency is 25.6kHz, and acquisition rate is 10kHz samples i.e. every
0.4 seconds. The sound source is generated using an air gun whose frequency range
lies within 500 – 1000 Hz. Thus a 25.6kHz sampling rate was sufficient.
A number of complications limit the potential accuracy of the system. Some of these
are due to physical phenomena that can never be corrected, and others are due to
inherent errors built into the processing, due to the design of the system. As
mentioned in the introduction, complications in locating the sound source that exist
outside of perfect conditions.
5.2 Experiment 3: Time delay of arrival
By estimating the measured sharp peak created by cross correlation of microphone
pairs, the time delay of arrival can be found. Given below is a figure showing the time
delay of arrival between microphones 1, 2 and 3. It can be seen in Fig 5.2 that ‘t1’ is
the extra time taken by the sound signal to reach microphone 2 and similarly ‘t3’ is
the extra time taken to reach microphone 3. Since the microphones are placed in a co-
linear fashion, on multiplying this time delay by the speed of sound, the distance
between the microphones is obtained. We have obtained the same using generalized
cross correlation. It was found to be highly accurate with +3 cm accuracy.
Fig. 5.2 Time delay of Arrival between microphone 1, 2, 3
5.3 Experiment 2: Direction of arrival
Department of Instrumentation Technology Page 46
Sound
Mic 3 Mic 2 Mic 1
t
t+Δt1
t+Δt2
R V College of Engineering
Once the time delay is estimated, it is used in a suitable algorithm as explained in
previous chapters to find the direction of sound source. For direction in 2D, consider
the plane formed by microphones 1, 2, 3 in figure 5.1. The table (5.1) shows the
estimated source location and the direction of the source. It gives the error when
measured in 2D. To normalize the error on both sides, instead of considering the
direction of the sound source from 0-180 degrees, 0 to (+90) and 0 to (-90) is
considered on both sides. The same is plotted as a graph in Figure 5.3.
Actual direction (Deg) Estimated direction (deg) %Error
10 12 20
20 23 15
30 35 10
50 46 6
80 79 3
90 90 0
-80 -75 4
-50 -45 8
-30 -37 11
-20 -22 14
-10 -16 19
Table 5.1
0 10 30 50 80 90 -80 -50 -30 -20 -100
5
10
15
20
25
Error
Figure 5.3
Department of Instrumentation Technology Page 47
ERROR
PERCENTAGE
ACTUAL TIME DELAY
R V College of Engineering
On observation it can be seen that the direction finding is most accurate in the range
of 80 – 100 degrees. As we go towards the extremes, the accuracy falls as the
microphones are unidirectional in nature. Hence, the signals are not picked up at its
best when it comes from the side. For best results, the sound source should be located
right in front of the microphone array. With omnidirectional microphones, this
constraint could be removed. But keeping the cost and availability in consideration,
we decided on the unidirectional microphones.
5.4. Experiment 3: Distance estimation
For distance finding in 2-D, the microphone array consists of 3 microphones. We
have conducted preliminary experiments with the 3 element microphone array. The
experiments involved acquiring signals from a sound source which is triggered by a
suitable mechanism. The source is located in a plane and its location is estimated
using the planar 3 microphone array.
The source is positioned at various places in 2-D space. The table 5.2 produces the
true, estimated source locations of the sound source. As mentioned in the chapter 3
Chan Ho’s linear array optimization method is utilized for solving the nonlinear
equations.
True distance (cm) Indicated distance (cm) %Error
10 13 30
50 55 10
75 79 5.3
100 104 4
120 125 4.1
150 157 4.6
180 189 5
210 221 5.23
250 275 10
Table 5.2
Department of Instrumentation Technology Page 48
R V College of Engineering
Distance to the sound source was found in 2-D. Table 5.2 tabulates the readings
obtained. On studying the same, varying levels of accuracy can be found. Larger
percentage of error is found when the sound source is placed too close to the
microphone or when the sound source is placed beyond 2 meters. So a safe range of
0.25 to 2 meters can be set.
The reason for this discrepancy is that if the sound source is placed too close to the
microphone array, it assumes a spherical approach. And our project works on the
assumption that sound signal travels in a straight manner i.e. the spherical nature of
the sound signal is not taken into account. Secondly, if the sound source is placed far
away, the sound signal reaches the microphones in an almost parallel manner so the
small time delay of arrival is not accounted for.
Department of Instrumentation Technology Page 49
R V College of Engineering
Chapter 6
CONCLUSIONS AND FUTURE WORK
Department of Instrumentation Technology Page 50
R V College of Engineering
Chapter 6
CONCLUSIONS AND FUTURE WORK
6.1 Conclusion
In this report we present an implementation of a sound-based localization technique
and introduce the platform we used in our lab. The report summarizes the basics of
sound-based localization as discussed in the literature. The process of time delay of
arrival estimation is explained. Then, the report explains the design which includes all
the algorithms, hardware, the assumptions and limitations. The implementation of the
concept is explained in detail. Finally, a comprehensive set of experimental results are
offered.
We find that in our current hardware deployment there are still many inevitable errors
in time of delay calculation. We proposed our algorithm which uses peak-weighted
goal function that detects sound source location in real time.
6.2 Future work
There are multiple factors which contribute towards errors in the sound-based
localization implementation. Future work will address reducing the impact of these
factors. These can be identified as follows:
(ii) Different materials exhibit different reflection and absorption coefficients. It has
been observed that the material of the floor between the microphone pair and the
sound source, affects the phase as well as amplitude of the signal received.
(iii) As the distance between the microphone pair and the sound source decreases, the
DOA estimates become coarser.
(iv)The position of sources of ambient noise in the room is important. This will affect
the nature of the percentage abnormality plot causing it to become non-symmetric.
(v) Position of reflective surfaces around the experimental setup contributes towards
the fluctuations.
Department of Instrumentation Technology Page 51
R V College of Engineering
(vi) Physical parameters such as speaker width and sensitivity of the microphone
contribute towards measurement errors.
(vii) The frequency response of the microphone elements also affects the fidelity of
the captured signal.
(viii) Accuracy of experimental setup and error due to elevation of microphone and
sound source are also factors which may cause errors.
The hyperbolic position location techniques presented in this thesis provides a general
overview of the capabilities of the system. Further research is needed to evaluate the
Dominant linear array algorithm for hyperbolic position location system. If improved
TDOA’s could be measured, the source position can be estimated very accurately.
Improving the performance of the algorithm for TDOA measurements reduces the
TDOA errors. The algorithm discussed for TDOA measurements is in its simplest
form.
Experiments were performed assuming that the source is stationary until all the
microphones have finished sampling the signals. Sophisticated multi-channel
sampling devices could be used to get rid of this stationary condition. While the
accuracy of the TDOA estimate appears to be a major limiting factor in the
performance of the hyperbolic position location system, the performance of the
hyperbolic position location algorithms is equally important. Position location
algorithms which are robust against TDOA noise and are able to provide
unambiguous solution to the set of nonlinear range difference equations are desirable.
For real-time implementations of source localization, closed-form solutions or
iterative techniques with fast convergence to the solution could be used. The trade-off
between computational complexity and accuracy exist for all position location
algorithms. The trade-off analysis through performance comparison of the closed-
form and iterative algorithms can be performed.
To find time delay only the most dominant peak is considered after performing
correlation. Exploring the possibility of taking advantage of second peak with particle
filtering must be done in order to get more reported sound source location data.
Department of Instrumentation Technology Page 52
R V College of Engineering
Chapter – 7
APPENDIX
Department of Instrumentation Technology Page 53
R V College of Engineering
BIBLIOGRAPHY
[1]Byoungho Kwon KAIST, Daejeon Gyeongho Kim ; Youngjin Park “Sound Source Localization Methods with Considering of Microphone Placement in Robot Platform”Robot and Human interactive Communication, 2007. RO-MAN 2007. The 16th IEEE International Symposium on 26-29 Aug. 2007
[2]Jean-Marc Valin, Franc¸ois Michaud, Jean Rouat, Dominic L´etourneau, “Robust Sound Source Localization Using a Microphone Array on a Mobile Robot”
[3]Y.T. Chan, senior member, IEEE, and K.C Ho, Member IEEE, “A simple and Efficient Estimator for Hyperbolic location”, IEEE transaction on signal processing, Vol 2, No. 8, Aug 1994.
[4] Ralph Bucher and D. Misra “A Synthesizable VHDL Model of the Exact Solution for Three-dimensional Hyperbolic Positioning System”, Volume 15 (2002), Issue 2, Pages 507-520.
[5] Johnson, Don H, Array Signal Processing: concepts and techniques.
[6] Lorraine Green Mazerolle, Ph.D, James Frank, Ph.D. “A Field Evaluation of the Shot spotter Gunshot Location System”
[7] Wang, H., Chu, P.: “Voice Source Localization for Automatic Camera Pointing System in Videoconferencing”. Proc. of IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, Mohonk, New Paltz, NY, USA (1997)
[8] Biniyam Tesfaye Taddese, “ Sound Source Separation and Localization” Honors Thesis in Computer Science Macalester College, May 1st 2006.
[9] Alessio Brutti, Maurizio Omologo, Piergiorgio Svaizer, “Comparison Between Different Sound Source Localization Techniques Based On A Real Data Collection”, IEEE Conf. On HSCMA 2008.
[10] M. Brandstein and H. Silverman, “A practical methodology for speech localization with microphone arrays”, Technical Report, Brown University, November 13, 1996
[11] J. O. Pickels, “An Introduction to the Physiology of Hearing”, Academic Press, London, 1982.