Detection Workshop Exploring automatic detection ...bioacoustics.us/Ishmael-Tutorial2017/Detection-Workshop-v5-ForWe… · Detection Workshop Exploring automatic detection capabilities

Dave Mellinger, Sharon Nieukirk, Sara Heimlich, Curtis Lending,

Liz Ferguson, Tom Norris

Detection Workshop

Exploring automatic detection

capabilities with Ishmael

Contents Features slide No. Automatic Detection: review 3 - 8 Introduction to Ishmael 9,10 Open and view a sound file in Ishmael 11 Spectrogram and waveform 11 Equalization & Normalization 12 Sound Playback & Saving sound 13 Preferences files: load and save 14 Multi-channel files 15 Unknown file formats 16 Acoustic Measurements 17 Converting sound file formats 18 Filtering sound files- getting rid of noise 19 Principles of Detection: Detection and Classification- definitions and differences 20 - 24 Detection Thresholds 25 Detector: Energy Sum Detector 26 Improving the detection function: spectrogram effects 27 Detector: Spectrogram correlation 28 - 31 Detections: Manual verification 32 Detector: Matched Filter 33 Detection: Repetitive calls 34,35 Downloading detectors 36 Detector: Whistle and Moan 37 Performance Evaluation: ROC and DET curves 38 - 43 Matlab interface with Ishmael 44 Localization 45 - 50

Why?

• basic research – understanding animals

• management / mitigation

• Endangered Species Act

• Marine Mammal Protection Act

• National Environmental Policy Act

What is measured?

• population

• absolute, relative, or trend

• distribution

• daily, seasonal, year-to-year

• movement

Automatic detection: Why?

Most acoustic surveys are done by manual spectrogram scans

looking for individual calls or examining long-term spectral averages

Data volumes are now too large for manual scanning

• many groups are using autonomous acoustic instruments

• fixed moorings and mobile vehicles

• cabled arrays: CTBTO, ocean observing

systems (OOS), SOSUS

Automated methods becoming commonplace

Automatic detection: Why?

Detection is one step of a much larger process for

learning about marine (or terrestrial) life

• Establishing study/monitoring goals

• Study/monitoring design

• Data collection

• instrument preparation, deployment, operation,

recovery

• data curation: quality assurance, archiving,

dissemination

Data → Information → Knowledge → Wisdom

Automatic Detection in Context

Detection is one step of a much larger process for

learning about marine (or terrestrial) life

• Data analysis: from data to information

• data exploration

• species/call determination

• levels of analysis

• species

• call type

• other groups…

Automatic Detection in Context

Data → Information → Knowledge → Wisdom

Detection is one step of a much larger process for

learning about marine (or terrestrial) life

• Data analysis: from data to information

• data processing – automatic detection

• checking of results

• performance analysis

• summary statistics

• comparative statistics

Automatic Detection in Context

Data → Information → Knowledge → Wisdom

Detection is one step of a much larger process for

learning about marine (or terrestrial) life

• Putting information in context

• animal life history

• animal population(s)

• impacts, natural and human

• Population Consequences of Disturbance (PCOD)

Automatic Detection in Context

Data → Information → Knowledge → Wisdom

What is Ishmael? A tool for analyzing sound

• viewing

• recording

• measurement

• call detection & verification

• localization

• annotation

Introduction to Ishmael

Recording

Viewing

Automatic

recognition

Localization

& tracking Ishmael

Real-time input Sound file input

Beamforming

0101010

1001000

1010101

0111101

0101010

0001000

1010101

0101101

0101010

1001000

1010101

0111101

) ) )

Orca1.wav

Orca2.wav

Orca3.wav

• • •

Annotation log

Sound clips song note at 10:53:01

weird noise at 10:55:08

song note at 10:57:20

song note at 10:58:31

song note at 10:59:45

finished for today; restart at

11:00:00

song note at 11:00:57

song note at 11:02:10

song note at 11:03:22

song note at 11:04:37

. . . .

Measurement

Open and view a sound file

• test.wav – spectrogram and waveform

• brightness, contrast

• vertical and horizontal scaling

• spectrogram parameters

• color

Using Ishmael

Equalization / normalization

• MBay 05190920.wav

• equalization, floor, ceiling

• readjust brightness and contrast

Color

Using Ishmael

Sound playback

• MBay 05190920.wav

• select & play

• change playback rate

Saving sound

• select & save

Now you try it

• energy sum\Fin-93-001-1217.ch04.wav

Using Ishmael

Preferences files

• full load/save

• partial save

• default preferences (IshDefault.ipf)

• .ipf as text file

Using Ishmael

• Multi-channel files

• 4channel-DB-100709-090600.wav

• channels to read in

• channels to view

• channels for detection

• channels to save

Using Ishmael

Unknown file formats

• 00016945.DAT

• unknown file extensions

• can handle wide variety of files

• but not files with “records”

Using Ishmael

• Acoustic measurements

• making measurements

• choosing what to measure

• saving measurements

• noise-resistant features

• 1pAB1-Mellinger-FeatureExtractionInIshmael.pptx

• Mellinger and Bradbury 2007 in articles folder

Using Ishmael

Converting sound file formats

• Blue-wh-NEP-etp-ne-00095-1500-short.wav

• convert to .aif

Using Ishmael

Filtering

• to eliminate noise

• test.wav [500 Hz]

• decimation to change sample rate [x2]

• energy sum\Humpback-2002March12-JJacobsen@390s.wav

Using Ishmael

Detection: finding potential sounds of interest in a signal

• often operates on a continuous signal

Classification: assigning these sounds to categories

• often operates on discrete segments of sound

No firm difference

• many techniques do some of both

The focus in Ishmael is on detection (so far)

Principles of Detection

The decision criterion

• separates detections from non-detections, or one class from

another

• every detector/classifier has one or more

Principles of Detection

Detection example

• decision criterion is a threshold line

Principles of Detection

time, s

(a)

Examples of classifiers

Principles of Detection

Decision criterion:

nearness to the cluster

centers

Decision criterion:

position on the

projection line

Multi-class example

Principles of Detection

x x x x

x

x x x

o o o

o

o

o + + +

+ +

+

* *

* * * * * * * * *

+

Factor 1

Fa

cto

r 2

x x

x x

* o

o

o

o

Decision criterion: position with respect to

the class separation lines

Detection: Thresholds

• higher thresholds give fewer wrong detections but more missed

calls

• vice versa for lower thresholds

Choice of threshold depends on task

• finding rare species low threshold

• don’t miss any calls

• index of abundance for common species high threshold

• few wrong detections; accurate index

• sometimes it depends on how much time you have to check

results

Principles of Detection

Detection: Energy sum

• s010723-141142L-onewhale.wav

• spectrogram [512]

• energy sum [1.5-11 kHz]

• threshold

• edit the action – logging

• Humpback-2002March12-JJacobsen@390s.wav

• Fin-93-001-1217.ch04.wav

Using Ishmael

Detection: Improving the detection function

• Humpback-2002March12-JJacobsen@390s.wav

• Detection neighborhood [1.5 s]

• Max/min duration (@ 33 s) [0.1 – 4 s]

• Smoothing [0.5 s]

• Sharpening

• s010723-141142L-onewhale.wav

Using Ishmael

Detection: Spectrogram correlation

• what is it?

Using Ishmael

time, s

de

tec

tio

n s

co

re

*

=

fre

qu

en

cy,

Hz

Detection: Spectrogram correlation

• BlueDemo-etp-ne-00069-1046+NOISE.wav

• 2 Hz contour, [0->9, 51->48.5]

• saving calls

• time before/after [10 10] or [5 20]

• logging

• BlueDemo-etp-ne-00069-1046.wav

Using Ishmael

Detection: Manual verification

• blue whale detector, save detections as files

• BlueDemo-etp-ne-00069-1046+NOISE.wav

Using Ishmael

Detection: matched filter

• cross-correlation w/kernel

• matched filter\BlueWhale-etp-ne-00069-1046.wav

• and kernel

• natural vs. synthetic kernel

• Zc06_204a22410-24210@500s.wav

• kernel CuviersDetector-MatchedFiltKernel-v3.wav

Using Ishmael

Detection: repetitive calls

• how does it work?

• Mellinger-AsaFall05-RegularSeqDet-v2.ppt

Using Ishmael

Detection: repetitive calls

• repeating calls\cuskeel_ptown_1.wav

• energy detector [1-2 kHz]

• equalization [3 s]

• regular sequences [0.01-0.3 s, 1 s window]

• Minke-93-001-2321.ch13.wav

• MAR-Feb99-CE-disk1-airgun-datafile.510@9999s.wav

Using Ishmael

Detection: downloading detectors

• fin whale - Atlantic

• Fin-080103-235959-obs16H.wav

• minke whale – Pacific (boing)

• minke_DCL5_locT123\27Apr09_174921_026_p1.wav

Using Ishmael

Detection: Whistles and Moans

• how does it work?

• Mellinger-Whistle Detection-DCL 09 Pavia.ppt

• tonals\s2k-000629-214906-dolphin.wav

• Beluga.wav

• 27Apr09_174921_026_p1-minke.wav

Using Ishmael

How well is a detector is doing?

• Compare to human detections (“ground truth”)

• Compare to other detectors

Performance always depends on

• noise level

• more generally, environmental conditions

Performance is always evaluated on a certain data set

• might be different for other data sets, other times/dates, other

noise environments

Performance evaluation

How well is a detector is doing?

• Compare to human detections (“ground truth”)

• Compare to other detectors

Performance always depends on

• noise level

• more generally, environmental conditions

Performance is always evaluated on a certain data set

• might be different for other data sets, other times/dates, other

noise environments

Performance evaluation

Performance evaluation

Receiver Operating Characteristic (ROC) curve

Performance evaluation

False positive rate

Tru

e p

ositiv

e r

ate

Detection Error Tradeoff (DET) curve

Performance evaluation

False positive rate (%)

Fals

e n

egative r

ate

(%

)

Detection: Performance evaluation

• Minke v1.ipf

• batch run minke_DCL5_locT123, make MinkeBoingDet09.log

• MATLAB

• go to ROC_DET_Matlab_code

• detector_eval.m

Using Ishmael

Detection: MATLAB interface

• MATLAB

• cd plugins/Energy Sum, startup

• Ishmael: Stubb->Energy Sum

• GPL detector demo

Using Ishmael

Background:

• Mellinger-Signal Processing.ppt

Dolphin whistle localization

• open “dolphin whistle @ 20,20.wav”

• load array file “dolphin whistle @ 20,20.arr”

• can also look at this file in Notepad

• make a spectrogram

• select part of the whistle

• click “hyperbolic location” button

• hydrophones are red dots

• location is a white dot

Localization

Localization: what happens after you locate something?

• can send it to whale tracking software

• WhalTrak (Jay Barlow)

• WhaleTrack II (Glenn Gailey)

• WILD (Whale Identification, Logging, and Display – Chris

Kyburg)

Localization

Tracking:

WILD

Localization

Tracking: WhalTrak

Localization

Sperm whale click bearings:

• open “sperm2001-010723-153000.wav”

• Localize->Loc options, load array file “JGordonArray.arr”

• check “Show results graphically”, uncheck “intermediate results”

• make a spectrogram, set up equalization

• make a detector, set a threshold

• Detect->Detection options->Saving calls (tab)

• set “before start of call” and “after end of call” to 0

• edit the action, click “calculate phone pair bearing”

Localization

Detection: Your own dataset!

… or use a data set from the file server.

Using Ishmael