NIH (NIDCD) initiative to develop real-time portable signal processing tools for advancing research on hearing loss compensation 1 Unless otherwise indicated the information provided is based on the author’s personal analysis and opinions relative to the NIH (NIDCD*1) initiative *1 The National Institute for Deafness and Other Communication Disorders
45
Embed
NIH (NIDCD) initiative to develop real-time portable signal … · 2019-03-09 · NIH (NIDCD) initiative to develop real-time portable signal processing tools for advancing research
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
NIH (NIDCD) initiative to develop real-time portable signal processing tools for advancing research on hearing loss compensation
1
Unless otherwise indicated the information provided is based on the author’s personal analysis and opinions relative to the NIH (NIDCD*1) initiative
*1 The National Institute for Deafness and Other Communication Disorders
Roger Miller & Amy Donahue (2016 IHCON)
“These research tools will
• lower barriers for hardware and software development,
• and facilitate translation of these advances into widespread use with hearing aids, cochlear implants, and consumer electronics devices.“
.
2
Projects funded
Six awards were given starting July 2016, and one July 2017
3
RO1 AwardSmartphone‐Based Open Research Platform for Hearing Improvement Studies Issa Panahi, Nasser Kehtarnavaz, and Linda Thibodeau University of Texas, Dallas, USA
Android and iOS systems, running C shells to:
• Use the phone as a remote microphone
• Wired and wireless version
• Use the phone itself as a hearing aid
• Wired and wireless version
• 4
RO1 Award
A Real-time, Open, Portable, and Extensible Speech Lab
Hari Gurudadri, UC San Diego
•Initial MHA implementation on Os X and Linux Computers
•Portable platform based on Snapdragon 8016 and Debian Unix.
•Single microphone Receiver-in-the-canal BTE delivery system
Caslav (Chas) Pavlovic1, Hendrik Kayser2, Paul Maanen3, Tobias Herzke3, Volker Hohmann1,3, Prakash S.R.1,4, and Reza Kassayan11 BatAndCat Corporation 2 University of Oldenburg 3 HörTech gGmbH Oldenburg 4 EarLens Corporation
Research reported in this presentation was supported by the NIDCD of the NIH. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.
The BTE Case Design supported by EarLens Corporation
We report on two closely related projects:RO1 Award R01DC015429 Volker Hohmann and Chas Pavlovic July 2016
SBIR Award R44DC016257 Chas Pavlovic July 2017
Hardware Software
Software: Open Master Hearing Aid (open MHA)
11
REQUIREMENTS
• end to end latency (fully loaded)< 10 ms • programmable in C++ • Linux, MacOS, Windows environment • hardware-independence
• High end ID and mechanical/acoustical design : Earlens case adapted for MHA purposes
• Front and back MEMS microphones • Differential amplifiers • 8 wire low-noise cable
Preliminary measurements
22
Battery Life: 6 hours
Equivalent Electrical Input Noise: MEMS mike: 26 dB SPLPower supply and circuit: 20 dB SPL (does not include the earpiece cable noise) Expected total (microphone, cable, circuit and power supply) under 28 dB SPL
openMHA software runs successfully with the following latencies: direct routing through JACK server (MHA):3.2 ms with MHA in time domain: 3.9ms with STFT and overlap-add: 5.2 ms with full processing load: under 10 ms
23
24
Libraries and a fitting example available Summer 2019
Device Control and Fitting System
• WiFi at the moment; BLE summer 2019
• Example gui: Earprint format
• Earprint: A self-fitting system (Pavlovic et al. US 8,112,166; now property of HIMPP)
Other related interests - that we may use MHA for and implement in the MHA libraries:
A. Fitting
25
Fitting Process Initial Fitting (formula):
Fine Tuning
Further Specific Environment Adjustments
Audiologist
Audiologist Self
Self
Self
Self Adjustment Difficulties
1. N dimensional space of parameters affecting the preference (N definitely more than 2).
2. Multiple local maxima and minima.
26
27
Device Control and Fitting System
• Each of the three levels require a different parameter space
• The space needs to be arranged so that there is a smooth transition from one cell to another
• Multidimensional EarPrint is possible and needed
All 3 levels: Initial, Fine Tuning, and Environment adjustment
Earprint: Used by Sound ID, SoundHawk, Starkey
28
Device Control and Fitting System
• EarPrit has the same accuracy as any of the other best psychophisical scaling procedures (CS, ME) or the direct speech recognition testing - but in 10% of the time (Ramani, Michael, Pavlovic, JASA, 2011)
• CS, ME, and the Objective Speech Recognition testing for the same accuracy take the same amount of time (Pair Comparisons efficiency decreases rapidly as the number of stimuli increases, Purdy and Pavlovic, 1992)
• FUTURE: Test time and accuracy of all this methods can be substantially increased by the use of DNN.
• It is highly likely that, as data accumulates, the fine tuning to different environments will become, with the assistance of the DNN, totally automatic. • A person walks into a bar • The Environment data sample and/or the location Geo tag
are sent to the Cloud • Adjustments calculated by a Cloud algorithm based on:
• person’s previous adjustments in similar environments • and/or adjustments done by individuals with similar
hearing profile, age, etc. • and/or adjustments typically done in this particular
environment (GPS environment tag)
29
DNN and automated fitting
Other related interest that we will further explore in the MHA B.Connectivity
• BT has become almost ubiquitous in hearing aids • current uses: phone calls; remote mics; TV, etc. • WiFi in larger wearables • possibility to connect to Cloud to access big data
storage and and super computer power
30
THE MERGER OF HEARIG AIDS AND WEARABLES:
• Wearables: devices that feature a number of functions; such as playing music, heart rate monitor, blood pressure, oxygen level, instant language translation, etc.
• Trend: hearing aids are getting many features of wearables and wearables are getting hearing aid functionality (i.e. become “hearables”)
31
Nuheara Bragi
Market Size US (annual sales):
• Hearing Aids: 8 M customers • Headphones: 400 M customers
32
Price
• Hearing aids: $2000 • Headphones: $200
33
What do the following facts suggest?
• Equally sophisticated HW • 50 times bigger headphone volume • 10 time smaller headphone price • and, legal backing for OTC sales at least in the
US
34
Over-the-Counter Hearing Aid Act of 2017
35
Plus the future:
• Wearables becoming smaller and smaller • Backed by huge volume and huge competition • Hearing aid will become a feature of
wearables
36
but “hearing aids should be:
Invisible or hide effectively as parts of desirable objects”
What is a desirable object?
What if it provides extraordinary hearing even for normal hearing people?