Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A Neural-Network Framework for the Design of Individualised Hearing-Loss Compensation (2207.07091v3)

Published 14 Jul 2022 in eess.AS and eess.SP

Abstract: Sound processing in the human auditory system is complex and highly non-linear, whereas hearing aids (HAs) still rely on simplified descriptions of auditory processing or hearing loss to restore hearing. Even though standard HA amplification strategies succeed in restoring audibility of faint sounds, they still fall short of providing targeted treatments for complex sensorineural deficits and adverse listening conditions. These shortcomings of current HA devices demonstrate the need for advanced hearing-loss compensation strategies that can effectively leverage the non-linear character of the auditory system. Here, we propose a differentiable deep-neural-network (DNN) framework that can be used to train DNN-based HA models based on biophysical auditory-processing differences between normal-hearing and hearing-impaired systems. We investigate different loss functions to accurately compensate for impairments that include outer-hair-cell (OHC) loss and cochlear synaptopathy (CS), and evaluate the benefits of our trained DNN-based HA models for speech processing in quiet and in noise. Our results show that auditory-processing enhancement was possible for all considered hearing-loss cases, with OHC loss proving easier to compensate than CS. Several objective metrics were considered to estimate the expected speech intelligibility after processing, and these simulations hold promise in yielding improved understanding of speech-in-noise for hearing-impaired listeners who use our DNN-based HA processing. Since our framework can be tuned to the hearing-loss profiles of individual listeners, we enter an era where truly individualised and DNN-based hearing-restoration strategies can be developed and be tested experimentally.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (82)
  1. D. Byrne and H. Dillon, “The National Acoustic Laboratories’(NAL) new procedure for selecting the gain and frequency response of a hearing aid,” Ear and Hearing, vol. 7, no. 4, pp. 257–265, 1986.
  2. S. Scollie, R. Seewald, L. Cornelisse, S. Moodie, M. Bagatto, D. Laurnagaray, S. Beaulac, and J. Pumford, “The desired sensation level multistage input/output algorithm,” Trends in Amplification, vol. 9, no. 4, pp. 159–197, 2005.
  3. G. Keidser, H. Dillon, M. Flax, T. Ching, and S. Brewer, “The NAL-NL2 prescription procedure,” Audiology Research, vol. 1, no. 1, pp. 88–90, 2011.
  4. J. M. Kates and K. H. Arehart, “The hearing-aid speech perception index (HASPI),” Speech Communication, vol. 65, pp. 75–93, 2014.
  5. B. C. Moore, B. R. Glasberg, and M. A. Stone, “Development of a new method for deriving initial fittings for hearing aids with multi-channel compression: CAMEQ2-HF,” International Journal of Audiology, vol. 49, no. 3, pp. 216–227, 2010.
  6. A. Osses Vecchi, L. Varnet, L. H. Carney, T. Dau, I. C. Bruce, S. Verhulst, and P. Majdak, “A comparative study of eight human auditory models of monaural processing,” Acta Acustica, vol. 6, p. 17, 2022.
  7. K. Smeds, M. Dahlquist, C. Paludan-Müller, J. Larsson, S. Hertzman, and S. Båsjö, “Proprietary hearing aid gain prescriptions: Changes over time,” Hearing Review, vol. 22, no. 5, pp. 16–22, 2015.
  8. J. E. Marriage, D. A. Vickers, T. Baer, B. R. Glasberg, and B. C. Moore, “Comparison of different hearing aid prescriptions for children,” Ear and Hearing, vol. 39, no. 1, pp. 20–31, 2018.
  9. E. A. Lopez-Poveda, P. T. Johannesen, P. Pérez-González, J. L. Blanco, S. Kalluri, and B. Edwards, “Predictors of hearing-aid outcomes,” Trends in Hearing, vol. 21, p. 2331216517730526, 2017.
  10. N. A. Lesica, N. Mehta, J. G. Manjaly, L. Deng, B. S. Wilson, and F.-G. Zeng, “Harnessing the power of artificial intelligence to transform hearing healthcare and research,” Nature Machine Intelligence, vol. 3, no. 10, pp. 840–849, 2021.
  11. M. Slaney, R. F. Lyon, R. Garcia, B. Kemler, C. Gnegy, K. Wilson, D. Kanevsky, S. Savla, and V. G. Cerf, “Auditory measures for the next billion users,” Ear and Hearing, vol. 41, pp. 131S–139S, 2020.
  12. E. W. Healy, E. M. Johnson, M. Delfarah, and D. Wang, “A talker-independent deep learning algorithm to increase intelligibility for hearing-impaired listeners in reverberant competing talker conditions,” The Journal of the Acoustical Society of America, vol. 147, no. 6, pp. 4106–4118, 2020.
  13. Z. Tu, N. Ma, and J. Barker, “DHASP: Differentiable hearing aid speech processing,” in ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).   IEEE, 2021, pp. 296–300.
  14. A. Nagathil, F. Göbel, A. Nelus, and I. C. Bruce, “Computationally efficient DNN-based approximation of an auditory model for applications in speech processing,” in ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).   IEEE, 2021, pp. 301–305.
  15. J. Bondy, S. Becker, I. Bruce, L. Trainor, and S. Haykin, “A novel signal-processing strategy for hearing-aid design: Neurocompensation,” Signal Processing, vol. 84, no. 7, pp. 1239–1253, 2004.
  16. Z. Chen, S. Becker, J. Bondy, I. C. Bruce, and S. Haykin, “A novel model-based hearing compensation design using a gradient-free optimization method,” Neural computation, vol. 17, no. 12, pp. 2648–2671, 2005.
  17. S. Haykin, S. Becker, I. Bruce, J. Bondy, L. Trainor, and R. J. Racine, “Binaural adaptive hearing aid,” Dec. 12 2006, uS Patent 7,149,320.
  18. D. Baby, A. Van Den Broucke, and S. Verhulst, “A convolutional neural-network model of human cochlear mechanics and filter tuning for real-time applications,” Nature Machine Intelligence, vol. 3, no. 2, pp. 134–143, 2021.
  19. F. Drakopoulos, D. Baby, and S. Verhulst, “A convolutional neural-network framework for modelling auditory sensory cells and synapses,” Communications Biology, vol. 4, no. 1, p. 827, Jul 2021.
  20. S. Verhulst, A. Altoe, and V. Vasilkov, “Computational modeling of the human auditory periphery: Auditory-nerve responses, evoked potentials and hearing loss,” Hearing Research, vol. 360, pp. 55–75, 2018.
  21. F. Drakopoulos and S. Verhulst, “A differentiable optimisation framework for the design of individualised DNN-based hearing-aid strategies,” in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).   IEEE, 2022, pp. 351–355.
  22. S. G. Kujawa and M. C. Liberman, “Adding insult to injury: cochlear nerve degeneration after “temporary” noise-induced hearing loss,” Journal of Neuroscience, vol. 29, no. 45, pp. 14 077–14 085, 2009.
  23. Y. Sergeyenko, K. Lall, M. C. Liberman, and S. G. Kujawa, “Age-related cochlear synaptopathy: an early-onset contributor to auditory functional decline,” Journal of Neuroscience, vol. 33, no. 34, pp. 13 686–13 694, 2013.
  24. A. C. Furman, S. G. Kujawa, and M. C. Liberman, “Noise-induced cochlear neuropathy is selective for fibers with low spontaneous rates,” Journal of Neurophysiology, vol. 110, no. 3, pp. 577–586, 2013.
  25. A. Parthasarathy, E. L. Bartlett, and S. G. Kujawa, “Age-related changes in neural coding of envelope cues: Peripheral declines and central compensation,” Neuroscience, vol. 407, pp. 21–31, 2019.
  26. S. Van Kuyk, W. B. Kleijn, and R. C. Hendriks, “An evaluation of intrusive instrumental intelligibility metrics,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 26, no. 11, pp. 2153–2166, 2018.
  27. M. R. Schädler, A. Warzybok, and B. Kollmeier, “Objective prediction of hearing aid benefit across listener groups using machine learning: Speech recognition performance with binaural noise-reduction algorithms,” Trends in Hearing, vol. 22, p. 2331216518768954, 2018.
  28. S. Jørgensen and T. Dau, “Predicting speech intelligibility based on the signal-to-noise envelope power ratio after modulation-frequency selective processing,” The Journal of the Acoustical Society of America, vol. 130, no. 3, pp. 1475–1487, 2011.
  29. K. Yamamoto, T. Irino, S. Araki, K. Kinoshita, and T. Nakatani, “Gedi: Gammachirp envelope distortion index for predicting intelligibility of enhanced speech,” Speech Communication, vol. 123, pp. 43–58, 2020.
  30. H. Li and J. Yamagishi, “Multi-metric optimization using generative adversarial networks for near-end speech intelligibility enhancement,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 3000–3011, 2021.
  31. T. H. Falk, V. Parsa, J. F. Santos, K. Arehart, O. Hazrati, R. Huber, J. M. Kates, and S. Scollie, “Objective quality and intelligibility prediction for users of assistive listening devices: Advantages and limitations of existing tools,” IEEE signal processing magazine, vol. 32, no. 2, pp. 114–124, 2015.
  32. J. M. Kates, K. H. Arehart, M. C. Anderson, R. K. Muralimanohar, and L. O. Harvey Jr, “Using objective metrics to measure hearing-aid performance,” Ear and hearing, vol. 39, no. 6, p. 1165, 2018.
  33. S. Graetzer, J. Barker, T. J. Cox, M. Akeroyd, J. F. Culling, G. Naylor, E. Porter, R. Viveros Munoz et al., “Clarity-2021 challenges: Machine learning challenges for advancing hearing aid processing,” in Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, vol. 2.   International Speech Communication Association (ISCA), 2021, pp. 686–690.
  34. L. A. Shaheen, M. D. Valero, and M. C. Liberman, “Towards a diagnosis of cochlear neuropathy with envelope following responses,” Journal of the Association for Research in Otolaryngology, vol. 16, no. 6, pp. 727–745, 2015.
  35. P. Wu, L. Liberman, K. Bennett, V. De Gruttola, J. O’Malley, and M. Liberman, “Primary neural degeneration in the human cochlea: Evidence for hidden hearing loss in the aging ear,” Neuroscience, vol. 407, pp. 8–20, 2019.
  36. D. D. Greenwood, “A cochlear frequency-position function for several species—29 years later,” The Journal of the Acoustical Society of America, vol. 87, no. 6, pp. 2592–2605, 1990.
  37. S. Keshishzadeh, M. Garrett, and S. Verhulst, “Towards personalized auditory models: Predicting individual sensorineural hearing-loss profiles from recorded human auditory physiology,” Trends in Hearing, vol. 25, p. 2331216520988406, 2021.
  38. M. C. Liberman, “Auditory-nerve response from cats raised in a low-noise chamber,” The Journal of the Acoustical Society of America, vol. 63, no. 2, pp. 442–455, 1978.
  39. L. A. Westerman and R. L. Smith, “A diffusion model of the transient response of the cochlear inner hair cell synapse,” The Journal of the Acoustical Society of America, vol. 83, no. 6, pp. 2266–2276, 1988.
  40. A. Van Den Broucke, D. Baby, and S. Verhulst, “Hearing-impaired bio-inspired cochlear models for real-time auditory applications,” in 21st Annual Conference of the International Speech Communication Association (INTERSPEECH 2020).   International Speech Communication Association (ISCA), 2020, pp. 2842–2846.
  41. S. Keshishzadeh, M. Garrett, V. Vasilkov, and S. Verhulst, “The derived-band envelope following response and its sensitivity to sensorineural hearing deficits,” Hearing Research, vol. 392, p. 107979, 2020.
  42. B. N. Buran, G. P. McMillan, S. Keshishzadeh, S. Verhulst, and N. F. Bramhall, “Predicting synapse counts in living humans by combining computational models with auditory physiology,” The Journal of the Acoustical Society of America, vol. 151, no. 1, pp. 561–576, 2022.
  43. V. Vasilkov, M. Garrett, M. Mauermann, and S. Verhulst, “Enhancing the sensitivity of the envelope-following response for cochlear synaptopathy screening in humans: The role of stimulus envelope,” Hearing Research, vol. 400, p. 108132, 2021.
  44. T. Dau, “The importance of cochlear processing for the formation of auditory brainstem and frequency following responses,” The Journal of the Acoustical Society of America, vol. 113, no. 2, pp. 936–950, 2003.
  45. J. Bourien, Y. Tang, C. Batrel, A. Huet, M. Lenoir, S. Ladrech, G. Desmadryl, R. Nouvian, J.-L. Puel, and J. Wang, “Contribution of auditory nerve fibers to compound action potential of the auditory nerve,” Journal of Neurophysiology, vol. 112, no. 5, pp. 1025–1039, 2014.
  46. C. Füllgrabe, M. A. Stone, and B. C. Moore, “Contribution of very low amplitude-modulation rates to intelligibility in a competing-speech task,” The Journal of the Acoustical Society of America, vol. 125, no. 3, pp. 1277–1280, 2009.
  47. J. Kubanek, P. Brunner, A. Gunduz, D. Poeppel, and G. Schalk, “The tracking of speech envelope in the human cortex,” PLoS ONE, vol. 8, no. 1, p. e53398, 2013.
  48. P. E. Souza, G. Ellis, K. Marks, R. Wright, and F. Gallun, “Does the speech cue profile affect response to amplitude envelope distortion?” Journal of Speech, Language, and Hearing Research, vol. 64, no. 6, pp. 2053–2069, 2021.
  49. F. Drakopoulos, V. Vasilkov, A. Osses Vecchi, T. Wartenberg, and S. Verhulst, “Model-based hearing-enhancement strategies for cochlear synaptopathy pathologies,” Hearing Research, vol. 424, p. 108569, 2022.
  50. J. S. Garofolo, L. F. Lamel, W. M. Fisher, J. G. Fiscus, and D. S. Pallett, “DARPA TIMIT acoustic-phonetic continous speech corpus CD-ROM. NIST speech disc 1-1.1,” NASA STI/Recon technical report n, vol. 93, p. 27403, 1993.
  51. D. Baby and S. Verhulst, “Sergan: Speech enhancement using relativistic generative adversarial networks with gradient penalty,” in ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).   IEEE, 2019, pp. 106–110.
  52. F. Drakopoulos, D. Baby, and S. Verhulst, “Real-time audio processing on a Raspberry Pi using deep neural networks,” in 23rd International Congress on Acoustics (ICA 2019).   Deutsche Gesellschaft für Akustik, 2019, pp. 2827–2834.
  53. M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard et al., “Tensorflow: A system for large-scale machine learning,” in 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), 2016, pp. 265–283.
  54. F. Chollet et al., “Keras: The python deep learning library,” Astrophysics Source Code Library, pp. ascl–1806, 2018.
  55. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.
  56. L. M. Viana, J. T. O’Malley, B. J. Burgess, D. D. Jones, C. A. Oliveira, F. Santos, S. N. Merchant, L. D. Liberman, and M. C. Liberman, “Cochlear neuropathy in human presbycusis: Confocal analysis of hidden hearing loss in post-mortem tissue,” Hearing Research, vol. 327, pp. 78–88, 2015.
  57. H. Luts, S. Jansen, W. Dreschler, and J. Wouters, “Development and normative data for the Flemish/Dutch Matrix test,” 2014.
  58. T. Vande Maele, S. Keshishzadeh, N. D. Poortere, I. Dhooge, H. Keppler, and S. Verhulst, “The variability in potential biomarkers for cochlear synaptopathy after recreational noise exposure,” Journal of Speech, Language, and Hearing Research, vol. 64, no. 12, pp. 4964–4981, 2021.
  59. A. W. Rix, J. G. Beerends, M. P. Hollier, and A. P. Hekstra, “Perceptual evaluation of speech quality (PESQ)-a new method for speech quality assessment of telephone networks and codecs,” in 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No. 01CH37221), vol. 2.   IEEE, 2001, pp. 749–752.
  60. C. H. Taal, R. C. Hendriks, R. Heusdens, and J. Jensen, “An algorithm for intelligibility prediction of time–frequency weighted noisy speech,” IEEE Transactions on Audio, Speech, and Language Processing, vol. 19, no. 7, pp. 2125–2136, 2011.
  61. J. Jensen and C. H. Taal, “An algorithm for predicting the intelligibility of speech masked by modulated noise maskers,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 24, no. 11, pp. 2009–2022, 2016.
  62. S. Van Kuyk, W. B. Kleijn, and R. C. Hendriks, “An instrumental intelligibility metric based on information theory,” IEEE Signal Processing Letters, vol. 25, no. 1, pp. 115–119, 2017.
  63. T. H. Falk, C. Zheng, and W.-Y. Chan, “A non-intrusive quality and intelligibility measure of reverberant and dereverberated speech,” IEEE Transactions on Audio, Speech, and Language Processing, vol. 18, no. 7, pp. 1766–1774, 2010.
  64. J. F. Santos, M. Senoussaoui, and T. H. Falk, “An improved non-intrusive intelligibility metric for noisy and reverberant speech,” in 2014 14th International Workshop on Acoustic Signal Enhancement (IWAENC).   IEEE, 2014, pp. 55–59.
  65. S. Jørgensen, S. D. Ewert, and T. Dau, “A multi-resolution envelope-power based model for speech intelligibility,” The Journal of the Acoustical Society of America, vol. 134, no. 1, pp. 436–446, 2013.
  66. P. Majdak, C. Hollomey, and R. Baumgartner, “Amt 1. x: A toolbox for reproducible research in auditory modeling,” Acta Acustica, vol. 6, p. 19, 2022.
  67. H. Relaño-Iborra, T. May, J. Zaar, C. Scheidiger, and T. Dau, “Predicting speech intelligibility based on a correlation metric in the envelope power spectrum domain,” The Journal of the Acoustical Society of America, vol. 140, no. 4, pp. 2670–2679, 2016.
  68. H. Relaño-Iborra, J. Zaar, and T. Dau, “A speech-based computational auditory signal processing and perception model,” The Journal of the Acoustical Society of America, vol. 146, no. 5, pp. 3306–3317, 2019.
  69. J. M. Kates and K. H. Arehart, “The hearing-aid speech perception index (HASPI) version 2,” Speech Communication, vol. 131, pp. 35–46, 2021.
  70. M. R. Schädler, A. Warzybok, S. D. Ewert, and B. Kollmeier, “A simulation framework for auditory discrimination experiments: Revealing the importance of across-frequency processing in speech perception,” The Journal of the Acoustical Society of America, vol. 139, no. 5, pp. 2708–2722, 2016.
  71. H. Kayser, T. Herzke, P. Maanen, M. Zimmermann, G. Grimm, and V. Hohmann, “Open community platform for hearing aid algorithm research: open master hearing aid (openMHA),” SoftwareX, vol. 17, p. 100953, 2022.
  72. D. Byrne, A. Parkinson, and P. Newall, “Hearing aid gain and frequency response requirements for the severely/profoundly hearing impaired.” Ear and Hearing, vol. 11, no. 1, pp. 40–49, 1990.
  73. S. Ewert and G. Grimm, “Model-based hearing aid gain prescription rule,” in Proceedings of the International Symposium on Auditory and Audiological Research, vol. 3, 2011, pp. 393–400.
  74. C. J. Plack, V. Drga, and E. A. Lopez-Poveda, “Inferred basilar-membrane response functions for listeners with mild to moderate sensorineural hearing loss,” The Journal of the Acoustical Society of America, vol. 115, no. 4, pp. 1684–1695, 2004.
  75. F. Drakopoulos, A. Van Den Broucke, and S. Verhulst, “A DNN-based hearing-aid strategy for real-time processing: One size fits all,” in ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023, pp. 1–5.
  76. L. W. Balling, O. Townend, G. Stiefenhofer, and W. Switalski, “Reducing hearing aid delay for optimal sound quality: A new paradigm in processing,” Hearing Review, vol. 27, no. 4, pp. 20–26, 2020.
  77. S. Haro, C. J. Smalt, G. A. Ciccarelli, and T. F. Quatieri, “Deep neural network model of hearing-impaired speech-in-noise perception,” Frontiers in Neuroscience, vol. 14, p. 588448, 2020.
  78. P. A. L. Bysted, J. Jensen, Z.-H. Tan, J. Østergaard, and L. Bramsløw, “A parameter-conditional neural network framework for modelling parameterized auditory models,” Euregio/BNAM, 2022.
  79. B. C. Moore, “The choice of compression speed in hearing aids: Theoretical and practical considerations and the role of individual differences,” Trends in Amplification, vol. 12, no. 2, pp. 103–112, 2008.
  80. M. Wouters, F. Drakopoulos, and S. Verhulst, “Hearing-impaired sound perception: What can we learn from a biophysical model of the human auditory periphery?” in 24th International Congress on Acoustics (ICA 2022), 2019, pp. 678–685.
  81. M. L. Jepsen, S. D. Ewert, and T. Dau, “A computational model of human auditory signal processing and perception,” The Journal of the Acoustical Society of America, vol. 124, no. 1, pp. 422–438, 2008.
  82. M. Kolbæk, Z.-H. Tan, S. H. Jensen, and J. Jensen, “On loss functions for supervised monaural time-domain speech enhancement,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 28, no. 1, pp. 825–838, 2020.
Citations (10)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. GitHub - HearingTechnology/DNN-HA: DNN-based hearing aid for real-time sound processing (3 stars)