No AccessEditor's AwardJournal of Speech, Language, and Hearing ResearchResearch Article15 Apr 2019

Listening Effort by Native and Nonnative Listeners Due to Noise, Reverberation, and Talker Foreign Accent During English Speech Perception

    Purpose

    Understanding speech in complex realistic acoustic environments requires effort. In everyday listening situations, speech quality is often degraded due to adverse acoustics, such as excessive background noise level (BNL) and reverberation time (RT), or talker characteristics such as foreign accent (Mattys, Davis, Bradlow, & Scott, 2012). In addition to factors affecting the quality of the input acoustic signals, listeners' individual characteristics such as language abilities can also make it more difficult and effortful to understand speech. Based on the Framework for Understanding Effortful Listening (Pichora-Fuller et al., 2016), factors such as adverse acoustics, talker accent, and listener language abilities can all contribute to increasing listening effort. In this study, using both a dual-task paradigm and a self-report questionnaire, we seek to understand listening effort in a wide range of realistic classroom acoustic conditions as well as varying talker accent and listener English proficiency.

    Method

    One hundred fifteen native and nonnative adult listeners with normal hearing were tested in a dual task of speech comprehension and adaptive pursuit rotor (APR) under 15 acoustic conditions from combinations of BNLs and RTs. Listeners provided responses on the NASA Task Load Index (TLX) questionnaire immediately after completing the dual task under each acoustic condition. The NASA TLX surveyed 6 dimensions of perceived listening effort: mental demand, physical demand, temporal demand, effort, frustration, and perceived performance. Fifty-six listeners were tested with speech produced by native American English talkers; the other 59 listeners, with speech from native Mandarin Chinese talkers. Based on their 1st language learned during childhood, 3 groups of listeners were recruited: listeners who were native English speakers, native Mandarin Chinese speakers, and native speakers of other languages (e.g., Hindu, Korean, and Portuguese).

    Results

    Listening effort was measured objectively through the APR task performance and subjectively using the NASA TLX questionnaire. Performance on the APR task did not vary with changing acoustic conditions, but it did suggest increased listening effort for native listeners of other languages compared to the 2 other listener groups. From the NASA TLX, listeners reported feeling more frustrated and less successful in understanding Chinese-accented speech. Nonnative listeners reported more listening effort (i.e., physical demand, temporal demand, and effort) than native listeners in speech comprehension under adverse acoustics. When listeners' English proficiency was controlled, higher BNL was strongly related to a decrease in perceived performance, whereas such relationship with RT was much weaker. Nonnative listeners who shared the foreign talkers' accent reported no change in listening effort, whereas other listeners reported more difficulty in understanding the accented speech.

    Conclusions

    Adverse acoustics required more effortful listening as measured subjectively with a self-report NASA TLX. This subjective scale was more sensitive than a dual task that involved speech comprehension, which was beyond sentence recall. It was better at capturing the negative impacts on listening effort from acoustic factors (i.e., both BNL and RT), talker accent, and listener language abilities.

    References

    • Ahlstrom, J. B., Horwitz, A. R., & Dubno, J. R. (2014). Spatial separation benefit for unaided and aided listening.Ear and Hearing, 35(1), 72–85.
    • Aud, S., Hussar, W., Planty, M., Snyder, T., Bianco, K., Fox, M. A., … Drake, L. (2010). The condition of education—2010. Washington, DC: U.S. Department of Education. Retrieved from https://nces.ed.gov/pubs2010/2010028.pdf
    • Bent, T., & Bradlow, A. R. (2003). The interlanguage speech intelligibility benefit.The Journal of the Acoustical Society of America, 114(3), 1600–1610. https://doi.org/10.1121/1.1603234
    • Bent, T., Kewley-Port, D., & Ferguson, S. H. (2010). Across-talker effects on non-native listeners' vowel perception in noise.The Journal of the Acoustical Society of America, 128(5), 3142–3151. https://doi.org/10.1121/1.3493428
    • Blazier, W. E. (1981). Revised noise criteria for design and rating of HVAC systems.ASHRAE Journal, 23(5), 34–38. https://doi.org/10.3397/1.2832172
    • Bologna, W. J., Chatterjee, M., & Dubno, J. R. (2013). Perceived listening effort for a tonal task with contralateral competing signals.The Journal of the Acoustical Society of America, 134(4), EL352–EL358.
    • Bradlow, A. R., & Alexander, J. A. (2007). Semantic and phonetic enhancements for speech-in-noise recognition by native and non-native listeners.The Journal of the Acoustical Society of America, 121(4), 2339–2349.
    • Bradlow, A. R., & Bent, T. (2002). The clear speech effect for non-native listeners.The Journal of the Acoustical Society of America, 112(1), 272–284. https://doi.org/10.1121/1.1487837
    • Broadbent, D. E. (1958). Perception and communication. London, England: Pergamon Press.
    • Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
    • Desjardins, J. L., & Doherty, K. A. (2013). Age-related changes in listening effort for various types of masker noises.Ear and Hearing, 34(3), 261–272. https://doi.org/10.1097/AUD.0b013e31826d0ba4
    • Educational Testing Service. (n.d.). TOEIC Listening and Reading Test. Retrieved January 2, 2011, from https://www.ets.org/toeic
    • Harris, K. C., Eckert, M. A., Ahlstrom, J. B., & Dubno, J. R. (2010). Age-related differences in gap detection: Effects of task difficulty and cognitive ability.Hearing Research, 264(1), 21–29.
    • Hart, S. G. (2006). NASA-task load index (NASA-TLX); 20 years later. Paper presented at the Proceedings of the Human Factors and Ergonomics Society Annual Meeting, San Francisco, CA.
    • Hart, S. G., & Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research.Human Mental Workload, 1, 139–183.
    • Hickok, G., & Poeppel, D. (2007). The cortical organization of speech processing.Nature Reviews Neuroscience, 8(5), 393–402.
    • Hodgson, M., Rempel, R., & Kennedy, S. (1999). Measurement and prediction of typical speech and background-noise levels in university classrooms during lectures.The Journal of the Acoustical Society of America, 105(1), 226–233. https://doi.org/10.1121/1.424600
    • Howard, C. S., Munro, K. J., & Plack, C. J. (2010). Listening effort at signal-to-noise ratios that are typical of the school classroom.International Journal of Audiology, 49(12), 928–932.
    • Kahneman, D. (1973). Attention and effort.The American Journal of Psychology, 88, 339–340. https://doi.org/10.2307/1421603
    • LacoutureParodi, Y. (2011). Analysis of design parameters for crosstalk cancellation filters applied to different loudspeaker configurations. Paper presented at the Audio Engineering Society Convention, New York, NY.
    • Ljung, R., & Kjellberg, A. (2009). Recall of spoken words presented with a prolonged reverberation time.Building Acoustics, 16, 301–312.
    • Mackersie, C. L., & Cones, H. (2011). Subjective and psychophysiological indexes of listening effort in a competing-talker task.Journal of the American Academy of Audiology, 22(2), 113–122. https://doi.org/10.3766/jaaa.22.2.6
    • Mattys, S. L., Davis, M. H., Bradlow, A. R., & Scott, S. K. (2012). Speech recognition in adverse conditions: A review.Language and Cognitive Processes, 27(7–8), 953–978. https://doi.org/10.1080/01690965.2012.705006
    • McGarrigle, R., Munro, K. J., Dawes, P., Stewart, A. J., Moore, D. R., Barry, J. G., & Amitay, S. (2014). Listening effort and fatigue: What exactly are we measuring? A British Society of Audiology Cognition in Hearing Special Interest Group “white paper.”.International Journal of Audiology, 53, 433–445. https://doi.org/10.3109/14992027.2014.890296
    • Muñoz-Sandoval, A. F., Cummins, J., Alvarado, C. G., & Ruef, M. L. (1998). Bilingual verbal ability tests: Comprehensive manual. Itasca, IL: Riverside Publishing.
    • National Science Board. (2012). Science and engineering indicators: 2012 digest. Arlington, VA: Author. Retrieved from https://www.nsf.gov/statistics/digest12/
    • Pearson. (2008). Versant English Test. White Plains, NY: Pearson Education, Inc.
    • Peelle, J. E. (2018). Listening effort: How the cognitive consequences of acoustic challenge are reflected in brain and behavior.Ear and Hearing, 39(2), 204–214. https://doi.org/10.1097/AUD.0000000000000494
    • Peng, Z. E. (2014). Designing acoustics for linguistically diverse classrooms: Effects of background noise, reverberation and talker foreign accent on speech comprehension by native and non-native English-speaking listeners.PhD dissertation, University of Nebraska, Lincoln, NE).
    • Peng, Z. E., & Wang, L. M. (2016). Effects of noise, reverberation and foreign accent on native and non-native listeners' performance of English speech comprehension.The Journal of the Acoustical Society of America, 139(5), 2772–2783.
    • Pichora-Fuller, M. K., Kramer, S. E., Eckert, M. A., Edwards, B., Hornsby, B. W. Y., Humes, L. E., … Wingfield, A. (2016). Hearing impairment and cognitive energy: The framework for understanding effortful listening (FUEL).Ear and Hearing, 37, 5S–27S. https://doi.org/10.1097/AUD.0000000000000312
    • Picou, E. M., Gordon, J., & Ricketts, T. A. (2016). The effects of noise and reverberation on listening effort in adults with normal hearing.Ear and Hearing, 37(1), 1–13.
    • Prodi, N., Visentin, C., & Farnetani, A. (2010). Intelligibility, listening difficulty and listening efficiency in auralized classrooms.The Journal of the Acoustical Society of America, 128(1), 172–181.
    • Rogers, C. L., Lister, J. J., Febo, D. M., Besing, J. M., & Abrams, H. B. (2006). Effects of bilingualism, noise, and reverberation on speech perception by listeners with normal hearing.Applied Psycholinguistics, 27(3), 465–485.
    • Ronsse, L. M., & Wang, L. M. (2010). Effects of noise from building mechanical systems on elementary school student achievement. ASHRAE Transactions, 116(2010), 347–354.
    • Sato, H., & Bradley, J. S. (2008). Evaluation of acoustical conditions for speech communication in working elementary school classrooms.The Journal of the Acoustical Society of America, 123(4), 2064–2077. https://doi.org/10.1121/1.2839283
    • Sato, H., Bradley, J. S., & Morimoto, M. (2005). Using listening difficulty ratings of conditions for speech communication in rooms.The Journal of the Acoustical Society of America, 117(3), 1157–1167.
    • Shield, B., Conetta, R., Dockrell, J., Connolly, D., Cox, T., & Mydlarz, C. (2015). A survey of acoustic conditions and noise levels in secondary school classrooms in England.The Journal of the Acoustical Society of America, 137(1), 177–188. https://doi.org/10.1121/1.4904528
    • Shield, B., & Dockrell, J. E. (2004). External and internal noise surveys of London primary schools.The Journal of the Acoustical Society of America, 115(2), 730–738. https://doi.org/10.1121/1.1635837
    • Srinivasan, N. K. (2010). The perception of natural, cell phone, and computer-synthesized speech during the performance of simultaneous visual–motor tasks.PhD dissertation, University of Nebraska, Lincoln, NE).
    • Van Engen, K. J., & Peelle, J. E. (2014). Listening effort and accented speech.Frontiers in Human Neuroscience, 8, 1–4.
    • Veneman, C. E., Gordon-Salant, S., Matthews, L. J., & Dubno, J. R. (2013). Age and measurement time-of-day effects on speech recognition in noise.Ear and Hearing, 34(3), 288–299.
    • Woodcock, R. W., McGrew, K. S., & Mather, N. (2001a). Woodcock-Johnson Tests of Achievement. Itasca, IL: Riverside Publishing.
    • Woodcock, R. W., McGrew, K. S., & Mather, N. (2001b). Woodcock-Johnson III Tests of Cognitive Abilities. Itasca, IL: Riverside Publishing.

    Additional Resources