Setting the scene: speech understanding and listening effort in virtual scenarios

Background – The assessment of speech understanding in ecologically valid listening scenarios has gained a lot of interest over the past few years. To answer the need of ecological test paradigms, we developed AVATAR, a comprehensive method and test set-up for the real-life assessment of auditory functioning. Compared to clinical speech-in-noise tests, which are most of the time static and one-dimensional, AVATAR aims to mimic the dynamic visual and auditory aspects of everyday listening scenarios. Hence, we aim to capture the auditory challenges people face during daily communication and take cognitive factors like listening effort into account.

In AVATAR, listeners are immersed in different virtual scenarios: a restaurant, living room and public transport environment, each including virtual humans uttering speech auditory-visually. Listening effort is measured by means of an extended dual-task paradigm, with secondary tasks on both auditory localization and visual memory. While previous measurements have shown that the restaurant scenario allows to assess both speech intelligibility and listening effort effectively, the living room and public transport scenarios have not been evaluated yet. The aim of this methodological study is to compare outcome measures in all three scenarios, for different age groups. Additionally, we want to investigate the effect of informational versus energetic masking on speech understanding and listening effort.

Methods – Young (N = 10) and middle-aged (N = 10) normal hearing, Dutch speaking adults will participate in the study. All perform an auditory-visual speech-in-noise task in the restaurant, living room and public transport scenario. Speech is presented in both a multitalker babble noise (energetic masker) and competing Swedish talker (informational masker). Next, secondary tasks are added to the speech-in-noise task to obtain a measure of behavioral listening effort. Finally, participants fill in a questionnaire on subjective listening effort and motivation for each scenario and noise type.

Results – First data on young adults suggest speech understanding is better when presented in a competing talker noise compared to multitalker babble, while the amount of listening effort seems equally high in both noise types. The type of scenario does not affect speech intelligibility, nor listening effort. So far, no clear correlations are found between the behavioral outcome measures and subjective listening effort or motivation.

Funding - a TBM-FWO grant from the Research Foundation-Flanders (grant number T002216N).