Group conversations: How speech, movement, and gaze behaviours of hearing impaired triads change when conversing in noisy environments

Most conversations do not occur in perfect quiet. Gossiping in a café, talking in a restaurant, or chatting in a car all require people to ignore the background noise to concentrate on what their partners are saying. While these situations are challenging for people with normal hearing, people with hearing impairment have much more difficulty, and may even shun such environments as they know they will fail to keep up. However, interlocutors can draw on a variety of strategies to aid successful communication. Prior studies of isolated speaking and listening have indicated that strategies such as increasing vocal intensity, adjusting head orientation, or directing gaze to the speaker are beneficial in such situations. However, it is not clear whether these strategies are used in a natural interaction context, which differs by including rapid alternation of speaking and listening, the availability of multimodal communicative cues, and the possibility for mutual adaptation.

We therefore investigate how hearing impaired individuals have conversations in noise and make themselves understood in an ecologically valid context. Specifically, we report the fine-grained dynamics of natural conversation between unfamiliar triads of age- and hearing impairment- matched interlocutors (n = 33), addressing how different levels and types of background noise affect speech, movement, and gaze. We investigate behaviour in both speech-shaped noise and 8-talker babble (54dB-78dB) to identify the importance of the information content of competing noise, and we also compare the behaviour of triads with previously collected data from dyads (n = 30) to identify strategies that generalise across interaction types. We show that many potentially beneficial behaviours are not used optimally, including increases in vocal intensity. We also show that interlocutors prioritise gaze cues over beneficial head orientations regardless of interaction type. Understanding these conversation behaviours could inform broader models of interpersonal communication, as well as being used to develop new communication technologies that take advantage of the behaviours that individuals naturally use.