Studying effects of vibrotactile stimulation on the neural tracking and intelligibility of continuous auditory speech

Viewing a speaker’s lip movements can improve the brain’s ability to ‘track’ the amplitude envelope of the speech signal and facilitate intelligibility. We hypothesize that such neural and perceptual benefits can also arise from tactually sensing the speech envelope on the skin. To test whether a tactile speech envelope can improve neural tracking of degraded (envelope-reduced) auditory speech and speech recognition, we present continuous audio-vibrotactile speech at various asynchronies to the ears and index fingers of normally-hearing listeners while simultaneously assessing auditory cortical speech-envelope tracking (using electroencephalography and a stimulus-reconstruction approach) and speech-recognition performance.

Our results so far indicate that tactile stimuli carrying speech-envelope information may benefit cortical tracking, but not intelligibility, of degraded auditory speech. The speech-tracking benefit is strongest when the tactile input leads the auditory input by ~50ms and it seems to be primarily driven by early (~100ms post tactile input) cortical responses in the delta (1-4Hz) range. These observations provide preliminary insights into how the human auditory system integrates continuous audio-tactile speech and how this affects intelligibility of degraded auditory speech, which may be relevant for understanding the Tadoma method.