Semantic incongruity influences response caution in audio-visual integration

详细信息    查看全文

• 作者：Benjamin Steinweg ; Fred W. Mast
• 关键词：Audio ; visual integration ; Multisensory ; Drift diffusion model ; Semantic congruency ; Reaction times
• 刊名：Experimental Brain Research
• 出版年：2017
• 出版时间：January 2017
• 年：2017
• 卷：235
• 期：1
• 页码：349-363
• 全文大小：
• 刊物类别：Biomedical and Life Sciences
• 刊物主题：Neurosciences; Neurology;
• 出版者：Springer Berlin Heidelberg
• ISSN：1432-1106
• 卷排序：235

文摘

Multisensory stimulus combinations trigger shorter reaction times (RTs) than individual single-modality stimuli. It has been suggested that this inter-sensory facilitation effect is found exclusively for semantically congruent stimuli, because incongruity would prevent multisensory integration. Here we provide evidence that the effect of incongruity is due to a change in response caution rather than prevention of stimulus integration. In two experiments, participants performed two-alternative forced-choice decision tasks in which they categorized auditory stimuli, visual stimuli or audio-visual stimulus pairs. The pairs were either semantically congruent (e.g. ambulance image and horn sound) or incongruent (e.g. ambulance image and bell sound). Shorter RTs and violations of the race model inequality on congruent trials are in accordance with previous studies. However, Bayesian hierarchical drift diffusion analyses contradict former co-activation-based explanations of the effects of congruency. Instead, they show that longer RTs on incongruent compared to congruent trials are most likely the result of an incongruity caution effect—more cautious response behaviour in face of semantically incongruent sensory input. Further, they show that response caution can be adjusted on a trial-by-trial basis depending on incoming information. Finally, stimulus modality influenced non-cognitive components of the response. We suggest that the combined stimulus energy from simultaneously presented stimuli reduces encoding time.