The effects of visual training on multisensory temporal processing
详细信息 查看全文
- 作者：Ryan A. Stevenson (1) (2) (3)
Magdalena M. Wilson (4)
Albert R. Powers (5)
Mark T. Wallace (1) (2) (3) (6) (7) - 关键词：Multisensory integration ; Perceptual training ; Audiovisual ; Vision ; Temporal processing ; Plasticity ; Synchrony
- 刊名：Experimental Brain Research
- 出版年：2013
- 出版时间：April 2013
- 年：2013
- 卷：225
- 期：4
- 页码：479-489
- 全文大小：617KB
- 参考文献：1. Adini Y, Sagi D, Tsodyks M (2002) Context-enabled learning in the human visual system. Nature 415:790-93 CrossRef
2. Alais D, Cass J (2010) Multisensory perceptual learning of temporal order: audiovisual learning transfers to vision but not audition. PLoS ONE 5:e11283 CrossRef
3. Andersen TS, Tiippana K, Sams M (2004) Factors influencing audiovisual fission and fusion illusions. Brain Res Cogn Brain Res 21:301-08 CrossRef
4. Bastien-Toniazzo M, Stroumza A, Cavé C (2009) Audio-visual perception and integration in developmental dyslexia: An exploratory study using the McGurk effect. Current Psychol Lett 25
5. Boddaert N, Chabane N, Gervais H, Good CD, Bourgeois M, Plumet MH, Barthelemy C, Mouren MC, Artiges E, Samson Y, Brunelle F, Frackowiak RS, Zilbovicius M (2004) Superior temporal sulcus anatomical abnormalities in childhood autism: a voxel-based morphometry MRI study. Neuroimage 23:364-69 CrossRef
6. Brainard DH (1997) The psychophysics toolbox. Spat Vis 10:433-36 CrossRef
7. Colonius H, Diederich A (2004) Multisensory interaction in saccadic reaction time: a time-window-of-integration model. J Cogn Neurosci 16:1000-009 CrossRef
8. Conrey BL, Pisoni DB (2004) Detection of auditory-visual asynchrony in speech and nonspeech signals. In: Pisoni DB (ed) Research on spoken language processing, vol 26. Indiana University, Bloomington, pp 71-4
9. Conrey B, Pisoni DB (2006) Auditory-visual speech perception and synchrony detection for speech and nonspeech signals. J Acoust Soc Am 119:4065-073 CrossRef
10. Diederich A, Colonius H (2004) Bimodal and trimodal multisensory enhancement: effects of stimulus onset and intensity on reaction time. Percept Psychophys 66:1388-404 CrossRef
11. Dixon NF, Spitz L (1980) The detection of auditory visual desynchrony. Perception 9:719-21 CrossRef
12. Foss-Feig JH, Kwakye LD, Cascio CJ, Burnette CP, Kadivar H, Stone WL, Wallace MT (2010) An extended multisensory temporal binding window in autism spectrum disorders. Exp Brain Res 203:381-89 CrossRef
13. Foucher JR, Lacambre M, Pham BT, Giersch A, Elliott MA (2007) Low time resolution in schizophrenia Lengthened windows of simultaneity for visual, auditory and bimodal stimuli. Schizophr Res 97:118-27 CrossRef
14. Fujisaki W, Shimojo S, Kashino M, Nishida S (2004) Recalibration of audiovisual simultaneity. Nat Neurosci 7:773-78 CrossRef
15. Gervais H, Belin P, Boddaert N, Leboyer M, Coez A, Sfaello I, Barthelemy C, Brunelle F, Samson Y, Zilbovicius M (2004) Abnormal cortical voice processing in autism. Nat Neurosci 7:801-02 CrossRef
16. Gibbon J (1977) Scalar expectancy theory and Weber’s law in animal timing. Psychol Rev 84:179-25 CrossRef
17. Gibbon J, Church RM, Meck WH (1984) Scalar timing in memory. Ann N Y Acad Sci 423:52-7 CrossRef
18. Hairston WD, Burdette JH, Flowers DL, Wood FB, Wallace MT (2005) Altered temporal profile of visual-auditory multisensory interactions in dyslexia. Exp Brain Res 166:474-80 CrossRef
19. Hershenson M (1962) Reaction time as a measure of intersensory facilitation. J Exp Psychol 63:289-93 CrossRef
20. Hillock AR, Powers AR, Wallace MT (2011) Binding of sights and sounds: age-related changes in multisensory temporal processing. Neuropsychologia 49:461-67 CrossRef
21. James TW, Stevenson RA, Kim S (2012) Inverse effectiveness in multisensory processing. In: Stein BE (ed) The new handbook of multisensory processes. MIT Press, Cambridge, MA
22. Keetels M, Vroomen J (2005) The role of spatial disparity and hemifields in audio-visual temporal order judgments. Exp Brain Res 167:635-40 CrossRef
23. Kim RS, Seitz AR, Shams L (2008) Benefits of stimulus congruency for multisensory facilitation of visual learning. PLoS ONE 3:e1532 CrossRef
24. Kwakye LD, Foss-Feig JH, Cascio CJ, Stone WL, Wallace MT (2011) Altered auditory and multisensory temporal processing in autism spectrum disorders. Front Integr Neurosci 4:129 CrossRef
25. Levitt JG, Blanton RE, Smalley S, Thompson PM, Guthrie D, McCracken JT, Sadoun T, Heinichen L, Toga AW (2003) Cortical sulcal maps in autism. Cereb Cortex 13:728-35 CrossRef
26. Lovelace CT, Stein BE, Wallace MT (2003) An irrelevant light enhances auditory detection in humans: a psychophysical analysis of multisensory integration in stimulus detection. Brain Res Cogn Brain Res 17:447-53 CrossRef
27. Macaluso E, George N, Dolan R, Spence C, Driver J (2004) Spatial and temporal factors during processing of audiovisual speech: a PET study. Neuroimage 21:725-32 CrossRef
28. McGurk H, MacDonald J (1976) Hearing lips and seeing voices. Nature 264:746-48 CrossRef
29. Meredith MA, Stein BE (1986a) Spatial factors determine the activity of multisensory neurons in cat superior colliculus. Brain Res 365:350-54 CrossRef
30. Meredith MA, Stein BE (1986b) Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. J Neurophysiol 56:640-62
31. Meredith MA, Nemitz JW, Stein BE (1987) Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. J Neurosci 7:3215-229
32. Miller LM, D’Esposito M (2005) Perceptual fusion and stimulus coincidence in the cross-modal integration of speech. J Neurosci 25:5884-893 CrossRef
33. Navarra J, Vatakis A, Zampini M, Soto-Faraco S, Humphreys W, Spence C (2005) Exposure to asynchronous audiovisual speech extends the temporal window for audiovisual integration. Brain Res Cogn Brain Res 25:499-07 CrossRef
34. Nelson WT, Hettinger LJ, Cunningham JA, Brickman BJ, Haas MW, McKinley RL (1998) Effects of localized auditory information on visual target detection performance using a helmet-mounted display. Hum Factors 40:452-60 CrossRef
35. Pekkola J, Laasonen M, Ojanen V, Autti T, Jaaskelainen IP, Kujala T, Sams M (2006) Perception of matching and conflicting audiovisual speech in dyslexic and fluent readers: an fMRI study at 3 T. Neuroimage 29:797-07 CrossRef
36. Pelli DG (1997) The VideoToolbox software for visual psychophysics: transforming numbers into movies. Spat Vis 10:437-42 CrossRef
37. Pelphrey KA, Carter EJ (2008a) Brain mechanisms for social perception: lessons from autism and typical development. Ann N Y Acad Sci 1145:283-99 CrossRef
38. Pelphrey KA, Carter EJ (2008b) Charting the typical and atypical development of the social brain. Dev Psychopathol 20:1081-102 CrossRef
39. P?ppel E, Schill K, von Steinbüchel N (1990) Sensory integration within temporally neutral systems states: a hypothesis. Naturwissenschaften 77:89-1 CrossRef
40. Powers AR 3rd, Hillock AR, Wallace MT (2009) Perceptual training narrows the temporal window of multisensory binding. J Neurosci 29:12265-2274 CrossRef
41. Powers AR 3rd, Hevey MA, Wallace MT (2012) Neural correlates of multisensory perceptual learning. J Neurosci 32:6263-274 CrossRef
42. Richards T, Stevenson J, Crouch J, Johnson LC, Maravilla K, Stock P, Abbott R, Berninger V (2008) Tract-based spatial statistics of diffusion tensor imaging in adults with dyslexia. AJNR Am J Neuroradiol 29:1134-139 CrossRef
43. Roach NW, Heron J, Whitaker D, McGraw PV (2011) Asynchrony adaption reveals neural population code for audio-visual timing. Proc R Soc 278:9 CrossRef
44. Royal DW, Carriere BN, Wallace MT (2009) Spatiotemporal architecture of cortical receptive fields and its impact on multisensory interactions. Exp Brain Res 198:127-36 CrossRef
45. Schall S, Quigley C, Onat S, Konig P (2009) Visual stimulus locking of EEG is modulated by temporal congruency of auditory stimuli. Exp Brain Res 198:137-51 CrossRef
46. Seitz AR, Nanez JE Sr, Holloway SR, Watanabe T (2006) Perceptual learning of motion leads to faster flicker perception. PLoS ONE 1:e28 CrossRef
47. Senkowski D, Talsma D, Grigutsch M, Herrmann CS, Woldorff MG (2007) Good times for multisensory integration: effects of the precision of temporal synchrony as revealed by gamma-band oscillations. Neuropsychologia 45:561-71 CrossRef
48. Shams L, Kamitani Y, Shimojo S (2000) Illusions. What you see is what you hear. Nature 408:788 CrossRef
49. Shenton ME, Dickey CC, Frumin M, McCarley RW (2001) A review of MRI findings in schizophrenia. Schizophr Res 49:1-2 CrossRef
50. Spence C (2007) Audiovisual multisensory integration. Acoustics Sci Technol 28:61-0 CrossRef
51. Stein B, Meredith MA (1993) The merging of the senses. MIT Press, Boston, MA
52. Stein BE, Wallace MT (1996) Comparisons of cross-modality integration in midbrain and cortex. Prog Brain Res 112:289-99 CrossRef
53. Stevenson RA, James TW (2009) Audiovisual integration in human superior temporal sulcus: inverse effectiveness and the neural processing of speech and object recognition. Neuroimage 44:1210-223 CrossRef
54. Stevenson RA, Wallace MW (under review) Multisensory temporal integration: task and stimulus dependencies Exp Brain Res
55. Stevenson RA, Altieri NA, Kim S, Pisoni DB, James TW (2010) Neural processing of asynchronous audiovisual speech perception. Neuroimage 49:3308-318 CrossRef
56. Stevenson RA, VanDerKlok RM, Pisoni DB, James TW (2011) Discrete neural substrates underlie complementary audiovisual speech integration processes. Neuroimage 55:1339-345 CrossRef
57. Stevenson RA, Zemtsov RK, Wallace MT (2012) Individual differences in the multisensory temporal binding window predict susceptibility to audiovisual illusions. J Exp Psychol Hum Percept Perform 38:1517-529
58. Talsma D, Senkowski D, Woldorff MG (2009) Intermodal attention affects the processing of the temporal alignment of audiovisual stimuli. Exp Brain Res 198:313-28 CrossRef
59. van Atteveldt NM, Formisano E, Blomert L, Goebel R (2007) The effect of temporal asynchrony on the multisensory integration of letters and speech sounds. Cereb Cortex 17:962-74 CrossRef
60. van Eijk RL, Kohlrausch A, Juola JF, van de Par S (2008) Audiovisual synchrony and temporal order judgments: effects of experimental method and stimulus type. Percept Psychophys 70:955-68 CrossRef
61. van Wassenhove V, Grant KW, Poeppel D (2007) Temporal window of integration in auditory-visual speech perception. Neuropsychologia 45:598-07 CrossRef
62. Vatakis A, Spence C (2006) Audiovisual synchrony perception for music, speech, and object actions. Brain Res 1111:134-42 CrossRef
63. Vatakis A, Spence C (2007) Crossmodal binding: evaluating the “unity assumption-using audiovisual speech stimuli. Percept Psychophys 69:744-56 CrossRef
64. Vatakis A, Navarra J, Soto-Faraco S, Spence C (2007) Temporal recalibration during asynchronous audiovisual speech perception. Exp Brain Res 181:173-81 CrossRef
65. Vatakis A, Ghazanfar AA, Spence C (2008) Facilitation of multisensory integration by the “unity effect-reveals that speech is special. J Vis 8(14):1-1
66. Vroomen J, Baart M (2009) Phonetic recalibration only occurs in speech mode. Cognition 110:254-59 CrossRef
67. Vroomen J, Keetels M, de Gelder B, Bertelson P (2004) Recalibration of temporal order perception by exposure to audio-visual asynchrony. Brain Res Cogn Brain Res 22:32-5 CrossRef
68. Wallace MT, Roberson GE, Hairston WD, Stein BE, Vaughan JW, Schirillo JA (2004) Unifying multisensory signals across time and space. Exp Brain Res 158:252-58 CrossRef
69. Wearden JH, Edwards H, Fakhri M, Percival A (1998) Why “sounds are judged longer than lights- application of a model of the internal clock in humans. Q J Exp Psychol B 51:97-20
70. Wilkinson LK, Meredith MA, Stein BE (1996) The role of anterior ectosylvian cortex in cross-modality orientation and approach behavior. Exp Brain Res 112:1-0 CrossRef
71. Zampini M, Guest S, Shore DI, Spence C (2005) Audio-visual simultaneity judgments. Percept Psychophys 67:531-44 CrossRef - 作者单位：Ryan A. Stevenson (1) (2) (3)
Magdalena M. Wilson (4)
Albert R. Powers (5)
Mark T. Wallace (1) (2) (3) (6) (7)
1. Department of Hearing and Speech Sciences, Vanderbilt University Medical Center, Medical Research Building III, Suite 7110C, Nashville, TN, USA
2. Vanderbilt Brain Institute, Nashville, TN, USA
3. Vanderbilt Kennedy Center, Nashville, TN, USA
4. Undergraduate Program in Neuroscience, Vanderbilt University, Nashville, TN, USA
5. Medical Scientist Training Program, Vanderbilt University Medical Center, Nashville, TN, USA
6. Department of Psychology, Vanderbilt University, Nashville, TN, USA
7. Department of Psychiatry, Vanderbilt University Medical Center, Nashville, TN, USA - ISSN：1432-1106
文摘
The importance of multisensory integration for human behavior and perception is well documented, as is the impact that temporal synchrony has on driving such integration. Thus, the more temporally coincident two sensory inputs from different modalities are, the more likely they will be perceptually bound. This temporal integration process is captured by the construct of the temporal binding window—the range of temporal offsets within which an individual is able to perceptually bind inputs across sensory modalities. Recent work has shown that this window is malleable and can be narrowed via a multisensory perceptual feedback training process. In the current study, we seek to extend this by examining the malleability of the multisensory temporal binding window through changes in unisensory experience. Specifically, we measured the ability of visual perceptual feedback training to induce changes in the multisensory temporal binding window. Visual perceptual training with feedback successfully improved temporal visual processing, and more importantly, this visual training increased the temporal precision across modalities, which manifested as a narrowing of the multisensory temporal binding window. These results are the first to establish the ability of unisensory temporal training to modulate multisensory temporal processes, findings that can provide mechanistic insights into multisensory integration and which may have a host of practical applications.