Effects of congruent and incongruent visual cues on speech perception and brain activity in cochlear implant users

详细信息    查看全文

• 作者：Jae-Jin Song (1)
  Hyo-Jeong Lee (2) (3)
  Hyejin Kang (4)
  Dong Soo Lee (4)
  Sun O. Chang (3) (5)
  Seung Ha Oh (3) (5)
  
  1. Department of Otorhinolaryngology-Head and Neck Surgery ; Seoul National University Bundang Hospital ; Seongnam ; Korea
  2. Department of Otorhinolaryngology-Head and Neck Surgery ; Hallym University College of Medicine ; Chun-Cheon ; Korea
  3. Sensory Organ Research Institute ; Seoul National University Medical Research Center ; Seoul ; Korea
  4. Department of Nuclear Medicine ; Seoul National University Hospital ; Seoul ; Korea
  5. Department of Otorhinolaryngology-Head and Neck Surgery ; Seoul National University Hospital ; 101 Daehak-Ro Jongno-Gu ; Seoul ; 110-744 ; Korea
  
• 关键词：Cochlear implant ; Deafness ; Positron emission tomography ; Audiovisual ; Plasticity
• 刊名：Brain Structure and Function
• 出版年：2015
• 出版时间：March 2015
• 年：2015
• 卷：220
• 期：2
• 页码：1109-1125
• 全文大小：2,699 KB
• 参考文献：1. Aboitiz, F, Garcia, R (2009) Merging of phonological and gestural circuits in early language evolution. Rev Neurosci 20: pp. 71-84
  2. Adam, R, Noppeney, U (2010) Prior auditory information shapes visual category-selectivity in ventral occipito-temporal cortex. Neuroimage 52: pp. 1592-1602 CrossRef
  3. Archbold, S, Lutman, ME, Marshall, DH (1995) Categories of auditory performance. Ann Otol Rhinol Laryngol Suppl 166: pp. 312-314
  4. Aron, AR, Fletcher, PC, Bullmore, ET, Sahakian, BJ, Robbins, TW (2003) Stop-signal inhibition disrupted by damage to right inferior frontal gyrus in humans. Nat Neurosci 6: pp. 115-116 CrossRef
  5. Beauchamp, MS (2005) Statistical criteria in FMRI studies of multisensory integration. Neuroinformatics 3: pp. 93-113 CrossRef
  6. Beauchamp, MS, Argall, BD, Bodurka, J, Duyn, JH, Martin, A (2004) Unraveling multisensory integration: patchy organization within human STS multisensory cortex. Nat Neurosci 7: pp. 1190-1192 CrossRef
  7. Beauchamp, MS, Lee, KE, Argall, BD, Martin, A (2004) Integration of auditory and visual information about objects in superior temporal sulcus. Neuron 41: pp. 809-823 CrossRef
  8. Beauchamp, MS, Nath, AR, Pasalar, S (2010) fMRI-guided transcranial magnetic stimulation reveals that the superior temporal sulcus is a cortical locus of the McGurk effect. J Neurosci 30: pp. 2414-2417 CrossRef
  9. Bernstein, LE, Auer, ET, Wagner, M, Ponton, CW (2008) Spatiotemporal dynamics of audiovisual speech processing. Neuroimage 39: pp. 423-435 CrossRef
  10. Bitan, T, Lifshitz, A, Breznitz, Z, Booth, JR (2010) Bidirectional connectivity between hemispheres occurs at multiple levels in language processing but depends on sex. J Neurosci 30: pp. 11576-11585 CrossRef
  11. Bunge, SA, Dudukovic, NM, Thomason, ME, Vaidya, CJ, Gabrieli, JD (2002) Immature frontal lobe contributions to cognitive control in children: evidence from fMRI. Neuron 33: pp. 301-311 CrossRef
  12. Calvert, GA, Campbell, R, Brammer, MJ (2000) Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Curr Biol 10: pp. 649-657 CrossRef
  13. Champoux, F, Lepore, F, Gagne, JP, Theoret, H (2009) Visual stimuli can impair auditory processing in cochlear implant users. Neuropsychologia 47: pp. 17-22 CrossRef
  14. Corbetta, M, Shulman, GL (2002) Control of goal-directed and stimulus-driven attention in the brain. Nat Rev Neurosci 3: pp. 201-215 CrossRef
  15. Doucet, ME, Bergeron, F, Lassonde, M, Ferron, P, Lepore, F (2006) Cross-modal reorganization and speech perception in cochlear implant users. Brain 129: pp. 3376-3383 CrossRef
  16. Eickhoff, SB, Stephan, KE, Mohlberg, H, Grefkes, C, Fink, GR, Amunts, K, Zilles, K (2005) A new SPM toolbox for combining probabilistic cytoarchitectonic maps and functional imaging data. Neuroimage 25: pp. 1325-1335 CrossRef
  17. Finney, EM, Fine, I, Dobkins, KR (2001) Visual stimuli activate auditory cortex in the deaf. Nat Neurosci 4: pp. 1171-1173 CrossRef
  18. Fletcher, PC, Henson, RN (2001) Frontal lobes and human memory: insights from functional neuroimaging. Brain 124: pp. 849-881 CrossRef
  19. Garavan, H, Ross, TJ, Stein, EA (1999) Right hemispheric dominance of inhibitory control: an event-related functional MRI study. Proc Natl Acad Sci USA 96: pp. 8301-8306 CrossRef
  20. Genovese, CR, Lazar, NA, Nichols, T (2002) Thresholding of statistical maps in functional neuroimaging using the false discovery rate. Neuroimage 15: pp. 870-878 CrossRef
  21. Giraud, AL, Lee, HJ (2007) Predicting cochlear implant outcome from brain organisation in the deaf. Restor Neurol Neurosci 25: pp. 381-390
  22. Giraud, AL, Truy, E (2002) The contribution of visual areas to speech comprehension: a PET study in cochlear implants patients and normal-hearing subjects. Neuropsychologia 40: pp. 1562-1569 CrossRef
  23. Giraud, AL, Price, CJ, Graham, JM, Truy, E, Frackowiak, RS (2001) Cross-modal plasticity underpins language recovery after cochlear implantation. Neuron 30: pp. 657-663 CrossRef
  24. Goldberg, RF, Perfetti, CA, Fiez, JA, Schneider, W (2007) Selective retrieval of abstract semantic knowledge in left prefrontal cortex. J Neurosci 27: pp. 3790-3798 CrossRef
  25. Guenther, FH, Ghosh, SS, Tourville, JA (2006) Neural modeling and imaging of the cortical interactions underlying syllable production. Brain Lang 96: pp. 280-301 CrossRef
  26. Haddon, JE, Killcross, S (2006) Prefrontal cortex lesions disrupt the contextual control of response conflict. J Neurosci 26: pp. 2933-2940 CrossRef
  27. Hein, G, Doehrmann, O, Muller, NG, Kaiser, J, Muckli, L, Naumer, MJ (2007) Object familiarity and semantic congruency modulate responses in cortical audiovisual integration areas. J Neurosci 27: pp. 7881-7887 CrossRef
  28. Jones, JA, Callan, DE (2003) Brain activity during audiovisual speech perception: an fMRI study of the McGurk effect. Neuroreport 14: pp. 1129-1133 CrossRef
  29. Kempe, V, Thoresen, JC, Kirk, NW, Schaeffler, F, Brooks, PJ (2012) Individual differences in the discrimination of novel speech sounds: effects of sex, temporal processing, musical and cognitive abilities. PLoS One 7: pp. e48623 CrossRef
  30. Koechlin, E, Hyafil, A (2007) Anterior prefrontal function and the limits of human decision-making. Science 318: pp. 594-598 CrossRef
  31. Koechlin, E, Ody, C, Kouneiher, F (2003) The architecture of cognitive control in the human prefrontal cortex. Science 302: pp. 1181-1185 CrossRef
  32. Koles, ZJ, Lind, JC, Flor-Henry, P (2010) Gender differences in brain functional organization during verbal and spatial cognitive challenges. Brain Topogr 23: pp. 199-204 CrossRef
  33. Kral, A, O鈥橠onoghue, GM (2010) Profound deafness in childhood. N Engl J Med 363: pp. 1438-1450 CrossRef
  34. Kreifelts, B, Ethofer, T, Huberle, E, Grodd, W, Wildgruber, D (2010) Association of trait emotional intelligence and individual fMRI-activation patterns during the perception of social signals from voice and face. Hum Brain Mapp 31: pp. 979-991 CrossRef
  35. Lazard, DS, Lee, HJ, Gaebler, M, Kell, CA, Truy, E, Giraud, AL (2010) Phonological processing in post-lingual deafness and cochlear implant outcome. Neuroimage 49: pp. 3443-3451 CrossRef
  36. Lazard, DS, Giraud, AL, Truy, E, Lee, HJ (2011) Evolution of non-speech sound memory in postlingual deafness: implications for cochlear implant rehabilitation. Neuropsychologia 49: pp. 2475-2482 CrossRef
  37. Lazard, DS, Giraud, AL, Gnansia, D, Meyer, B, Sterkers, O (2012) Understanding the deafened brain: implications for cochlear implant rehabilitation. Eur Ann Otorhinolaryngol Head Neck Dis 129: pp. 98-103 CrossRef
  38. Lazard DS, Lee HJ, Truy E, Giraud AL (2013) Bilateral reorganization of posterior temporal cortices in post-lingual deafness and its relation to cochlear implant outcome. Hum Brain Mapp 34:1208鈥?219
  39. Lee, DS, Lee, JS, Oh, SH, Kim, SK, Kim, JW, Chung, JK, Lee, MC, Kim, CS (2001) Cross-modal plasticity and cochlear implants. Nature 409: pp. 149-150 CrossRef
  40. Lee, JS, Lee, DS, Oh, SH, Kim, CS, Kim, JW, Hwang, CH, Koo, J, Kang, E, Chung, JK, Lee, MC (2003) PET evidence of neuroplasticity in adult auditory cortex of postlingual deafness. J Nucl Med 44: pp. 1435-1439
  41. Lee, HJ, Giraud, AL, Kang, E, Oh, SH, Kang, H, Kim, CS, Lee, DS (2007) Cortical activity at rest predicts cochlear implantation outcome. Cereb Cortex 17: pp. 909-917 CrossRef
  42. Lee, HJ, Truy, E, Mamou, G, Sappey-Marinier, D, Giraud, AL (2007) Visual speech circuits in profound acquired deafness: a possible role for latent multimodal connectivity. Brain 130: pp. 2929-2941 CrossRef
  43. Marquis, JP, Killcross, S, Haddon, JE (2007) Inactivation of the prelimbic, but not infralimbic, prefrontal cortex impairs the contextual control of response conflict in rats. Eur J Neurosci 25: pp. 559-566 CrossRef
  44. McGurk, H, MacDonald, J (1976) Hearing lips and seeing voices. Nature 264: pp. 746-748 CrossRef
  45. Meredith, MA, Stein, BE (1983) Interactions among converging sensory inputs in the superior colliculus. Science 221: pp. 389-391 CrossRef
  46. Mesulam, MM (1998) From sensation to cognition. Brain 121: pp. 1013-1052 CrossRef
  47. Miller, LM, D鈥橢sposito, M (2005) Perceptual fusion and stimulus coincidence in the cross-modal integration of speech. J Neurosci 25: pp. 5884-5893 CrossRef
  48. Molenberghs, P, Brander, C, Mattingley, JB, Cunnington, R (2010) The role of the superior temporal sulcus and the mirror neuron system in imitation. Hum Brain Mapp 31: pp. 1316-1326 CrossRef
  49. Nath, AR, Beauchamp, MS (2011) Dynamic changes in superior temporal sulcus connectivity during perception of noisy audiovisual speech. J Neurosci 31: pp. 1704-1714 CrossRef
  50. Nath, AR, Beauchamp, MS (2012) A neural basis for interindividual differences in the McGurk effect, a multisensory speech illusion. Neuroimage 59: pp. 781-787 CrossRef
  51. Nishitani, N, Hari, R (2002) Viewing lip forms: cortical dynamics. Neuron 36: pp. 1211-1220 CrossRef
  52. Noppeney, U, Josephs, O, Hocking, J, Price, CJ, Friston, KJ (2008) The effect of prior visual information on recognition of speech and sounds. Cereb Cortex 18: pp. 598-609 CrossRef
  53. Park, H, Kang, E, Kang, H, Kim, JS, Jensen, O, Chung, CK, Lee, DS (2011) Cross-frequency power correlations reveal the right superior temporal gyrus as a hub region during working memory maintenance. Brain Connect 1: pp. 460-472 CrossRef
  54. Rauschecker, JP (1999) Auditory cortical plasticity: a comparison with other sensory systems. Trends Neurosci 22: pp. 74-80 CrossRef
  55. Ross, LA, Saint-Amour, D, Leavitt, VM, Javitt, DC, Foxe, JJ (2007) Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments. Cereb Cortex 17: pp. 1147-1153 CrossRef
  56. Rouger, J, Lagleyre, S, Fraysse, B, Deneve, S, Deguine, O, Barone, P (2007) Evidence that cochlear-implanted deaf patients are better multisensory integrators. Proc Natl Acad Sci USA 104: pp. 7295-7300 CrossRef
  57. Rouger, J, Fraysse, B, Deguine, O, Barone, P (2008) McGurk effects in cochlear-implanted deaf subjects. Brain Res 1188: pp. 87-99 CrossRef
  58. Rouger, J, Lagleyre, S, Demonet, JF, Fraysse, B, Deguine, O, Barone, P (2012) Evolution of crossmodal reorganization of the voice area in cochlear-implanted deaf patients. Hum Brain Mapp 33: pp. 1929-1940 CrossRef
  59. Ruff, DA, Marrett, S, Heekeren, HR, Bandettini, PA, Ungerleider, LG (2010) Complementary roles of systems representing sensory evidence and systems detecting task difficulty during perceptual decision making. Front Neurosci 4: pp. 190 CrossRef
  60. Schaefer, A, Braver, TS, Reynolds, JR, Burgess, GC, Yarkoni, T, Gray, JR (2006) Individual differences in amygdala activity predict response speed during working memory. J Neurosci 26: pp. 10120-10128 CrossRef
  61. Skipper, JI, Nusbaum, HC, Small, SL (2005) Listening to talking faces: motor cortical activation during speech perception. Neuroimage 25: pp. 76-89 CrossRef
  62. Skipper, JI, Goldin-Meadow, S, Nusbaum, HC, Small, SL (2007) Speech-associated gestures, Broca鈥檚 area, and the human mirror system. Brain Lang 101: pp. 260-277 CrossRef
  63. Song, JJ, Choi, HG, Oh, SH, Chang, SO, Kim, CS, Lee, JH (2009) Unilateral sensorineural hearing loss in children: the importance of temporal bone computed tomography and audiometric follow-up. Otol Neurotol 30: pp. 604-608 CrossRef
  64. Song, JJ, Ridder, D, Heyning, P, Vanneste, S (2012) Mapping tinnitus-related brain activation: an activation-likelihood estimation metaanalysis of PET studies. J Nucl Med 53: pp. 1550-1557 CrossRef
  65. Song, JJ, Yoo, YT, An, YH, Yoo, JC, Kim, JS, Koo, JW (2012) Comorbid benign paroxysmal positional vertigo in idiopathic sudden sensorineural hearing loss: an ominous sign for hearing recovery. Otol Neurotol 33: pp. 137-141 CrossRef
  66. Song, JJ, Punte, AK, Ridder, D, Vanneste, S, Heyning, P (2013) Neural substrates predicting improvement of tinnitus after cochlear implantation in patients with single-sided deafness. Hear Res 299: pp. 1-9 CrossRef
  67. Song JJ, Vanneste S, Schlee W, Van de Heyning P, De Ridder D (2013b) Onset-related differences in neural substrates of tinnitus-related distress: the anterior cingulate cortex in late-onset tinnitus, and the frontal cortex in early-onset tinnitus. Brain Struct Funct [Epub ahead of print]
  68. Strelnikov, K, Rosito, M, Barone, P (2011) Effect of audiovisual training on monaural spatial hearing in horizontal plane. PLoS One 6: pp. e18344 CrossRef
  69. Strelnikov K, Rouger J, Demonet JF, Lagleyre S, Fraysse B, Deguine O, Barone P (2013) Visual activity predicts auditory recovery from deafness after adult cochlear implantation. Brain 136:3682鈥?695
  70. Sumby, WH, Pollack, I (1954) Visual contribution to speech intelligibility in noise. J Acoust Soc Am 26: pp. 212-215 CrossRef
  71. Szycik, GR, Jansma, H, Munte, TF (2009) Audiovisual integration during speech comprehension: an fMRI study comparing ROI-based and whole brain analyses. Hum Brain Mapp 30: pp. 1990-1999 CrossRef
  72. Talairach, J, Tornoux, P (1988) Co-planar stereotaxic atlas of the human brain: 3-dimensional proportional system: An approach to cerebral imaging. Georg Thieme, Stuttgart
  73. Talsma, D, Doty, TJ, Woldorff, MG (2007) Selective attention and audiovisual integration: is attending to both modalities a prerequisite for early integration?. Cereb Cortex 17: pp. 679-690 CrossRef
  74. Tremblay, C, Champoux, F, Lepore, F, Theoret, H (2010) Audiovisual fusion and cochlear implant proficiency. Restor Neurol Neurosci 28: pp. 283-291
  75. Atteveldt, N, Formisano, E, Goebel, R, Blomert, L (2004) Integration of letters and speech sounds in the human brain. Neuron 43: pp. 271-282 CrossRef
  76. Atteveldt, NM, Formisano, E, Goebel, R, Blomert, L (2007) Top-down task effects overrule automatic multisensory responses to letter-sound pairs in auditory association cortex. Neuroimage 36: pp. 1345-1360 CrossRef
  77. Wassenhove, V, Grant, KW, Poeppel, D (2005) Visual speech speeds up the neural processing of auditory speech. Proc Natl Acad Sci USA 102: pp. 1181-1186 CrossRef
  78. Vanneste, S, Plazier, M, Loo, E, Heyning, PV, Congedo, M, Ridder, D (2010) The neural correlates of tinnitus-related distress. Neuroimage 52: pp. 470-480 CrossRef
  79. Vigneau, M, Beaucousin, V, Herve, PY, Duffau, H, Crivello, F, Houde, O, Mazoyer, B, Tzourio-Mazoyer, N (2006) Meta-analyzing left hemisphere language areas: phonology, semantics, and sentence processing. Neuroimage 30: pp. 1414-1432 CrossRef
  80. Werner, S, Noppeney, U (2010) Distinct functional contributions of primary sensory and association areas to audiovisual integration in object categorization. J Neurosci 30: pp. 2662-2675 CrossRef
  81. Werner, S, Noppeney, U (2010) Superadditive responses in superior temporal sulcus predict audiovisual benefits in object categorization. Cereb Cortex 20: pp. 1829-1842 CrossRef
  82. Yun, RJ, Krystal, JH, Mathalon, DH (2010) Working memory overload: fronto-limbic interactions and effects on subsequent working memory function. Brain Imaging Behav 4: pp. 96-108 CrossRef
  
• 刊物主题：Neurosciences; Cell Biology; Neurology;
• 出版者：Springer Berlin Heidelberg
• ISSN：1863-2661

文摘

While deafness-induced plasticity has been investigated in the visual and auditory domains, not much is known about language processing in audiovisual multimodal environments for patients with restored hearing via cochlear implant (CI) devices. Here, we examined the effect of agreeing or conflicting visual inputs on auditory processing in deaf patients equipped with degraded artificial hearing. Ten post-lingually deafened CI users with good performance, along with matched control subjects, underwent H 2 15 O-positron emission tomography scans while carrying out a behavioral task requiring the extraction of speech information from unimodal auditory stimuli, bimodal audiovisual congruent stimuli, and incongruent stimuli. Regardless of congruency, the control subjects demonstrated activation of the auditory and visual sensory cortices, as well as the superior temporal sulcus, the classical multisensory integration area, indicating a bottom-up multisensory processing strategy. Compared to CI users, the control subjects exhibited activation of the right ventral premotor-supramarginal pathway. In contrast, CI users activated primarily the visual cortices more in the congruent audiovisual condition than in the null condition. In addition, compared to controls, CI users displayed an activation focus in the right amygdala for congruent audiovisual stimuli. The most notable difference between the two groups was an activation focus in the left inferior frontal gyrus in CI users confronted with incongruent audiovisual stimuli, suggesting top-down cognitive modulation for audiovisual conflict. Correlation analysis revealed that good speech performance was positively correlated with right amygdala activity for the congruent condition, but negatively correlated with bilateral visual cortices regardless of congruency. Taken together these results suggest that for multimodal inputs, cochlear implant users are more vision-reliant when processing congruent stimuli and are disturbed more by visual distractors when confronted with incongruent audiovisual stimuli. To cope with this multimodal conflict, CI users activate the left inferior frontal gyrus to adopt a top-down cognitive modulation pathway, whereas normal hearing individuals primarily adopt a bottom-up strategy.