Neural Substrates of Tonal Processing
By HaoWen Jiang
Languages that use pitch to distinguish lexical or grammatical meaning are called tone languages. Most languages in East Asia (e.g. Mandarin, Cantonese, Min, Hakka, Thai, Vietnamese, etc.) and sub-Saharan
Africa(e.g. Luo, Nubian, Zulu, etc.) are tonal. The auditory correlate of tone is pitch, and thus tone languages offer neuroscientists a good chance to investigate how speech prosody is processed in the human brain.
Most of the studies on tonal processing in the brain are done by Jackson T. Gandour from Department of Speech, Language, and Hearing Sciences at
as well as his colleagues and students. The survey presented here is mainly drawn from the following reviews of previous studies (in the order of relevance): Purdue University
l Gandour (2006) [click here to access] is an encyclopedic review of three types of studies on how linguistic tones are processed in the brain, including dichotic listening, lesion-deficit, and neuroimaging studies.
l Gandour (1998) [click here to access] is a thorough survey of aphasia literature in tone languages, especially in Mandarin, Thai, Cantonese, and Norwegian.
l Moen (2007) [click here to access] is a chapter in Clinical Aphasiology, reviewing studies on tone production and perception in brain-damaged patients, similar in spirits to Gandour (1998).
l Wong (2002) [click here to access] is a concise review of studies on hemispheric specialization of linguistic pitch patterns, with special reference to two hypotheses of how pitch patterns are lateralized.
Evidence from Dichotic Listening
Dichotic listening is a procedure used to test selective attention to auditory stimuli. In most cases, subjects are presented with two different auditory stimuli simultaneously, one in each ear, and then asked to report the stimuli they hear. For most right-handed people, they are more accurate in reporting stimuli presented to the right ear than those presented to the left ear, a phenomenon termed “Right Ear Advantage” (REA).
l Van Lancker (1980) [click here to access] is one of the seminal studies on dichotic listening of tones.
Natives speakers of Thai (experiment group)
Native speakers of English, musically untrained (control group)
Native speakers of English, musically trained (control group)
Tone words: Thai words distinguished by tone
Consonant words: Thai words distinguished by initial consonant
Hums: Hums distinguished by pitch homologous to tone words
The Thai group showed a significant REA for tone words and consonant words, but no ear advantage for hums.
The English group showed a REA for consonant words only, whether they are musically trained or untrained.
l Wang et al. (2001) [click here to access] is a more recent study on dichotic listening of tones, which shows results compatible with those found in Van Lancker’s (1980).
20 adult native speakers of Mandarin Chinese
20 adult natives speakers of American English
16 monosyllabic Mandarin words, consisting of 4 syllable types, with each type combined the 4 tones of Mandarin.
Overall error rates for the Chinese listeners (45%) were similar to those for the American listeners (46%).
The Chinese listeners showed a significant REA while the American listeners did not.
The implication of dichotic listening studies like Van Lancker (1980) and Wang et al. (2001) is that pitch perception is lateralized to the left hemisphere (as evidenced by Right Ear Advantage) when pitch variations bear language-specific meanings, such as lexical tones.
Evidence from Brain Damage
Although it is estimated that there are more than 1000 tone languages in Africa alone (Fromkin and Rodman 1993), current data on tonal breakdown come from only a few languages, particularly Mandarin, Cantonese, Thai, and Norwegian.
It is well-known that the left hemisphere is responsible for linguistic functions (such as phonological recognition and production) while the right hemisphere is closely associated with pitch and affective prosody. Since the primary auditory correlate of tones is pitch, an interesting question that may arise is “What is hemispheric specialization driven by?” The answer lies in lesion-deficit studies on tonal breakdown. If tonal deficits subsequent to right hemisphere (RH) lesions are observed, we may infer that hemispheric specialization is driven by acoustic cues regardless of function. On the other than, if tonal deficits subsequent to left hemisphere (LH) lesions are observed, we would have a good reason to believe that hemispheric specialization is driven by language function.
The major findings on the breakdown of tone production and perception are listed below, followed by studies that support the findings.
O Tones, just like consonants and vowels, may be disrupted subsequent to left-hemisphere damage.
l Packard (1986) [click here to access] demonstrates that the deficit experienced by LH-damaged nonfluent aphasic speakers of Chinese in tonal production is both quantitatively and qualitatively equivalent to that in consonant production.
O Damage to the right hemisphere does not impair tone production.
l Gandour et al. (1992) [click here to access] investigates the five lexical tones of Thai produced by patients with unilateral lesions in the left and right hemisphere.
10 young normal adults
10 old normal adults
11 right hemisphere nonaphasics
9 left hemisphere fluent aphasics
8 left hemisphere nonfluent aphasics
None of the RH-damaged patients had difficulty producing the five Thai tones.
The nonfluent apahsics (85%) performed significantly worse than the young normals (97%), old normals (96%), RH-damaged patients (93%), or fluent aphasics (96%).
O Tone perception deficits occur subsequent to aphasia in LH-damaged patients.
l Gandour & Dardarananda (1983) [click here to access] investigates the perception of the five tones in Thai by LH-damaged and RH-damaged patients.
4 LH-damaged aphasics (2 Broca’s, 1 transcortical motor, and 1 conduction)
1 RH-damaged nonaphasic
1 normal control
All LH-damaged aphasics performed significantly worse than either the RH-damaged nonaphasic or the normal control.
No significant differences were found in the performance of the RH-damaged nonaphasic and the normal control.
O Hemispheric lesions that cause aphasia give rise to difficulties in tone perception.
l Yiu & Fok (1995) [click here to access] is a larger project on perceptual impairment, which investigates tone disruption in Cantonese aphasic speakers, both fluent and nonfluent.
21 aphasics (11 anomic, 2 conduction, 2 Wernicke’s, 3 Broca’s, 4 transcortical motor)
8 normal controls
The aphasics (50%) performed significantly worse than either the normals (93%) or dysarthrics (91%).
No significant differences were found in the performance of nonflluent and fluent aphasics.
On the whole, both production and perception deficits of the pitch patterns in tone languages result from lesions to the left hemisphere, not to the right hemisphere. The following quote from Gandour (1998: 135) presents a good summary of the studies on tonal breakdown.
“From this survey of contemporary aphasia literature, we are led to conclude that language representation in the brains of tone language speakers is essentially the same as that in nontone language speakers.”
Evidence from Neuroimaging
Neuroimaging techniques make it possible to assess what areas of a well-functioning brain in either hemisphere participate in tonal processing, something that neither dichotic listening nor aphasia studies could possibly achieve. In the past two decades, a variety of studies have been done using Positron Emission Tomography (PET) and functional Magnetic Resonance Imaging (fMRI).
The following case studies are intended to illustrate the broad spectrum of research on tonal processing, but they are not meant to be exhaustive.
Tones vs. Consonants/Vowels
l Hsieh et al. (2001) [click here to access] is a PET study of the perceptual processing of Mandarin consonants, vowels, and tones.
10 adult native speakers of Mandarin Chinese (native listeners)
10 adult native speakers of American English (nonnative listeners)
Speech stimuli: 45 strings of 5 monosyllabic Mandarin morphemes, with no two adjacent morphemes in a string being a lexical item in Mandarin
Nonspeech stimuli: low-pass-filtered version of the same speech stimuli
Passive task: listen to nonspeech stimuli
Active task: listen to stimuli and make (same or different) judgments of the first and last morpheme in a string of stimuli, with respect to consonants, vowels, and tones for the speech stimuli and with respect to pitch for the nonspeech stimuli
The Mandarin group showed increased activity in left premotor cortex, pars opercularis, and pars triangularis across the four tasks (i.e. consonant, vowel, tone, and pitch task).
The English group showed increased activity in left inferior frontal gyrus (IFG) only in the vowel task and in right IFG in the pitch task.
The implication of studies like this is that only when the pitch patterns are phonologically significant to the listener is pitch processing lateralized to the left hemisphere; otherwise, it is lateralized to the right hemisphere. In addition, left hemisphere plays a crucial role in the processing of linguistic information, whether the phonological unit is segmental (i.e. vowels and consonants) or suprasegmental (tones in this case).
Tones vs. Vowel Length
l Gandour et al. (2002) [click here to access] is an fMRI study of the perceptual processing of Thai tones and vowel length.
10 native speakers of Thai (native listeners)
10 native speakers of Mandarin (nonnative listeners)
Speech stimuli: 40 pairs of monosyllabic Thai pseudowords, distinguished by tones or vowel length
Nonspeech stimuli: synthesized hums of the same speech stimuli
Passive task: listen to nonspeech stimuli
Active task: listen to and make (same or different) judgments of a pair of stimuli, with respect to tones and vowel length for the speech stimuli and with respect to pitch contours and duration patterns for the nonspeech stimuli
Only the Thai group showed activation in the left inferior prefrontal cortex in speech minus nonspeech contrasts for spectral (i.e. tone minus pitch) and temporal (i.e. vowel length minus duration) cues.
Both the Thai and Chinese groups showed similar fronto-parietal activation patterns in nonspeech hums minus passive listening contrasts for spectral (i.e. pitch minus passive listening) and temporal (i.e. duration minus passive listening) cues.
The implication of studies like this is that lower level specialization for acoustic cues (e.g. pitch contours and duration patterns) in the spectral and temporal domains cannot be generalized to higher level of phonological processing (e.g. tones and vowel length). In other words, despite the fact that the neural mechanisms underlying low-level auditory processing are universal, hemispheric specialization is sensitive to language-specific factors.
Native Tones vs. Non-native Tones
l Xu et al (2006) [click here to access] is an fMRI study of the perceptual processing of native tones and non-native tones, using an innovative design of hybrid stimuli which result from superimposing the tones in one language onto the syllables in another.
10 native speakers of Mandarin
10 native speakers of Thai
Speech stimuli: 2 identical sets of 40 nonminimal pairs of Mandarin syllables with 80 unique onset-rhyme combinations, one set superimposed with the four Mandarin tones (Chinese words) and the other set with the five Thai tones (tonal chimeras)
Nonspeech stimuli: synthesized hums of the same speech stimuli
Passive task: listen to nonspeech stimuli
Active task: listen to and make (same or different) judgments of a pair of speech stimuli with respect to tones
Two overlapping areas of activation between the Chinese and the Thai groups were identified, one in the anterior-medial portion of the left planum temporale (PT) and the other in the left ventral precentral gyrus (PrCG).
Within left PT, tonal chimeras (Mandarin syllables with Thai tones) elicit stronger activity relative to Chinese words for the Thai group whereas Chinese words elicit stronger activity relative to tonal chimeras for the Chinese group.
The implication of studies like this is that cortical processing of pitch information can be shaped by language experience (as evidenced by the double dissociation within PT between language experience and neural processing of pitch). Moreover, while the overlapping activation in PrCG can be attributed to the motor system at work in speech perception, lateralized PT activation may suggest that the left PT is a computational hub associated with phonological representations of discrete tonal categories.
Tones vs. Second Language Acquisition
l Wang et al. (2003) [click here to access] is an fMRI study of cortical changes in the brain of English speakers during their early stages of learning Mandarin tones.
6 college-age native speakers of American English (all right-handed)
8 sessions of training, each being 40 minutes long
Training stimuli: 180 monosyllabic Mandarin real words
Task stimuli: 40 monosyllabic Mandarin real words
Identify auditory presentations of the task stimuli and indicate with a pointer secured to the right hand the corresponding tone mark located on a viewing screen
Two identical fMRI scans: an initial scan prior to the training and a follow-up scan 2 weeks after the training
Across all subjects, improvements in performance were associated with an expansion of activation in left superior temporal gyrus (STG, or BA 22), and with the emergence of activity in adjacent Heschl’s gyrus (HG, or BA 42) as well as in right inferior frontal gyrus (IFG, BA 44).
The implication of studies like this is that the early cortical effects of learning tone languages as a second language involve both expansion of preexisting language-related areas (e.g. BA 22) and recruitment of neighboring areas (e.g. BA 42). In addition, speakers of nontone languages acquire tonal differentiations by enhancing an existing area specialized for the processing of pitch (i.e. RH BA 44).
ReferencesGo to Menu (for non-frame version.)
- Fromkin, V., and R. Rodman. 1993. An Introduction to Language (5th ed.). New York: Harcourt Brace Jovanovich.
- Gandour, J. 1998. Aphasia in tone languages. Aphasia in atypical populations: 117-141.
- Gandour, J. 2006. Tone: neurophonetics. Encyclopedia of language and linguistics (ed. K. Brown): 751–760.
- Gandour, J., and R. Dardarananda. 1983. Identification of tonal contrasts in Thai aphasic patients. Brain and Language 18: 98-114.
- Gandour, J., S. Ponglorpisit, F. Khunadorn, S. Dechongkit, P. Boongird, R. Boonklam, and S. Potisuk. 1992. Lexical tones in Thai after unilateral brain damage. Brain and language 43, no. 2: 275.
- Gandour, J., D. Wong, M. Lowe, M. Dzemidzic, N. Satthamnuwong, Y. Tong, and X. Li. 2002. A cross-linguistic fMRI study of spectral and temporal cues underlying phonological processing. Journal of cognitive neuroscience 14, no. 7: 1076-1087.
- Hsieh, L., J. Gandour, D. Wong, and G. D. Hutchins. 2001. Functional heterogeneity of inferior frontal gyrus is shaped by linguistic experience. Brain and Language 76, no. 3: 227-252.
- Moen, I. 2007. Production and perception of word tones in patients with brain damage. Clinical Aphasiology: Future Directions : 125-135.
- Packard, J. L. 1986. Tone production deficits in nonfluent aphasic Chinese speech. Brain and Language 29, no. 2: 212.
- Van Lancker, D. 1980. Cerebral lateralization of pitch cues in the linguistic signal. Papers in Linguistics Edmonton 13, no. 1-2: 201-277.
- Wang, Y., A. Jongman, and J. A. Sereno. 2001. Dichotic perception of Mandarin tones by Chinese and American listeners. Brain and Language 78, no. 3: 332-348.
- Wang, Y., J. A. Sereno, A. Jongman, and J. Hirsch. 2003. fMRI evidence for cortical modification during learning of Mandarin lexical tone. Journal of Cognitive Neuroscience 15, no. 7: 1019-1027.
- Wong, P. C. M. 2002. Hemispheric specialization of linguistic pitch patterns. Brain Research Bulletin 59, no. 2: 83-95.
- Xu, Y., J. Gandour, T. Talavage, D. Wong, M. Dzemidzic, Y. Tong, X. Li, and M. Lowe. 2006. Activation of the left planum temporale in pitch processing is shaped by language experience. Human Brain Mapping 27, no. 2.
- Yiu, E. M. L., and A. Y. Y. Fok. 1995. Lexical tone disruption in Cantonese aphasic speakers. Clinical Linguistics & Phonetics 9, no. 1: 79-92.
Copyrighted2009 Rice University. This document, or any portion hereof, may be used for non-commercial informational purposes only. Any copy of this document, or portion hereof, must include the copyright notice (http://www.rice.edu/about/cr-notice.html).