Speech-Language Pathologists' Ratings of Speech Accuracy in Children With Speech Sound Disorders
Publication: American Journal of Speech-Language Pathology
Volume 31, Number 1
Pages 419-430
Abstract
Purpose:
Forming accurate and consistent speech judgments can be challenging when working with children with speech sound disorders who produce a large number and varied types of error patterns. Rating scales offer a systematic approach to assessing the whole word rather than individual sounds. Thus, these scales can be an efficient way for speech-language pathologists (SLPs) to monitor treatment progress. This study evaluated the interrater reliability of an existing 3-point rating scale using a large group of SLPs as raters.
Method:
Utilizing an online platform, 30 SLPs completed a brief training and then rated single words produced by children with typical speech patterns and children with speech sound disorders. Words were closely balanced across the three rating categories of the scale. The interrater reliability of the SLPs ratings to a consensus judgment was examined.
Results:
The majority of SLPs (87%) reached substantial interrater reliability to a consensus judgment using the 3-point rating scale. Correct productions had the highest interrater reliability. Productions with extensive errors had higher agreement than those with minor errors. Certain error types, such as vowel distortions, were especially challenging for SLPs to judge.
Conclusions:
This study demonstrated substantial interrater reliability to a consensus judgment among a large majority of 30 SLPs using a 3-point rating. The clinical implications of the findings are discussed along with proposed modifications to the training procedure to guide future research.
Get full access to this article
View all available purchase options and get full access to this article.
References
Allison, K. M., Russell, M., & Hustad, K. C. (2021). Reliability of perceptual judgments of phonetic accuracy and hypernasality among speech-language pathologists for children with dysarthria. American Journal of Speech-Language Pathology, 30(3S), 1558–1571.
American Speech-Language-Hearing Association. (2017). Childhood apraxia of speech. https://www.asha.org/public/speech/disorders/Childhood-Apraxia-of-Speech/
Amorosa, H., von Benda, U., Wagner, E., & Keck, A. (1985). Transcribing phonetic detail in the speech of unintelligible children: A comparison of procedures. British Journal of Disorders of Communication, 20(3), 281–287.
Baayen, R. H. (2008). Analyzing linguistic data: A practical introduction to statistics using R. Cambridge University Press.
Bates, D., Maechler, M., Bolker, B., & Walker, S. (2015). Lme4: Linear mixed-effects models using Eigen and S4. R package version 1.1–7. 2014.
Baum, S. R., & McNutt, J. C. (1990). An acoustic analysis of frontal misarticulation of /s/ in children. Journal of Phonetics, 18(1), 51–63.
Becker, M., & Levine, J. (2010). Experigen–an online experiment platform.
Buckingham, H. W., & Yule, G. (1987). Phonemic false evaluation: Theoretical and clinical aspects. Clinical Linguistics & Phonetics, 1(2), 113–125.
Cole, R. A. (1973). Listening for mispronunciations: A measure of what we hear during speech. Perception & Psychophysics, 13(1), 153–156.
Ferber, R. (1991). Slip of the tongue or slip of the ear? On the perception and transcription of naturalistic slips of the tongue. Journal of Psycholinguistic Research, 20(2), 105–122.
Franklin, L., & Morgan, J. (2017). On the nature of vocalic representation during lexical access. The Journal of the Acoustical Society of America, 141(5), 4038–4038.
Fromkin, V. (1980). Errors in linguistic performance: Slips of the tongue, ear, pen, and hand. Academic Press.
Garnes, S., & Bond, Z. S. (1975). Slips of the ear: Errors in perception of casual speech. Chicago Linguistic Society, 11, 214–225.
Grigos, M. I., Moss, A., & Lu, Y. (2015). Oral articulatory control in childhood apraxia of speech. Journal of Speech, Language, and Hearing Research, 58(4), 1103–1118.
Harel, D., & McAllister, T. (2019). Multilevel models for communication sciences and disorders. Journal of Speech, Language, and Hearing Research, 62(4), 783–801.
Howard, S. J., & Heselwood, B. C. (2002). Learning and teaching phonetic transcription for clinical purposes. Clinical Linguistics & Phonetics, 16(5), 371–401.
Kewley-Port, D., Burkle, T. Z., & Lee, J. H. (2007). Contribution of consonant versus vowel information to sentence intelligibility for young normal-hearing and elderly hearing-impaired listeners. The Journal of the Acoustical Society of America, 122(4), 2365–2375.
Klein, H. B., Grigos, M. I., McAllister Byun, T., & Davidson, L. (2012). The relationship between inexperienced listeners' perceptions and acoustic correlates of children's /r/ productions. Clinical Linguistics & Phonetics, 26(7), 628–645.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174.
Maas, E., Butalla, C. E., & Farinella, K. A. (2012). Feedback frequency in treatment for childhood apraxia of speech. American Journal of Speech-Language Pathology, 21(3), 239–257.
Maas, E., & Farinella, K. A. (2012). Random versus blocked practice in treatment for childhood apraxia of speech. Journal of Speech, Language, and Hearing Research, 55(2), 561–578.
Maas, E., Robin, D. A., Wright, D. L., & Ballard, K. J. (2008). Motor programming in apraxia of speech. Brain and Language, 106(2), 107–118.
Miller, G. A., & Nicely, P. E. (1955). An analysis of perceptual confusions among some English consonants. The Journal of the Acoustical Society of America, 27(2), 338–352.
Pollock, K. E., & Hall, P. K. (1991). An analysis of the vowel misarticulations of five children with developmental apraxia of speech. Clinical Linguistics & Phonetics, 5(3), 207–224.
R Core Team (2014). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. http://www.R-project.org/
Shriberg, L. D., Kwiatkowski, J., & Hoffmann, K. (1984). A procedure for phonetic transcription by consensus. Journal of Speech and Hearing Research, 27(3), 456–465.
Shriberg, L. D., & Lof, G. L. (1991). Reliability studies in broad and narrow phonetic transcription. Clinical Linguistics & Phonetics, 5(3), 225–279.
Shriberg, L. D., Strand, E. A., Fourakis, M., Jakielski, K. J., Hall, S. D., Karlsson, H. B., Mabie, H. L., McSweeny, J. L., Tilkens, C. M., & Wilson, D. L. (2017). A diagnostic marker to discriminate childhood apraxia of speech from speech delay: I. Development and description of the pause marker. Journal of Speech, Language, and Hearing Research, 60(4), S1096–S1117.
Shuster, L. I., & Wambaugh, J. L. (2000). Perceptual and acoustic analyses of speech sound errors in apraxia of speech accompanied by aphasia. Aphasiology, 14(5–6), 635–651.
Strand, E. A. (2020). Dynamic temporal and tactile cueing: A treatment strategy for childhood apraxia of speech. American Journal of Speech-Language Pathology, 29(1), 30–48.
Strand, E. A., Stoeckel, R., & Baas, B. (2006). Treatment of severe childhood apraxia of speech: A treatment efficacy study. Journal of Medical Speech-Language Pathology, 14(4), 297–307.
Tent, J., & Clark, J. E. (1980). An experimental investigation into the perception of slips of the tongue. Journal of Phonetics, 8(3), 317–325.
Vanbelle, S. (2016). A new interpretation of the weighted kappa coefficients. Psychometrika, 81(2), 399–410.
Warren, R. M. (1970). Perceptual restoration of missing speech sounds. Science, 167(3917), 392–393.
Warren, R. M. (1984). Perceptual restoration of obliterated sounds. Psychological Bulletin, 96(2), 371–383.
Warren, R. M., & Sherman, G. L. (1974). Phonemic restorations based on subsequent context. Perception & Psychophysics, 16(1), 150–156.
White, K. S., Yee, E., Blumstein, S. E., & Morgan, J. L. (2013). Adults show less sensitivity to phonetic detail in unfamiliar words, too. Journal of Memory and Language, 68(4), 362–378.
Information & Authors
Information
Published In
American Journal of Speech-Language Pathology
Volume 31 • Number 1 • January 2022
Pages: 419-430
PubMed: 34788553
Copyright
Copyright © 2021 American Speech-Language-Hearing Association.
History
- Received: Dec 8, 2020
- Revised: Jun 1, 2021
- Accepted: Aug 30, 2021
- Published online: Nov 17, 2021
- Published in issue: Jan 18, 2022
Authors
Metrics & Citations
Metrics
Article Metrics
View all metricsCitations
Download article citation data for:
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
Citing Literature
- Upeka De Silva, Samaneh Madanian, Sharon Olsen, John Michael Templeton, Christian Poellabauer, Sandra L Schneider, Ajit Narayanan, Rahmina Rubaiat, Clinical Decision Support Using Speech Signal Analysis: Systematic Scoping Review of Neurological Disorders, Journal of Medical Internet Research, 10.2196/63004, 27, (e63004), (2025).
- Do Hyung Kim, Joo Won Jeong, Dayoung Kang, Taekyung Ahn, Yeonjung Hong, Younggon Im, Jaewon Kim, Min Jung Kim, Dae-Hyun Jang, Usefulness of Automatic Speech Recognition Assessment of Children With Speech Sound Disorders: Validation Study, Journal of Medical Internet Research, 10.2196/60520, 27, (e60520), (2025).
- Sofia Strömbergsson, Molly Fröjdh, Magdalena Pettersson, Tamás Grósz, Yaroslav Getman, Mikko Kurimo, Listening like a speech-training app: Expert and non-expert listeners’ goodness ratings of children’s speech, Clinical Linguistics & Phonetics, 10.1080/02699206.2024.2355470, 39, 2, (144-165), (2024).
- Julie Case, Susan Caspari, Pooja Aggarwal, Ruth Stoeckel, A Goal-Writing Framework for Motor-Based Intervention for Childhood Apraxia of Speech, American Journal of Speech-Language Pathology, 10.1044/2024_AJSLP-24-00014, 33, 4, (1590-1607), (2024).
- Dana Gabriela Budală, Costin Iulian Lupu, Roxana Ionela Vasluianu, Nicoleta Ioanid, Oana Maria Butnaru, Elena-Raluca Baciu, A Contemporary Review of Clinical Factors Involved in Speech-Perspectives from a Prosthodontist Point of View, Medicina, 10.3390/medicina59071322, 59, 7, (1322), (2023).
- Julie Case, Emily W. Wang, Maria I. Grigos, The Multilevel Word Accuracy Composite Scale: A Novel Measure of Speech Production in Childhood Apraxia of Speech, American Journal of Speech-Language Pathology, 10.1044/2023_AJSLP-22-00166, 32, 4S, (1866-1883), (2023).
- Kate Margetson, Sharynne McLeod, Sarah Verdon, Van H. Tran, Transcribing multilingual children’s and adults’ speech, Clinical Linguistics & Phonetics, 10.1080/02699206.2022.2051073, 37, 4-6, (415-435), (2022).
- Seyoung Jung, Linye Jing, Maria Grigos, Graduate Student Clinicians' Perceptions of Child Speech Sound Errors, Perspectives of the ASHA Special Interest Groups, 10.1044/2022_PERSP-21-00332, 7, 4, (1275-1283), (2022).
View Options
Sign In Options
ASHA member? If so, log in with your ASHA website credentials for full access.
Member Login