A Motor Speech Assessment for Children With Severe Speech Disorders: Reliability and Validity Evidence
Publication: Journal of Speech, Language, and Hearing Research
Volume 56, Number 2
Pages 505-520
Abstract
Purpose
In this article, the authors report reliability and validity evidence for the Dynamic Evaluation of Motor Speech Skill (DEMSS), a new test that uses dynamic assessment to aid in the differential diagnosis of childhood apraxia of speech (CAS).
Method
Participants were 81 children between 36 and 79 months of age who were referred to the Mayo Clinic for diagnosis of speech sound disorders. Children were given the DEMSS and a standard speech and language test battery as part of routine evaluations. Subsequently, intrajudge, interjudge, and test–retest reliability were evaluated for a subset of participants. Construct validity was explored for all 81 participants through the use of agglomerative cluster analysis, sensitivity measures, and likelihood ratios.
Results
The mean percentage of agreement for 171 judgments was 89% for test–retest reliability, 89% for intrajudge reliability, and 91% for interjudge reliability. Agglomerative hierarchical cluster analysis showed that total DEMSS scores largely differentiated clusters of children with CAS vs. mild CAS vs. other speech disorders. Positive and negative likelihood ratios and measures of sensitivity and specificity suggested that the DEMSS does not overdiagnose CAS but sometimes fails to identify children with CAS.
Conclusions
The value of the DEMSS in differential diagnosis of severe speech impairments was supported on the basis of evidence of reliability and validity.
Get full access to this article
View all available purchase options and get full access to this article.
References
American Speech-Language-Hearing Association. (2007). Childhood apraxia of speech: Nomenclature, definition, roles and responsibilities, and a call for action [Position statement]. Rockville, MD: Author.
Arndt, W. B., Shelton, R. L., Johnson, A. F., & Furr, M. L. (1977). Identification and description of homogeneous subgroups within a sample of misarticulating children. Journal of Speech and Hearing Research, 20, 263–292.
Bain, B. A. (1994). A framework for dynamic assessment in phonology: Stimulability revisited. Clinics in Communication Disorders, 4, 12–22.
Ball, L. J., Bernthal, J. E., & Beukelman, D. R. (2002). Profiling communication characteristics of children with developmental apraxia of speech. Journal of Medical Speech-Language Pathology, 10, 221–229.
Campbell, T. (2003). Childhood apraxia of speech: Clinical symptoms and speech characteristics. InShriberg, L. D., & Campbell, T. F. (Eds.), Proceedings of the 2002 Childhood Apraxia of Speech Symposium (pp. 37–40). Carlsbad, CA: The Hendrix Foundation.
Carrow-Woolfolk, E. (1997). Oral & Written Language Scales. Circle Pines, MN: AGS.
Caruso, A., & Strand, E. (1999). Motor speech disorders in children: Definitions, background and a theoretical framework. InCaruso, A., & Strand, E. A. (Eds.), Clinical management of motor speech disorders in children (pp. 1–27). New York, NY: Thieme.
Conti-Ramsden, G., & Botting, N. (1999). Classification of children with specific language impairment: Longitudinal considerations. Journal of Speech, Language, and Hearing Research, 42, 1195–1204.
Crary, M. A. (1993). Developmental motor speech disorders. San Diego, CA: Singular.
Davis, B. L., Jakielski, K. J., & Marquardt, T. P. (1998). Developmental apraxia of speech: Determiners of differential diagnosis. Clinical Linguistics & Phonetics, 12, 25–45.
DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988). Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach. Biometrics, 44, 837–845.
Dollaghan, C. (2007). The handbook for evidence-based practice in communication disorders. Baltimore, MD: Brookes.
Downing, S. M., & Haladyna, T. M. (2006). Handbook of test development. Mahwah, NJ: Erlbaum.
Duffy, J. (2005). Motor speech disorders: Substrates, differential diagnosis and management ((2nd ed.)). St. Louis, MO: Elsevier Mosby.
Dunn, L., & Dunn, L. (1997). Peabody Picture Vocabulary Test—III. Circle Pines, MN: AGS.
Glaspey, A., & Stoel-Gammon, C. (2007). A dynamic approach to phonological assessment. Advances in Speech Language Pathology, 9, 286–296.
Goffman, L. (2004). Assessment and classification: An integrative model of language and motor contributions to phonological development. InKamhi, A., & Pollock, K. (Eds.), Phonological disorders in children: Clinical decision making in assessment and intervention (pp. 51–64). Baltimore, MD: Brookes.
Goldman, R., & Fristoe, M. (2000). Goldman Fristoe Test of Articulation—Second Edition. Circle Pines, MN: AGS.
Guyette, T. W. (2001). Review of the Apraxia profile. InPlake, B. S., & Impara, J. C. (Eds.), The fourteenth mental measurements yearbook (pp. 57–58). Lincoln, NE: Buros Institute of Mental Measurements.
Hastie, T., Tibshirani, R., & Friedman, J. H. (2003). The elements of statistical learning. New York, NY: Springer-Verlag.
Hayden, D., & Square, P. (1999). Verbal Motor Production Assessment for Children. San Antonio, TX: Pearson, PsychCorp.
Johnson, A. F., Shelton, R. L., & Arndt, W. B. (1982). A technique for identifying the subgroup membership of certain misarticulating children. Journal of Speech and Hearing Research, 25, 162–166.
Kehoe, M., Stoel-Gammon, C., & Buder, E. H. (1995). Acoustic correlates of stress in young children's speech. Journal of Speech and Hearing Research, 38, 338–350.
Kent, R. D. (2004). Models of speech motor control: Implications from recent developments in neurophysiological and neurobehavioral science. InMaassen, B., Kent, R. D., Peters, H. F. M., van Lieshout, P. H. M., & Hultsijn, W. (Eds.), Speech motor control in normal and disordered speech (pp. 3–28). London, England: Oxford University Press.
Kent, R. D., Kent, J., & Rosenbek, J. (1987). Maximal performance tests of speech production. Journal of Speech and Hearing Disorders, 52, 367–387.
Lidz, C. S., & Peña, E. D. (1996). Dynamic assessment: The model, its relevance as a non-biased approach and its application to Latino American preschool children. Language, Speech, and Hearing Services in Schools, 27, 367–372.
McCauley, R. J. (2001). Assessment of language disorders in children. Mahwah, NJ: Erlbaum.
McCauley, R. J. (2003). Review of Screening Test for Developmental Apraxia of Speech(2nd ed.)Plake, B., Impara, J. C., & Spies, R. A. (Eds.), The fifteenth mental measurements yearbook (pp. 786–789). Austin, TX: Pro-Ed.
McCauley, R. J., & Strand, E. A. (2008). A review of standardized tests of nonverbal oral and speech motor performance in children. American Journal of Speech-Language Pathology, 17, 1–11.
McLeod, S., Harrison, L. J., & McCormack, J. (2012). The intelligibility in context scale: Validity and reliability of a subjective rating measure. Journal of Speech, Language, and Hearing Research, 55, 648–656.
McNeil, M. R., Robin, D. A., & Schmidt, R. A. (2009). Apraxia of speech: Definition, differential, and treatment. InMcNeil, M. (Ed.), Clinical management of sensorimotor speech disorders ((2nd ed., pp. 249–268). New York, NY: Thieme.
Munson, B., Byorum, E., & Windsor, J. (2003). Acoustic and perceptual correlates of stress in nonwords produced by children with suspected developmental apraxia of speech and children with phonological disorder. Journal of Speech, Language, and Hearing Research, 46, 189–202.
Nijland, L., Maassen, B., & van der Meulen, S. (2003). Evidence of motor programming deficits in children with DAS. Journal of Speech, Language, and Hearing Research, 46, 437–450.
Odekar, A., & Hollowell, B. (2005). Comparison of alternatives to multidimensional scoring in the assessment of language comprehension in aphasia. American Journal of Speech-Language Pathology, 14, 337–345.
Odell, K. H., & Shriberg, L. D. (2001). Prosody-voice characteristics of children and adults with apraxia of speech. Clinical Linguistics & Phonetics, 15, 275–307.
Ozanne, A. (1995). The search for developmental verbal dyspraxia. InDodd, B. (Ed.), Differential diagnosis and treatment of children with speech disorder (pp. 91–101). San Diego, CA: Singular.
Peña, E. R., Gillam, R. B., Malek, M., Ruez-Felter, R., Recendez, M., & Sabel, T. (2006). Dynamic assessment of school-age children's narrative ability: An investigation of reliability and validity. Journal of Speech, Language, and Hearing Research, 49, 1037–1057.
Peter, B., & Stoel-Gammon, C. (2008). Central timing deficits in subtypes of primary speech disorders. Clinical Linguistics & Phonetics, 22, 171–198.
Pinheiro, J. C., & Bates, D. M. (2002). Mixed effects models in S and S-Plus. New York, NY: Springer.
Rvachew, S., Hodge, M., & Ohberg, A. (2005). Obtaining and interpreting maximum performance tasks from children: A tutorial. Journal of Speech-Language Pathology and Audiology, 29, 146–157.
Shriberg, L. (2003). Diagnostic markers for child speech-sound disorders: Introductory comments. Clinical Linguistics & Phonetics, 17, 501–505.
Shriberg, L. D., Aram, D. M., & Kwiatkowski, J. (1997). Developmental apraxia of speech: II. Toward a diagnostic marker. Journal of Speech, Language, and Hearing Research, 40, 286–312.
Shriberg, L. D., Campbell, T. F., Karlsson, H. B., Brown, R. L., McSweeny, J. L., & Nadler, C. J. (2003). A diagnostic marker for childhood apraxia of speech: The lexical stress ratio. Clinical Linguistics & Phonetics, 17, 549–574.
Shriberg, L. D., Lohmeier, H. L., Strand, E. A., & Jakielski, K. J. (2012). Encoding, memory, and transcoding deficits in childhood apraxia of speech. Clinical Linguistics & Phonetics, 26, 445–482.
Skahan, S. M., Watson, M., & Lof, G. L. (2007). Speech-language pathologists' assessment practices for children with suspected speech sound disorders: Results of a national survey. American Journal of Speech-Language Pathology, 16, 246–249.
Smith, A., & Goffman, L. (2004). Interaction of motor and linguistic factors in the development of speech production. InMaassen, B., Kent, R., Peters, H., van Lieshout, P., & Hulstijn, W. (Eds.), Speech motor control in normal and disordered speech (pp. 225–252). London, England: Oxford University Press.
Strand, E. A. (1992). The integration of motor-speech processes and language formulation in process models of language acquisition. InChapman, R. S. (Ed.), Child talk: Processes in language acquisition and disorder (pp. 86–107). St. Louis, MO: Mosby.
Strand, E. A. (2003). Childhood apraxia of speech: Suggested diagnostic markers for the younger child. InShriberg, L. D., & Campbell, T. F. (Eds.), Proceedings of the 2002 Childhood Apraxia of Speech Symposium (pp. 75–79). Carlsbad, CA: The Hendrix Foundation.
Strand, E. A., & McCauley, R. J. (1999). Assessment procedures for treatment planning in children with phonologic and motor speech disorders. InCaruso, A., & Strand, E. A. (Eds.), Clinical management of motor speech disorders of children (pp. 73–108). New York, NY: Thieme.
Streiner, D. L., & Norman, G. R. (2003). Health measurement scales: A practical guide to their development and use ((3rd ed.)). New York, NY: Oxford University Press.
Terband, H., & Maassen, B. (2010). Speech motor development in childhood apraxia of speech: Generating testable hypotheses by neurocomputational modeling. Folia Phoniatrica et Logopaedica, 62, 134–142.
Terband, H., Maassen, B., Guenther, F., & Brumberg, J. (2009). Computational neural modeling of speech motor control in childhood apraxia of speech (CAS). Journal of Speech, Language, and Hearing Research, 52, 1595–1609.
Thoonen, G., Maassen, B., Gabreels, F., & Schreuder, R. (1999). Validity of maximum performance tasks to diagnose motor speech disorders in children. Clinical Linguistics & Phonetics, 13, 1–23.
Thoonen, G., Maassen, B., Wit, J., Gabreels, F., & Schreuder, R. (1996). The integrated use of maximum performance tasks in differential diagnostic evaluations among children with motor speech disorders. Clinical Linguistics & Phonetics, 10, 311–336.
Tibshirani, R., Walther, G., & Hastie, T. (2001). Estimating the number of clusters in a data set via the gap statistic. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 63, 411–423.
Verté, S., Geurts, H. M., Roeyers, H., Rosseel, Y., Oosterlaan, J., & Sergeant, J. A. (2006). Can the Children's Communication Checklist differentiate autism spectrum subtypes?. Autism, 10, 266–287.
Wetherby, A. M., Allen, L., Cleary, J., Kublin, K., & Goldstein, H. (2002). Validity and reliability of the Communication and Symbolic Behavior Scales Developmental Profile with very young children. Journal of Speech, Language, and Hearing Research, 45, 1202–1218.
Yorkston, K., Beukelman, D., Strand, E. A., & Hackel, M. (2010). Management of motor speech disorders in children and adults. Austin, TX: Pro-Ed.
Zimmerman, L. L., Steiner, V. G., & Pond, R. E. (2002). Preschool Language Scales (4th ed.). San Antonio, TX: Pearson PsychCorp.
Information & Authors
Information
Published In
Copyright
Copyright © 2013 American Speech-Language-Hearing Association.
History
- Received: Mar 24, 2012
- Revised: Jul 9, 2012
- Accepted: Aug 18, 2012
- Published in issue: Apr 1, 2013
Keywords
Authors
Metrics & Citations
Metrics
Article Metrics
View all metricsCitations
Download article citation data for:
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
Citing Literature
- Asif Maitheen, Divya Seth, Sudhin Karuppali, Perspectives of Indian Speech-language Pathologists towards the assessment of Childhood Apraxia of Speech, Speech, Language and Hearing, 10.1080/2050571X.2025.2474840, 28, 1, (2025).
- Petri Partanen, Anna Eva Hallin, Exploring the need for dynamic assessment in the diagnosis of children with suspected language disorders, Child Language Teaching and Therapy, 10.1177/02656590251326275, (2025).
- Marju Lahtein-Kürsa, Marika Padrik, Simona Daniutė, Daiva Kairienė, Anna-Leena Martikainen, Minna Vanhala-Haukijärvi, Marja-Liisa Mailend, Diagnostic Features of Childhood Apraxia of Speech: A Survey Study of Estonian, Finnish, and Lithuanian Speech-Language Pathologists, American Journal of Speech-Language Pathology, 10.1044/2024_AJSLP-24-00035, 34, 1, (97-117), (2024).
- Andrea Burgess, Carly Luke, Michelle Jackman, Jane Wotherspoon, Koa Whittingham, Katherine Benfer, Sarah Goodman, Rebecca Caesar, Tiffney Nesakumar, Samudragupta Bora, David Honeyman, Danielle Copplin, Sarah Reedman, John Cairney, Natasha Reid, Leanne Sakzewski, Roslyn N. Boyd, Clinical utility and psychometric properties of tools for early detection of developmental concerns and disability in young children: A scoping review, Developmental Medicine & Child Neurology, 10.1111/dmcn.16076, (2024).
- Patricia McCabe, Johanna Korkalainen, Donna Thomas, Diagnostic Uncertainty in Childhood Motor Speech Disorders: A Review of Recent Tools and Approaches, Current Developmental Disorders Reports, 10.1007/s40474-024-00295-x, 11, 3, (105-112), (2024).
- Ahmed Alduais, Hind Alfadda, Childhood Apraxia of Speech: A Descriptive and Prescriptive Model of Assessment and Diagnosis, Brain Sciences, 10.3390/brainsci14060540, 14, 6, (540), (2024).
- Julie Case, Susan Caspari, Pooja Aggarwal, Ruth Stoeckel, A Goal-Writing Framework for Motor-Based Intervention for Childhood Apraxia of Speech, American Journal of Speech-Language Pathology, 10.1044/2024_AJSLP-24-00014, 33, 4, (1590-1607), (2024).
- Margaret Kehoe, Mélodie Matrat, Dynamic assessment in phonology, Multilingual Acquisition and Learning, 10.1075/sibil.67.11keh, (295-323), (2024).
- Maria I. Grigos, Julie Case, Ying Lu, Zhuojun Lyu, Dynamic Temporal and Tactile Cueing in Young Children With Childhood Apraxia of Speech: A Multiple Single-Case Design, Journal of Speech, Language, and Hearing Research, 10.1044/2024_JSLHR-23-00415, 67, 4, (1042-1071), (2024).
- Letícia Bitencourt Uberti, Marcia Keske-Soares, Karina Carlesso Pagliarin, Development of the Digital Speech Assessment Instrument, American Journal of Speech-Language Pathology, 10.1044/2024_AJSLP-23-00155, 33, 3, (1317-1336), (2024).
- See more
View Options
Sign In Options
ASHA member? If so, log in with your ASHA website credentials for full access.
Member Login