Should We Stop Using Lexical Diversity Measures in Children's Language Sample Analysis?
Publication: American Journal of Speech-Language Pathology
Volume 33, Number 4
Pages 1986-2001
Abstract
Purpose:
Prior work has identified weaknesses in commonly used indices of lexical diversity in spoken language samples, such as type–token ratio (TTR) due to sample size and elicitation variation, we explored whether TTR and other diversity measures, such as number of different words/100 (NDW), vocabulary diversity (VocD), and the moving average TTR would be more sensitive to child age and clinical status (typically developing [TD] or developmental language disorder [DLD]) if samples were obtained from standardized prompts.
Method:
We utilized archival data from the norming samples of the Test of Narrative Language and the Edmonton Narrative Norms Instrument. We examined lexical diversity and other linguistic properties of the samples, from a total of 1,048 children, ages 4–11 years; 798 of these were considered TD, whereas 250 were categorized as having a language learning disorder.
Results:
TTR was the least sensitive to child age or diagnostic group, with good potential to misidentify children with DLD as TD and TD children as having DLD. Growth slopes of NDW were shallow and not very sensitive to diagnostic grouping. The strongest performing measure was VocD. Mean length of utterance, TNW, and verbs/utterance did show both good growth trajectories and ability to distinguish between clinical and typical samples.
Conclusions:
This study, the largest and best controlled to date, re-affirms that TTR should not be used in clinical decision making with children. A second popular measure, NDW, is not measurably stronger in terms of its psychometric properties. Because the most sensitive measure of lexical diversity, VocD, is unlikely to gain popularity because of reliance on computer-assisted analysis, we suggest alternatives for the appraisal of children's expressive vocabulary skill.
Get full access to this article
View all available purchase options and get full access to this article.
References
American Speech-Language-Hearing Association. (2004). Preferred practice patterns for the profession of speech-language pathology [Preferred Practice Patterns]. https://www.asha.org/policy/pp2004-00191/
Arjmandi, M. K., Houston, D., & Dilley, L. C. (2022). Variability in quantity and quality of early linguistic experience in children with cochlear implants: Evidence from analysis of natural auditory environments. Ear and Hearing, 43(2), 685–698.
Arkansas Department of Human Services. (2016). Provider manual update transmittal THERAPY-3-15. Retrieved March 2, 2024, from https://www.sos.arkansas.gov/uploads/rulesRegs/Arkansas%20Register/2016/july2016/016.06.16-013.pdf [PDF]
Bawayan, R., & Brown, J. A. (2022). Language sample analysis consideration and use: A survey of school-based speech language pathologists. Clinical Archives of Communication Disorders, 7(1), 15–28.
Bernstein Ratner, N., & MacWhinney, B. (2016). Your laptop to the rescue: Using the Child Language Data Exchange System archive and CLAN utilities to improve child language sample analysis. Seminars in Speech and Language, 37(2), 074–084.
Bitetti, D. (2023). Accuracy of undergraduate students' language sample analysis in online and face-to-face courses. Perspectives of the ASHA Special Interest Groups, 8(6), 1422–1431.
Bleses, D., Makransky, G., Dale, P. S., Højen, A., & Ari, B. A. (2016). Early productive vocabulary predicts academic achievement 10 years later. Applied Psycholinguistics, 37(6), 1461–1476.
Brown, R. (1973). A first language: The early stages. Harvard University Press.
Channell, M. M., Loveall, S. J., Conners, F. A., Harvey, D. J., & Abbeduto, L. (2018). Narrative language sampling in typical development: Implications for clinical trials. American Journal of Speech-Language Pathology, 27(1), 123–135.
Charest, M., & Skoczylas, M. J. (2019). Lexical diversity versus lexical error in the language transcripts of children with developmental language disorder: Different conclusions about lexical ability. American Journal of Speech-Language Pathology, 28(3), 1275–1282.
Charest, M., Skoczylas, M. J., & Schneider, P. (2020). Properties of lexical diversity in the narratives of children with typical language development and developmental language disorder. American Journal of Speech-Language Pathology, 29(4), 1866–1882.
DeMayo, B., Kellier, D., Braginsky, M., Bergmann, C., Hendriks, C., Rowland, C. F., Frank, M., & Marchman, V. (2021). Web-CDI: A system for online administration of the MacArthur-Bates Communicative Development Inventories. Language Development Research, 1(1), 55–98.
Fergadiotis, G., Wright, H. H., & West, T. M. (2013). Measuring lexical diversity in narrative discourse of people with aphasia. American Journal of Speech-Language Pathology, 22(2), S397–S408.
Finestack, L. H., Rohwer, B., Hilliard, L., & Abbeduto, L. (2020). Using Computerized Language Analysis to evaluate grammatical skills. Language, Speech, and Hearing Services in Schools, 51(2), 184–204.
Finestack, L. H., & Satterlund, K. E. (2018). Current practice of child grammar intervention: A survey of speech-language pathologists. American Journal of Speech-Language Pathology, 27(4), 1329–1351.
Gallagher, J. F., & Hoover, J. R. (2020). Measure what you treat: Using language sample analysis for grammatical outcome measures in children with developmental language disorder. Perspectives of the ASHA Special Interest Groups, 5(2), 350–363.
Gillam, R. B., & Pearson, N. (2004). Test of Narrative Language. Pro-Ed.
Gonzalez Villasanti, H., Justice, L. M., Chaparro-Moreno, L. J., Lin, T.-J., & Purtell, K. (2020). Automatized analysis of children's exposure to child-directed speech in reschool settings: Validation and application. PLOS ONE, 15(11), Article e0242511.
Green, C., Keogh, K., Sun, H., & O'Brien, B. (2023). The Children's Picture Books Lexicon (CPB-LEX): A large-scale lexical database from children's picture books. Behavior Research Methods. Advance online publication.
Guo, L. Y., Eisenberg, S., Schneider, P., & Spencer, L. (2019). Percent grammatical utterances between 4 and 9 years of age for the Edmonton Narrative Norms Instrument: Reference data and psychometric properties. American Journal of Speech-Language Pathology, 28(4), 1448–1462.
Guo, L. Y., Eisenberg, S., Schneider, P., & Spencer, L. (2020). Finite verb morphology composite between age 4 and age 9 for the Edmonton Narrative Norms Instrument: Reference data and psychometric properties. Language, Speech, and Hearing Services in Schools, 51(1), 128–143.
Guo, L. Y., Schneider, P., & Harrison, W. (2021). Clausal density between ages 4 and 9 years for the Edmonton Narrative Norms Instrument: Reference data and psychometric properties. Language, Speech, and Hearing Services in Schools, 52(1), 354–368.
Harris Wright, H., Silverman, S., & Newhoff, M. (2003). Measures of lexical diversity in aphasia. Aphasiology, 17(5), 443–452.
Hegde, M. N., & Pomaville, F. (2021). Assessment of communication disorders in children: Resources and protocols. Plural.
Heilmann, J. J., & Miller, J. F. (2023). Systematic Analysis of Language Transcripts solutions: A tutorial. Perspectives of the ASHA Special Interest Groups, 8(1), 1–18.
Heilmann, J. J., Miller, J. F., & Nockerts, A. (2010). Using language sample databases. Language, Speech, and Hearing Services in Schools, 41(1), 84–95.
Ireland, M., & Conrad, B. J. (2016). Evaluation and eligibility for speech-language services in schools. Perspectives of the ASHA Special Interest Groups, 1(16), 78–90.
Kapantzoglou, M., Fergadiotis, G., & Auza Buenavides, A. (2019). Psychometric evaluation of lexical diversity indices in Spanish narrative samples from children with and without developmental language disorder. Journal of Speech, Language, and Hearing Research, 62(1), 70–83.
Kapantzoglou, M., Fergadiotis, G., & Restrepo, M. A. (2017). Language sample analysis and elicitation technique effects in bilingual children with and without language impairment. Journal of Speech, Language, and Hearing Research, 60(10), 2852–2864.
Klatte, I. S., van Heugten, V., Zwitserlood, R., & Gerrits, E. (2022). Language sample analysis in clinical practice: Speech-language pathologists' barriers, facilitators, and needs. Language, Speech, and Hearing Services in Schools, 53(1), 1–16.
Klee, T. (1992). Developmental and diagnostic characteristics of quantitative measures of childrenʼs language production. Topics in Language Disorders, 12(2), 28–41.
Klee, T., Stokes, S. F., Wong, A. M.-Y., Fletcher, P., & Gavin, W. J. (2004). Utterance length and lexical diversity in Cantonese-speaking children with and without specific language impairment. Journal of Speech, Language, and Hearing Research, 47(6), 1396–1410.
Kubota, M., Chondrogianni, V., Clark, A. S., & Rothman, J. (2021). Linguistic consequences of toing and froing: Factors that modulate narrative development in bilingual returnee children. International Journal of Bilingual Education and Bilingualism, 25(7), 2363–2381.
Libertus, M. E., Odic, D., Feigenson, L., & Halberda, J. (2015). A Developmental Vocabulary Assessment for Parents (DVAP): Validating parental report of vocabulary size in 2- to 7-year-old children. Journal of Cognition and Development, 16(3), 442–454.
Library of Medicine. (n.d.). Common data elements. Retrieved March 2, 2024, from https://www.nlm.nih.gov/oet/ed/cde/tutorial/03-100.html
Magimairaj, B. M., Capin, P., Gillam, S. L., Vaughn, S., Roberts, G., Fall, A. M., & Gillam, R. B. (2022). Online administration of the Test of Narrative Language–Second Edition: Psychometrics and considerations for remote assessment. Language, Speech, and Hearing Services in Schools, 53(2), 404–416.
Makransky, G., Dale, P. S., Havmose, P., & Bleses, D. (2016). An item response theory–based, computerized adaptive testing version of the MacArthur–Bates Communicative Development Inventory: Words & Sentences (CDI:WS). Journal of Speech, Language, and Hearing Research, 59(2), 281–289.
Malvern, D., & Richards, B. (2002). Investigating accommodation in language proficiency interviews using a new measure of lexical diversity. Language Testing, 19(1), 85–104.
Malvern, D., Richards, B., Chipere, N., & Durán, P. (2004). Lexical diversity and language development: Quantification and assessment. Palgrave Macmillan.
Manning, B. L., Harpole, A., Harriott, E. M., Postolowicz, K., & Norton, E. S. (2020). Taking language samples home: Feasibility, reliability, and validity of child language samples conducted remotely with video chat versus in-person. Journal of Speech, Language, and Hearing Research, 63(12), 3982–3990.
Mayor, J., & Mani, N. (2019). A short version of the MacArthur–Bates Communicative Development Inventories with high validity. Behavior Research Methods, 51(5), 2248–2255.
MacWhinney, B. (2000). The CHILDES Project: Tools for analyzing talk (3rd ed.). Lawrence Erlbaum Associates.
McCarthy, P. M., & Jarvis, S. (2010). MTLD, VocD-D, and HD-D: A validation study of sophisticated approaches to lexical diversity assessment. Behavior Research Methods, 42(2), 381–392.
Miller, J. (1981). Assessing language production in children: Experimental procedures. Allyn & Bacon.
Miller, J., Andriacchi, K., & Nockerts, A. (2015). Assessing language production using SALT software. SALT Software, LLC.
Miller, J. F., & Chapman, R. S. (1981). The relation between age and mean length of utterance in morphemes. Journal of Speech, Language, and Hearing Research, 24(2), 154–161.
Miller, J. F., Freiberg, C., Holland, M. B., & Reeves, M. A. (1992). Implementing computerized language sample analysis in the public school. Topics in Language Disorders, 12(2), 69–82.
Montag, J. L., Jones, M. N., & Smith, L. B. (2018). Quantity and diversity: Simulating early word learning environments. Cognitive Science, 42(S2), 375–412.
Murphy, K. A., Springle, A. P., Sultani, M. J., McIlraith, A., & Language and Reading Research Consortium (LARRC). (2022). Predicting language performance from narrative language samples. Journal of Speech, Language, and Hearing Research, 65(2), 775–748.
North Carolina Department of Public Instruction. (2006). North Carolina Guidelines for Speech-Language Pathology Services in Schools. Retrieved March 2, 2024, from https://www.dpi.nc.gov/documents/ec/speech-guide/download?attachment [PDF]
Oh, S. J., Yoon, J. H., & Lee, Y. (2020). A qualitative study on experiences and needs of language sample analysis by speech-language pathologists: Focused on children with language disorders. Communication Sciences & Disorders, 25(2), 169–189.
Overton, C., Baron, T., Pearson, B. Z., & Bernstein Ratner, N. (2021). Using free computer-assisted language sample analysis to evaluate and set treatment goals for children who speak African American English. Language, Speech, and Hearing Services in Schools, 52(1), 31–50.
Owen, A. J., & Leonard, L. B. (2002). Lexical diversity in the spontaneous speech of children with specific language impairment: Application of D. Journal of Speech, Language, and Hearing Research, 45(5), 927–937.
Owens, R. E., Jr. (2017). Early language intervention for infants, toddlers, and preschoolers. Pearson.
Owens, R. E., Jr. (2022). Language disorders: A functional approach to assessment and intervention in children. Plural.
Paul, R., Norbury, C., & Gosse, C. (2018). Language disorders from infancy through adolescence. Elsevier.
Pavelko, S. L., Owens, R. E., Jr., Ireland, M., & Hahs-Vaughn, D. L. (2016). Use of language sample analysis by school-based SLPs: Results of a nationwide survey. Language, Speech, and Hearing Services in Schools, 47(3), 246–258.
Pennsylvania Intermediate Unit 1 Department of Communication Disorders and Sensory Impairments. (n.d.). Guidelines for case selection, continuance and dismissal for speech/language services within Intermediate Unit 1. Retrieved March 2, 2024, from, https://campussuite-storage.s3.amazonaws.com/prod/1558550/996e2c54-539e-11e8-b9a1-0a0344361b00/1937877/f9662e30-85e9-11e9-b5d2-125b907f6c20/file/criteriaManual.pdf [PDF]
Pezold, M. J., Imgrund, C. M., & Storkel, H. L. (2020). Using computer programs for language sample analysis. Language, Speech, and Hearing Services in Schools, 51(1), 103–114.
R Core Team. (2024). R: A language and environment for statistical computing [Computer software]. R Foundation for Statistical Computing. https://www.R-project.org
Ramos, M. N., Collins, P., & Peña, E. D. (2022). Sharpening our tools: A systematic review to identify diagnostically accurate language sample measures. Journal of Speech, Language, and Hearing Research, 65(10), 3890–3907.
Retherford, K., Schreiber, L., & Jarzynski, R. (2019). Guide to the analysis of language transcripts (4th ed.). Pro-Ed.
Rice, M. L., & Bode, J. V. (1993). GAPS in the verb lexicons of children with specific language impairment. First Language, 13(37), 113–131.
Schneider, P., Hayward, D., & Dubé, R. V. (2006). Storytelling from pictures using the Edmonton Narrative Norms Instrument. Canadian Journal of Speech-Language Pathology and Audiology, 30, 224–238. https://www.cjslpa.ca/detail.php?ID=936〈=en
Silverman, S., & Bernstein Ratner, N. (2002). Measuring lexical diversity in children who stutter: Application of vocd. Journal of Fluency Disorders, 27(4), 289–304.
Simon-Cereijido, G., & Gutiérrez-Clellen, V. F. (2009). A cross-linguistic and bilingual evaluation of the interdependence between lexical and grammatical domains. Applied Psycholinguistics, 30(2), 315–337.
Singleton, N., & Shulman, B. (2020). Language development: Foundations, processes and clinical applications (3rd ed.). Jones & Bartlett.
Tager-Flusberg, H., Calkins, S., Nolin, T., Baumberger, T., Anderson, M., & Chadwick-Dias, A. (1990). A longitudinal study of language acquisition in autistic and Down syndrome children. Journal of Autism and Developmental Disorders, 20(1), 1–21.
Templin, M. C. (1957). Certain language skills in children: Their development and interrelationships (New ed., Vol. 26). University of Minnesota. http://www.jstor.org/stable/10.5749/j.ctttv2st
Texas Speech and Hearing Association. (2020). TSHA SI disability determination guidelines for language disorder. Retrieved March 2, 2024, from https://www.txsha.org/Portals/0/Documents/Education/201213%20Language%20Disability%20Guidelines.pdf [PDF]
Tomblin, J. B., Records, N., & Zhang, X. (1996). A system for the diagnosis of specific language impairment in kindergarten children. Journal of Speech and Hearing Research, 39(6), 1284–1294.
Turnbull, K., & Justice, L. (2017). Language development from theory to practice. Pearson.
Vermeer, A. (2000). Coming to grips with lexical richness in spontaneous speech data. Language Testing, 17(1), 65–83.
Watkins, R. V., Kelly, D. J., Harbers, H. M., & Hollis, W. (1995). Measuring children's lexical diversity: Differentiating typical and impaired language learners. Journal of Speech and Hearing Research, 38(6), 1349–1355.
Wong, A. M.-Y., Klee, T., Stokes, S. F., Fletcher, P., & Leonard, L. B. (2010). Differentiating Cantonese-speaking preschool children with and without SLI using MLU and lexical diversity (D). Journal of Speech, Language, and Hearing Research, 53(3), 794–799.
Wu, S.-Y., Huang, R.-J., & Tsai, I.-F. (2019). The applicability of D, MTLD, and MATTR in Mandarin–speaking children. Journal of Communication Disorders, 77, 71–79.
Yang, J. S., MacWhinney, B., & Bernstein Ratner, N. (2021). The Index of Productive Syntax: Psychometric properties and suggested modifications. American Journal of Speech-Language Pathology, 31(1), 239–256.
Yang, J. S., Rosvold, C., & Bernstein Ratner, N. (2022). Measurement of lexical diversity in children's spoken language: Computational and conceptual considerations. Frontiers in Psychology, 13.
Information & Authors
Information
Published In
American Journal of Speech-Language Pathology
Volume 33 • Number 4 • July 2024
Pages: 1986-2001
PubMed: 38838249
Copyright
Copyright © 2024 American Speech-Language-Hearing Association.
History
- Received: Dec 7, 2023
- Revised: Mar 9, 2024
- Accepted: Apr 11, 2024
- Published online: Jun 5, 2024
- Published in issue: Jul 3, 2024
Authors
Metrics & Citations
Metrics
Article Metrics
View all metricsCitations
Download article citation data for:
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
Citing Literature
- Chengqi He, Shoko Miyamoto, Yu Lin, Narratives of Chinese kindergarteners with Autism spectrum disorder: Comparison with typically developing children, Clinical Linguistics & Phonetics, 10.1080/02699206.2025.2451968, (1-25), (2025).
View Options
Sign In Options
ASHA member? If so, log in with your ASHA website credentials for full access.
Member Login