Sample Length Affects the Reliability of Language Sample Measures in 3-Year-Olds: Evidence From Parent-Elicited Conversational Samples
Publication: Language, Speech, and Hearing Services in Schools
Volume 46, Number 2
Pages 141-153
Abstract
Purpose
The goal of this study was to investigate the extent to which sample length affected the reliability of total number of words (TNW), number of different words (NDW), and mean length of C-units in morphemes (MLCUm) in parent-elicited conversational samples for 3-year-olds.
Method
Participants were sixty 3-year-olds. A 22-min language sample was collected from each child during free play with the parent in the laboratory. Samples of 1, 3, 7, and 10 min were extracted from the 22-min samples. TNW, NDW, and MLCUm were computed from each shorter sample and the 22-min sample. TNW and NDW were adjusted by number of minutes for comparisons. The differences and correlations between each shorter sample cut and the 22-min sample on MLCUm and adjusted TNW and NDW were computed.
Results
The shorter samples and the 22-min samples significantly differed in adjusted TNW and NDW, but not in MLCUm. TNW reached an acceptable reliability level (i.e., r = .90) in 7-min samples. NDW and MLCUm approached the acceptable reliability level (rs = .88) in 7-min samples and reached it in 10-min samples.
Conclusion
For conversational language samples with similar collection procedures, samples of 7 to 10 min are desirable for calculating TNW, NDW, and MLCUm in 3-year-olds.
Get full access to this article
View all available purchase options and get full access to this article.
References
American Speech-Language-Hearing Association. (2012). ASHA 2012 schools survey: SLP caseload characteristics report. Rockville, MD: Author.
Bedore, L. M., & Leonard, L. B. (1998). Specific language impairment and grammatical morphology: A discriminant function analysis. Journal of Speech, Language, and Hearing Research, 41, 1185–1192.
Bogue, E., DeThorne, L., & Schaefer, B. (2014). A psychometric analysis of childhood vocabulary tests. Contemporary Issues in Communication Science and Disorders, 41, 55–69. doi:1092-5171/14/4101-0055
Bornstein, M. H., Haynes, O. M., Painter, K. M., & Genevro, J. L. (2000). Child language with mother and with stranger at home and in the laboratory: A methodological study. Journal of Child Language, 27, 407–420.
Brorson, K., & Dewey, C. (2005). Effect of language sample size on MLUw. Hearsay, 17, 46–56.
Brown, R. (1973). A first language: The early stages. Cambridge, MA: Harvard University Press.
Bruton, A., Conway, J. H., & Holgate, S. T. (2000). Reliability: What is it, and how is it measured? Physiotherapy, 86, 94–99.
Casby, M. W. (2011). An examination of the relationship of sample size and mean length of utterance for children with developmental language impairment. Child Language Teaching and Therapy, 27, 286–293. doi:10.1177/0265659010394387
Cole, K. N., Mills, P. E., & Dale, P. S. (1989). Examination of test–retest and split-half reliability for measures derived from language samples of young handicapped children. Language, Speech, and Hearing Services in Schools, 20, 259–268.
Darley, F. L., & Moll, K. L. (1960). Reliability of language measures and size of language sample. Journal of Speech and Hearing Research, 3, 166–173.
Dawson, J., Stout, C., Eyer, J., Tattersall, P., Fonkalsrud, J., & Croley, K. (2004). Structured Photographic Expressive Language Test–Preschool: Second Edition. DeKalb, IL: Janelle.
DeThorne, L. S., Deater-Deckard, K., Mahurin-Smith, J., Coletto, M.-K., & Petrill, S. A. (2011). Volubility as a mediator in the associations between conversational language measures and child temperament. International Journal of Language & Communication Disorders, 46, 700–713. doi:10.1111/j.1460-6984.2011.00034.x
Dunn, M., Flax, J., Sliwinski, M., & Aram, D. (1996). The use of spontaneous language measures as criteria for identifying children with specific language impairment: An attempt to reconcile clinical and research incongruence. Journal of Speech and Hearing Research, 39, 643–654.
Eisenberg, S. L., Fersko, T. M., & Lundgren, C. (2001). The use of MLU for identifying language impairment in preschool children: A review. American Journal of Speech-Language Pathology, 10, 323–342. doi:10.1044/1058-0360(2001/028)
Eisenberg, S. L., & Guo, L.-Y. (2013). Differentiating children with and without language impairment based on grammaticality. Language, Speech, and Hearing Services in Schools, 44, 20–31. doi:10.1044/0161-1461(2012/11-0089)
Evans, J. L., & Craig, H. K. (1992). Language sample collection and analysis: Interview compared to freeplay assessment contexts. Journal of Speech and Hearing Research, 35, 343–353. doi:10.1044/jshr.3502.343
Fey, M. E. (1986). Language intervention with young children. San Diego, CA: College-Hill Press.
Field, A. (2009). Discovering statistics using SPSS (4th ed.). Thousand Oaks, CA: Sage.
Fluharty, N. (2001). Fluharty Preschool Speech and Language Screening Test–Second Edition. Austin, TX: Pro-Ed.
Gavin, W. J., & Giles, L. (1996). Sample size effects on temporal reliability of language sample measures of preschool children. Journal of Speech and Hearing Research, 39, 1258–1262.
Gladfelter, A., & Leonard, L. B. (2013). Alternative tense and agreement morpheme measures for assessing grammatical deficits during the preschool period. Journal of Speech, Language, and Hearing Research, 56, 542–552. doi:10.1044/1092-4388(2012/12-0100)
Gray, S., Plante, E., Vance, R., & Henrichsen, M. (1999). The diagnostic accuracy of four vocabulary tests administered to preschool-age children. Language, Speech, and Hearing Services in Schools, 30, 196–206. doi:10.1044/0161-1461.3002.196
Greenslade, K., Plante, E., & Vance, R. (2009). The diagnostic accuracy and construction validity of the Structured Photographic Expressive Language Test–Preschool: Second Edition. Language, Speech, and Hearing Services in Schools, 40, 150–160. doi:10.1044/0161-1461(2008/07-0049)
Guo, L.-Y., & Eisenberg, S. (2013, November). Are shorter samples as good as longer samples? Poster presented at the 2013 Convention of the American Speech-Language-Hearing Association, Chicago, IL.
Guo, L.-Y., & Eisenberg, S. (2014). The diagnostic accuracy of two tense measures for identifying 3-year-olds with language impairment. American Journal of Speech-Language Pathology, 23(2), 203–212. doi:10.1044/2013_AJSLP-13-0007
Heilmann, J., Nockerts, A., & Miller, J. F. (2010). Language sampling: Does the length of the transcript matter? Language, Speech, and Hearing Services in Schools, 41, 393–404. doi:10.1044/0161-1461(2009/09-0023)
Hewitt, L. E., Hammer, C. S., Yont, K. M., & Tomblin, J. B. (2005). Language sampling for kindergarten children with and without SLI: Mean length of utterance, IPSYN, and NDW. Journal of Communication Disorders, 38, 197–213. doi:10.1016/j.jcomdis.2004.10.002
Hoff, E. (2010). Context effects on young children's language use: The influence of conversational setting and partner. First Language, 30, 461–472. doi:10.1177/0142723710370525
Hopkins, K. D., Stanley, J., & Hopkins, B. (1990). Educational and psychological measurement and evaluation (7th ed.). Upper Saddle River, NJ: Pearson Education.
Horton-Ikard, R. (2010). Language sample analysis with children who speak non-mainstream dialects of English. Perspectives on Language Learning and Education, 17(1), 16–23. doi:10.1044/lle17.1.16
Hutchinson, T. A. (1996). What to look for in the technical manual: Twenty questions for users. Language, Speech, and Hearing Services in Schools, 27, 109–121. doi:10.1044/0161-1461.2702.109
Kemp, K., & Klee, T. (1997). Clinical language sampling practices: Results of a survey of speech-language pathologists in the United States. Child Language Teaching and Therapy, 13, 161–176. doi:10.1177/026565909701300204
Lahey, M. (1988). Language disorders and language development. Needham, MA: Macmillan.
Lee, L. (1974). Developmental sentence analysis: A grammatical assessment procedure for speech and language clinicians. Evanston, IL: Northwestern University Press.
Liles, B. Z. (1985). Cohesion in the narratives of normal and language-disordered children. Journal of Speech and Hearing Research, 28, 123–133. doi:10.1044/jshr.2801.123
Loban, W. (1976). Language development: Kindergarten through grade twelve. Urbana, IL: National Council of Teachers of English.
McCarthy, D. A. (1975). The language development of the preschool child. Westport, CT: Greenwood Press.
McCauley, R. J. (2001). Assessment of language disorders in children. Mahwah, NJ: Erlbaum.
McCauley, R. J., & Swisher, L. (1984). Psychometric review of language and articulation tests for preschool children. Journal of Speech and Hearing Disorders, 49, 34–42.
McKee, G., Malvern, D., & Richards, B. (2000). Measuring vocabulary diversity using dedicated software. Literary and Linguistic Computing, 15(3), 323–338. doi:10.1093/llc/15.3.323
Miller, J. F. (1981). Assessing language production in children: Experimental procedures. Baltimore, MD: University Park Press.
Miller, J., Andriacchi, K., & Nockerts, A. (2011). Assessing language production using SALT software: A clinician's guide to language sample analysis. Madison, WI: Language Analysis Laboratory, Waisman Center, University of Wisconsin–Madison.
Miller, J., & Iglesias, A. (2010). Systematic Analysis of Language Transcripts (Version 10). Madison, WI: Language Analysis Laboratory, Waisman Center, University of Wisconsin–Madison.
Minifie, F. D., Darley, F. L., & Sherman, D. (1963). Temporal reliability of seven language measures. Journal of Speech and Hearing Research, 6, 139–148.
Nippold, M. A., Frantz-Kaspar, M. W., Cramond, P. M., Kirk, C., Hayward-Mayhew, C., & MacKinnon, M. (2013). Conversational and narrative speaking in adolescents: Examining the use of complex syntax. Journal of Speech, Language, and Hearing Research, 57, 1–11. doi:10.1044/1092-4388(2013/13-0097)
Nippold, M. A., Hesketh, L. J., Duthie, J. K., & Mansfield, T. C. (2005). Conversational versus expository discourse: A study of syntactic development in children, adolescents, and adults. Journal of Speech, Language, and Hearing Research, 48, 1048–1064. doi:10.1044/1092-4388(2005/073)
Oetting, J. B., Newkirk, B. L., Hartfield, L. R., Wynn, C. G., Pruitt, S. L., & Garrity, A. W. (2010). Index of productive syntax for children who speak African American English. Language, Speech, and Hearing Services in Schools, 41, 328–339. doi:10.1044/0161-1461(2009/08-0077)
Owen, A. J., & Leonard, L. B. (2002). Lexical diversity in the spontaneous speech of children with specific language impairment: Application of D. Journal of Speech, Language, and Hearing Research, 45, 927–937. doi:10.1044/1092-4388(2002/075)
Paul, R., & Norbury, C. F. (2012). Language disorders from infancy through adolescence: Listening, speaking, reading, writing, and communicating (4th ed.). St. Louis, MO: Elsevier.
Pearson, B. Z., Jackson, J. E., & Wu, H. (2014). Seeking a valid gold standard for an innovative, dialect-neutral language test. Journal of Speech, Language, and Hearing Research, 57, 495–508. doi:10.1044/2013_JSLHR-L-12-0126
Peña, E. D., Spaulding, T. J., & Plante, E. (2006). The composition of normative groups and diagnostic decision making: Shooting ourselves in the foot. American Journal of Speech-Language Pathology, 15, 247–254. doi:10.1044/1058-0360(2006/023)
Perona, K., Plante, E., & Vance, R. (2005). Diagnostic accuracy of the Structured Photographic Expressive Language Test–Third Edition (SPELT-3). Language, Speech, and Hearing Services in Schools, 36, 103–115. doi:10.1044/0161-1461(2005/010)
Plante, E., & Vance, R. (1994). Selection of preschool language tests: A data-based approach. Language, Speech, and Hearing Services in Schools, 25, 15–24. doi:10.1044/0161-1461.2501.15
Plante, E., & Vance, R. (1995). Diagnostic accuracy of two tests of preschool language. American Journal of Speech-Language Pathology, 4(2), 70–76. doi:10.1044/1058-0360.0402.70
Restrepo, M. A., Schwanenflugel, P. J., Blake, J., Neuharth-Pritchett, S., Cramer, S. E., & Ruston, H. P. (2006). Performance on the PPVT–III and the EVT: Applicability of the measures with African American and European American preschool children. Language, Speech, and Hearing Services in Schools, 37, 17–27. doi:10.1044/0161-1461(2006/003)
Reynolds, C., & Kamphaus, R. (2003). Reynolds Intellectual Screening Test. Lutz, FL: Psychological Assessment Resources.
Rice, M. L., Smolik, F., Perpich, D., Thompson, T., Rytting, N., & Blossom, M. (2010). Mean length of utterance levels in 6-month intervals for children 3 to 9 years with and without language impairments. Journal of Speech, Language, and Hearing Research, 53, 333–349. doi:10.1044/1092-4388(2009/08-0183)
Richards, B. (1987). Type/token ratios: What do they really tell us? Journal of Child Language, 14, 201–209. doi:10.1017/S0305000900012885
Rispoli, M. (2003). Changes in the nature of sentence production during the period of grammatical development. Journal of Speech, Language, and Hearing Research, 46, 818–830. doi:10.1044/1092-4388(2003/064)
Rondal, J. A., & DeFays, D. (1978). Reliability of mean length of utterance as a function of sample size in early language development. The Journal of Genetic Psychology: Research and Theory on Human Development, 133, 305–306.
Shriberg, L. D., Kwiatkowski, J., & Hoffman, K. (1984). A procedure for phonetic transcription by consensus. Journal of Speech and Hearing Research, 27, 456–465.
Southwood, F., & Russell, A. F. (2004). Comparison of conversation, freeplay, and story generation as methods of language sample elicitation. Journal of Speech, Language, and Hearing Research, 47, 366–376. doi:10.1044/1092-4388(2004/030)
Spaulding, T., Plante, E., & Farinella, K. (2006). Eligibility criteria for language impairment: Is the low end of normal always appropriate? Language, Speech, and Hearing Services in Schools, 37, 61–72. doi:10.1044/0161-1461(2006/007)
Stockman, I. J., Guillory, B., Seibert, M., & Boult, J. (2013). Toward validation of a minimal competence core of morphosyntax for African American children. American Journal of Speech-Language Pathology, 22, 40–56. doi:10.1044/1058-0360(2012/11-0124)
Templin, M. C. (1957). Certain language skills in children: Their development and interrelationships [Child Welfare Monograph No. 26]. Minneapolis: University of Minnesota Press.
Tommerdahl, J., & Kilpatrick, C. (2013). Analyzing frequency and temporal reliability of children's morphosyntactic production in spontaneous language samples of varying lengths. Child Language Teaching and Therapy, 29, 171–183. doi:10.1177/0265659012459528
Ukrainetz, T. A., & Blomquist, C. (2002). The criterion validity of four vocabulary tests compared with a language sample. Child Language Teaching and Therapy, 18, 59–78. doi:10.1191/0265659002ct227oa
Watkins, R. V., Kelly, D. J., Harbers, H. M., & Hollis, W. (1995). Measuring children's lexical diversity: Differentiating typical and impaired language learners. Journal of Speech and Hearing Research, 38, 1349–1355.
Westerveld, M. F., & Claessen, M. (2014). Clinician survey of language sampling practices in Australia. International Journal of Speech-Language Pathology, 16, 242–249. doi:10.3109/17549507.2013.871336
Information & Authors
Information
Published In
Language, Speech, and Hearing Services in Schools
Volume 46 • Number 2 • April 2015
Pages: 141-153
PubMed: 25615272
Copyright
Copyright © 2015 American Speech-Language-Hearing Association.
History
- Received: May 6, 2014
- Revised: Jul 17, 2014
- Accepted: Dec 18, 2014
- Published in issue: Apr 1, 2015
Authors
Metrics & Citations
Metrics
Article Metrics
View all metricsCitations
Download article citation data for:
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
Citing Literature
- Christine da Silva-Genest, Caroline Masson, Marine Le Mené Guigourès, Évaluer les compétences morphosyntaxiques et discursives d’enfants d’âge scolaire : analyse d’une situation de jeu libre, Éla. Études de linguistique appliquée, 10.3917/ela.210.0053, N° 210, 2, (179-195), (2024).
- Yibin Zhang, Brian MacWhinney, Jing Zhou, Indicators for Building a Norm-Referenced Dataset Based on Conversational Language Samples in Typical Mandarin-speaking Children: An Exploratory Study, Sage Open, 10.1177/21582440241239466, 14, 1, (2024).
- Chia-Ying Chu, Pei-Hua Chen, Yi-Shin Tsai, Chieh-An Chen, Yi-Chih Chan, Yan-Jhe Ciou, Effect of sample length on MLU in Mandarin-speaking hard-of-hearing children, Journal of Deaf Studies and Deaf Education, 10.1093/deafed/enae007, 29, 3, (388-395), (2024).
- Hin-Tat Cheung, Chia-Hui Lin, Chien-Ju Chang, Measuring productive syntactic abilities in Mandarin-speaking children in Taiwan, Clinical Linguistics & Phonetics, 10.1080/02699206.2024.2302549, 38, 11, (1067-1084), (2024).
- Amy Wilder, Sean M. Redmond, Updates on Clinical Language Sampling Practices: A Survey of Speech-Language Pathologists Practicing in the United States, Language, Speech, and Hearing Services in Schools, 10.1044/2024_LSHSS-24-00035, 55, 4, (1151-1166), (2024).
- Janna B. Oetting, Tahmineh Maleki, Transcription Decisions of Conjoined Independent Clauses Are Equitable Across Dialects but Impact Measurement Outcomes, Language, Speech, and Hearing Services in Schools, 10.1044/2024_LSHSS-23-00180, 55, 3, (870-883), (2024).
- Marjolein Mues, Ellen Demurie, Maide Erdogan, Sarah Schaubroeck, Manon Krol, Amy Goodwin, Jan Buitelaar, Eva Loth, Herbert Roeyers, Uh and um in autism: The case of hesitation marker usage in Dutch-speaking autistic preschoolers , Journal of Child Language, 10.1017/S0305000924000321, (1-17), (2024).
- Rebecca Bright, Elaine Ashton, Cristina Mckean, Yvonne Wren, The development of a digital story-retell elicitation and analysis tool through citizen science data collection, software development and machine learning, Frontiers in Psychology, 10.3389/fpsyg.2023.989499, 14, (2023).
- Ulrike Lüdtke, Hanna Ehlert, Dana Gaigulo, Juan Bornman, Research on the methodology of LSA with preschool children: a scoping review, Clinical Archives of Communication Disorders, 10.21849/cacd.2022.00738, 8, 2, (29-46), (2023).
- Lynda Feenaughty, Linguistic performance during monologues and correlates of neuropsychological function for adults with multiple sclerosis, Aphasiology, 10.1080/02687038.2022.2099527, 37, 9, (1484-1507), (2022).
- See more
View Options
Sign In Options
ASHA member? If so, log in with your ASHA website credentials for full access.
Member Login