Dynamic Norming for Systematic Analysis of Language Transcripts
This article has a letter.
VIEW LETTERPublication: Journal of Speech, Language, and Hearing Research
Volume 65, Number 1
Pages 320-333
Abstract
Purpose:
This exploratory study sought to establish the psychometric stability of a dynamic norming system using the Systematic Analysis of Language Transcripts (SALT) databases. Dynamic norming is the process by which clinicians select a subset of the normative database sample matched to their individual client's demographic characteristics.
Method:
The English Conversation and Student-Selected Story (SSS) Narrative databases from SALT were used to conduct the analyses in two phases. Phase 1 was an exploratory examination of the standard error of measure (SEM) of six clinically relevant transcript metrics at predetermined sampling intervals to determine (a) whether the dynamic norming process resulted in samples with adequate stability and (b) the minimum sample size required for stable results. Phase 2 was confirmatory, as random samples were taken from the SALT databases to simulate clinical comparison samples. These samples were examined (a) for stability of SEM estimations and (b) to confirm the sample size findings from Phase 1.
Results:
Results of Phase 1 indicated that the SEMs for the six transcript metrics across both databases were low relative to each metric's scale. Samples as small as 40–50 children in the Conversation database and 20–30 children in the SSS Narrative database resulted in stable SEM estimations. Phase 2 confirmed these findings, indicating that age bands as small as ±4 months from a given center-point resulted in stable estimations provided there were approximately 35 children or more in the comparison sample.
Conclusion:
Psychometrically stable comparison samples can be achieved using SALT's dynamic norming system that are much smaller than the standard sample size recommended in most tests of children's language.
Get full access to this article
View all available purchase options and get full access to this article.
References
Andersson, L. (2005). Determining the adequacy of tests of children's language. Communication Disorders Quarterly, 26(4), 207–225.
Bedore, L. M., & Leonard, L. B. (1998). Specific language impairment and grammatical morphology. Journal of Speech, Language, and Hearing Research, 41(5), 1185–1192.
Bedore, L. M., Pena, E. D., Gillam, R. B., & Ho, T. H. (2010). Language sample measures and language ability in Spanish-English bilingual kindergarteners. Journal of Communication Disorders, 43(6), 498–510.
Bishop, D. V. M., & Edmundson, A. (1987). Language-impaired 4-year-olds. Journal of Speech and Hearing Disorders, 52(2), 156–173.
Colozzo, P., Gillam, R. B., Wood, M., Schnell, R. D., & Johnston, J. R. (2011). Content and form in the narratives of children with specific language impairment. Journal of Speech, Language, and Hearing Research, 54(6), 1609–1627.
Common Core State Standards Initiative. (2019). Common core state standards for English language arts and literacy in history/social studies, science, and technical subjects. http://www.corestandards.org/wp-content/uploads/ELA_Standards1.pdf
Dale, P. S., McMillan, A. J., Hayiou-Thomas, M. E., & Plomin, R. (2014). Illusory recovery: Are recovered children with early language delay at continuing elevated risk? American Journal of Speech-Language Pathology, 23(3), 437–447.
Fiestas, C. E., Bedore, L. M., Peña, E. D., Nagy, V. J., Cohen, J., & McAlister, K. T. (2005). Use of mazes in the narrative language samples of bilingual and monolingual 4- to 7-year old children. In Cohen, J., McAlister, K. T., Rolstad, K., & MacSwan, J. (Eds.), ISB4: Proceedings of the 4th International Symposium on Bilingualism (pp. 730–740). Cascadilla Press.
Gavin, W. J., & Giles, L. (1996). Sample size effects on temporal reliability of language sample measures of preschool children. Journal of Speech and Hearing Research, 39(6), 1258–1262.
Heilmann, J., DeBrock, L., & Riley-Tillman, T. C. (2013). Stability of measures from children's interviews: The effects of time, sample length, and topic. American Journal of Speech-Language Pathology, 22(3), 463–475.
Heilmann, J. J., Miller, J. F., & Nockerts, A. (2010). Using language sample databases. Language, Speech, and Hearing Services in Schools, 41(1), 84–95.
Heilmann, J., Nockerts, A., & Miller, J. F. (2010). Language sampling: Does the length of the transcript matter. Language, Speech, and Hearing Services in Schools, 41(4), 393–404.
Heilmann, J., Tucci, A., Plante, E., & Miller, J. F. (2020). Assessing functional language in school-aged children using language sample analysis. Perspectives of the ASHA Special Interest Groups, 5(3), 622–636.
Hewitt, L. E., Hammer, C. S., Yont, K. M., & Tomblin, J. B. (2005). Language sampling for kindergarten children with and without SLI: Mean length of utterance, IPSYN, and NDW. Journal of Communication Disorders, 38(3), 197–213.
Kazemi, Y., Klee, T., & Stringer, H. (2015). Diagnostic accuracy of language sample measures with Persian-speaking preschool children. Clinical Linguistics & Phonetics, 29(4), 304–318.
Klecan-Aker, J. S., & Hedrick, D. L. (1985). A study of the syntactic language skills of normal school-age children. Language, Speech, and Hearing Services in Schools, 16(3), 187–198.
Leadholm, B., & Miller, J. F. (1992). Language sample analysis: The Wisconsin guide. Department of Public Instruction.
MacWhinney, B. (2000). The CHILDES project: The database (Vol. 2). Psychology Press.
Miller, J. (1981). Assessing language production in children. University Park Press.
Miller, J. (1991). Quantifying productive language disorders. In J. F. Miller (Ed.), Research on child language disorders: A decade of progress (pp. 211–220). Pro-Ed.
Miller, J. F., & Iglesias, A. (2017). Systematic Analysis of Language Transcripts (SALT), Research Version 18 [Computer software] . Salt Software, LLC.
Pankratz, M., Plante, E., Vance, R., & Insalaco, D. (2007). The diagnostic and predictive validity of The Renfrew Bus Story. Language, Speech, and Hearing Services in Schools, 38(4), 390–399.
Paul, R., & Alforde, S. (1993). Grammatical morpheme acquisition in 4-year-olds with normal, impaired, and late-developing language. Journal of Speech and Hearing Research, 36(6), 1271–1275.
Paul, R., Hernandez, R., Taylor, L., & Johnson, K. (1996). Narrative development in late talkers. Journal of Speech and Hearing Research, 39(6), 1295–1303.
Paul, R., & Smith, R. L. (1993). Narrative skills in 4-year-olds with normal, impaired, and late-developing language. Journal of Speech and Hearing Research, 36(3), 592–598.
Pavelko, S. L., & Owens, R. E., Jr. (2017). Sampling Utterances and Grammatical Analysis Revised (SUGAR): New normative values for language sample analysis measures. Language, Speech, and Hearing Services in Schools, 48(3), 197–215.
Pezold, M. J., Imgrund, C. M., & Storkel, H. L. (2020). Using computer programs for language sample analysis. Language, Speech, and Hearing Services in Schools, 51(1), 103–114.
R Core Team. (2017). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/
Rice, M. L., Smolik, F., Perpich, D., Thompson, T., Rytting, N., & Blossom, M. (2010). Mean length of utterance levels in 6-month intervals for children 3 to 9 years with and without language impairments. Journal of Speech, Language, and Hearing Research, 53(2), 333–349.
Revelle, W. (2019). psych: Procedures for psychological, psychometric, and personality research (R package Version 1.9.12). Northwestern University. https://CRAN.R-project.org/package=psych
SALT Software. (2020a). Conversation – Database description. https://www.saltsoftware.com/media/wysiwyg/reference_database/ConRDBDoc.pdf
SALT Software. (2020b). Narrative SSS – Database description. https://www.saltsoftware.com/media/wysiwyg/reference_database/NarSSSRDBDoc.pdf
Sarkar, D. (2008). Lattice: Multivariate data visualization with R. Springer.
Schauberger, P., & Walker, A. (2020). openxlsx: Read, write and edit xlsx files. R package. https://CRAN.R-project.org/package=openxlsx
Simón-Cereijido, G., & Gutiérrez-Clellen, V. F. (2009). A cross-linguistic and bilingual evaluation of the interdependence between lexical and grammatical domains. Applied Psycholinguistics, 30(2), 315–337.
Snow, C. (1983). Literacy and language: Relationships during the preschool years. Harvard Educational Review, 53(2), 165–189.
Sprinthall, R. C. (2007). Basic statistical analysis (8th ed.). Allyn & Bacon.
Tilstra, J., & McMaster, K. (2007). Productivity, fluency, and grammaticality measures from narratives: Potential indicators of language proficiency. Communication Disorders Quarterly, 29(1), 43–53.
Wickham, H., François, R., Henry, L., & Müller, K. (2020). dplyr: A grammar of data manipulation. R package. https://CRAN.R-project.org/package=dplyr
Information & Authors
Information
Published In
Journal of Speech, Language, and Hearing Research
Volume 65 • Number 1 • January 2022
Pages: 320-333
PubMed: 34890246
Copyright
Copyright © 2021 American Speech-Language-Hearing Association.
History
- Received: Apr 21, 2021
- Revised: Aug 3, 2021
- Accepted: Sep 13, 2021
- Published online: Dec 10, 2021
- Published in issue: Jan 12, 2022
Authors
Metrics & Citations
Metrics
Article Metrics
View all metricsCitations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
Citing Literature
- Raúl Rojas, Farzan Irani, Svenja Gusewski, Natalia Camacho, A cross-sectional investigation of disfluencies in typically developing Spanish-English bilingual children, Journal of Fluency Disorders, 10.1016/j.jfludis.2023.105988, 77, (105988), (2023).
- Rebecca Bright, Elaine Ashton, Cristina Mckean, Yvonne Wren, The development of a digital story-retell elicitation and analysis tool through citizen science data collection, software development and machine learning, Frontiers in Psychology, 10.3389/fpsyg.2023.989499, 14, (2023).
- John Heilmann, Jon F. Miller, Systematic Analysis of Language Transcripts Solutions: A Tutorial, Perspectives of the ASHA Special Interest Groups, 10.1044/2022_PERSP-22-00148, 8, 1, (1-18), (2023).
- Brian MacWhinney, Davida Fromm, Language Sample Analysis With TalkBank: An Update and Review, Frontiers in Communication, 10.3389/fcomm.2022.865498, 7, (2022).
- Brian MacWhinney, Nan Bernstein Ratner, Dynamic Norming and Open Science, Journal of Speech, Language, and Hearing Research, 10.1044/2022_JSLHR-22-00019, 65, 3, (1183-1185), (2022).
View Options
Sign In Options
ASHA member? If so, log in with your ASHA website credentials for full access.
Member Login