Abstract
Purpose
Students with language-learning disabilities (LLD) demonstrate difficulties with written language, especially in the areas of productivity, complexity, and grammar. It is not clear how these deficits affect their performance on high-stakes tests, such as those required by the No Child Left Behind Act (U.S. Department of Education, 2002). This study used writing samples to compare how students with and without LLD scored on analytic writing measures that are typically used in writing research and on a more holistic measure of writing, the six-traits writing rubric (STWR; Education Northwest, 2006), which is used in high-stakes writing assessments.
Method
Fifty-six 4th and 5th graders with typical development (TD) or LLD produced 1 narrative and 1 expository writing sample. Measures of oral language ability and handwriting accuracy–speed were also obtained. The narrative and expository samples were scored using 5–6 separate analytic measures and 6 separate traits on the STWR.
Results
On narratives, the TD group scored significantly higher than the LLD group on 5 analytic measures and all 6 traits. Similarly, for expository, the TD group outscored the LLD group on 3 analytic measures and all 6 traits. Results demonstrate that the analytic scores of productivity, sentence complexity, and lexical diversity were correlated significantly with a higher overall score on the STWR for narrative writing samples only.
Discussion
Results of this study suggest that exclusive use of analytic scores to select treatment goals and document writing progress may not translate into increased scores on writing rubrics, particularly for expository writing samples.
REFERENCES
- American Speech-Language-Hearing Association. (2001). Roles and responsibilities of speech-language pathologists with respect to reading and writing in children and adolescents [Guidelines]. Available from www.asha.org/policy
-
Anderson, S. A. (2002). Is the MEAP writing test reliable? A case study.Michigan Reading Journal, 34(4)23–25. -
Apel, K., & Masterson, J. J. (2001). Theory-guided spelling assessment and intervention: A case study.Language, Speech, and Hearing Services in Schools, 32, 182–195. -
Apel, K., Masterson, J. J., & Hart, P. (2004). Integration of language components in spelling: Instruction that maximizes student’s learning.InE. R. Silliman, & L. C. Wilkinson (Eds.), Language and literacy learning in schools (pp. 292–316). New York, NY: Guilford Press. - Arizona Department of Education. (2006a). AIMS report wizard. Retrieved from http://www.ade.state.az.us/profile/publicview/
- Arizona Department of Education. (2006b). AIMS six trait analytic writing rubric official scoring guide. Retrieved from http://www.ade.state.az.us/SBTL/6traits/Default.asp
-
Crawford, L., Tindal, G., & Carpenter, D. M. I. (2006). Exploring the validity of the Oregon Extended Writing Assessment.Journal of Special Education, 40(1)16–27. -
Davidson, M., Howell, K. W., & Hoekema, P. (2000). Effects of ethnicity and violent content on rubric scores in writing samples.Journal of Educational Research, 93(6)367–372. -
Diener, M. J. (2010). Cohen’s d.Corsini Encyclopedia of Psychology. doi: 10.1002/9780470479216.corpsy0200 -
Dockrell, J. E., Lindsay, G., Connelly, V., & Mackie, C. (2007). Constraints in the production of written text in children with specific language impairments.Exceptional Children, 73(2)147–164. - Education Northwest. (2006). 6 + 1 trait writing. Retrieved from http://www.educationnorthwest.org
-
V. B. Edwards (Ed.), 2002). Quality counts 2002: Building blocks for success. State efforts in early childhood education.Education Week, 21(17) -
Fey, M. E., Catts, H. W., Proctor-Williams, K., Tomblin, J. B., & Zhang, X. (2004). Oral and written story composition skills of children with language impairment.Journal of Speech, Language, and Hearing Research, 47(6)1301–1318. -
Field, A. (2009). Discovering statistics using SPSS: Third edition. London, England: Sage. -
Gamaroff, R. (2000). Rater reliability in language assessment: The bug of all bears.System, 28(1)31–53. -
Gillam, R. B., & Johnston, J. R. (1992). Spoken and written language relationships in language/learning-impaired and normally achieving school-age children.Journal of Speech and Hearing Research, 35(6)1303–1315. -
Goertz, M. E., Duffy, M. C., & Le Floch, K. C. (2001). Assessment and accountability systems in the 50 states, 1999–2000 (CPRE Research Report Series No. RR-046). Philadelphia, PA: Consortium for Policy Research in Education. -
Graham, S. (1990). The role of production factors in learning disabled students' compositions.Journal of Educational Psychology, 82(4)781–791. -
Graham, S. (1999). Handwriting and spelling instruction for students with learning disabilities: A review.Learning Disability Quarterly, 22(2)78–98. -
Graham, S., Berninger, V., Weintraub, N., & Schafer, W. (1998). Development of handwriting speed and legibility in grades 1–9.Journal of Educational Research, 92(1)42–52. -
Graham, S., Harris, K. R., & Larsen, L. (2001). Prevention and intervention of writing difficulties for students with learning disabilities.Learning Disabilities: Research & Practice, 16(2)74–84. -
Heefner, D. L., & Shaw, P. C. (1996). Assessing the written narratives of deaf students using the six-trait analytical scale.Volta Review, 98(1)147–168. -
Hughes, D., McGillivray, L., & Schmidek, M. (1997). Guide to narrative language: Procedures for assessment. Eau Claire, WI: Thinking Publications. -
Hunt, K. W. (1970). Syntactic maturity in schoolchildren and adults.Monographs of the Society for Research in Child Development, 35(1)1–67. -
Johnson, D. J., & Carlisle, J. F. (1996). A study of handwriting in written stories of normal and learning disabled children.Reading and Writing: An Interdisciplinary Journal, 8(1)45–59. -
Leonard, L. (1998). Children with specific language impairment. Cambridge, MA: MIT Press. -
Mabry, L. (1999). Writing to the rubric: Lingering effects of traditional standardized testing on direct writing assessment.Phi Delta Kappan, 80(9)673–679. -
Mackie, C., & Dockrell, J. E. (2004). The nature of written language deficits in children with SLI.Journal of Speech, Language, and Hearing Research, 47(6)1469–1482. -
Masterson, J. J., & Crede, L. A. (1999). Learning to spell: Implications for assessment and intervention.Language, Speech, and Hearing Services in Schools, 30, 243–254. -
McFadden, T. U., & Gillam, R. B. (1996). An examination of the quality of narratives produced by children with language disorders.Language, Speech, and Hearing Services in Schools, 27(1)48–56. -
Merritt, D. D., & Liles, B. Z. (1987). Story grammar ability in children with and without language disorder: Story generation, story retelling, and story comprehension.Journal of Speech and Hearing Research, 30(4)539–552. -
Miller, J., & Iglesias, A. (2004). Systematic Analysis of Language Transcripts, English & Spanish (Version 8) [Computer software].Madison, WI: University of Wisconsin–Madison, Waisman Center, Language Analysis Laboratory. -
Myers, M. (1981). A procedure for writing assessment and holistic scoring. Urbana, IL: NCTE. -
Myklebust, H. R. (1965). Picture Story Language Test. New York, NY: Grune & Stratton. -
Nelson, N. W., & Van Meter, A. M. (2007). Measuring written language ability in narrative samples.Reading & Writing Quarterly, 23, 287–309. -
Osborn-Popp, S. E. (2001). Comparability and linking in direct writing assessment: Benchmarks, discourse mode, and grade level. (Doctoral dissertation). Retrieved from Dissertations & Theses at Arizona State University. (No. AAT 3031474). -
Popp, S. E. O., Ryan, J. M., Thompson, M. S., & Behrens, J. T. (2003, April). Operationalizing the rubric: The effect of benchmark selection on the assessed quality of writing.Paper presented at the annual meeting of the American Educational Research Association, Chicago, IL . -
Reilly, J., Losh, M., Bellugi, U., & Wulfeck, B. (2004). “Frog, where are you?” Narratives in children with specific language impairment, early focal brain injury, and Williams syndrome.Brain and Language, 2(88)229–247. -
Schoonen, R., Vergeer, M., & Eiting, M. (1997). The assessment of writing ability: Expert readers versus lay readers.Language Testing, 14(2)157–184. -
Scott, C. M., & Windsor, J. (2000). General language performance measures in spoken and written narrative and expository discourse of school-age children with language learning disabilities.Journal of Speech, Language, and Hearing Research, 43(2)324–339. -
Semel, E., Wiig, E., & Secord, W. (2003). Clinical Evaluation of Language Fundamentals—Fourth Edition. San Antonio, TX: The Psychological Corporation. -
Stein, N. L., & Glenn, C. G. (1979). An analysis of story comprehension in elementary school children: A test of a schema.InR. O. Freedle (Ed.), New directions in discourse processing (pp. 53–120). Norwood, NJ: Ablex. -
Stuhlmann, J., Daniel, C., Dellinger, A., Denny, K. R., & Powers, T. (1999). A generalizability study of the effects of training on teachers' ability to rate children’s writing using a rubric.Reading Psychology, 20(2)107–127. -
Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (3rd ed.). New York, NY: Pearson. - U.S. Department of Education. (2002). No child left behind. Retrieved http://www.ed.gov/nclb/landing.jhtml
-
Wiederholt, J. L., & Bryant, B. R. (2001). Gray Oral Reading Tests—Fourth Edition. Austin, TX: Pro-Ed. -
Windsor, J., Scott, C. M., & Street, C. K. (2000). Verb and noun morphology in the spoken and written language of children with language learning disabilities.Journal of Speech, Language, and Hearing Research, 43(6)1322–1336.