Standardizing Assessment of Spoken Discourse in Aphasia: A Working Group With Deliverables
Publication: American Journal of Speech-Language Pathology
Volume 30, Number 1S
Pages 491-502
Abstract
Purpose
The heterogeneous nature of measures, methods, and analyses reported in the aphasia spoken discourse literature precludes comparison of outcomes across studies (e.g., meta-analyses) and inhibits replication. Furthermore, funding and time constraints significantly hinder collecting test–retest data on spoken discourse outcomes. This research note describes the development and structure of a working group, designed to address major gaps in the spoken discourse aphasia literature, including a lack of standardization in methodology, analysis, and reporting, as well as nominal data regarding the psychometric properties of spoken discourse outcomes.
Method
The initial initiatives for this working group are to (a) propose recommendations regarding standardization of spoken discourse collection, analysis, and reporting in aphasia, based on the results of an international survey and a systematic literature review and (b) create a database of test–retest spoken discourse data from individuals with and without aphasia. The survey of spoken discourse collection, analysis, and interpretation procedures was distributed to clinicians and researchers involved in aphasia assessment and rehabilitation from September to November 2019. We will publish survey results and recommend standards for collecting, analyzing, and reporting spoken discourse in aphasia. A multisite endeavor to collect test–retest spoken discourse data from individuals with and without aphasia will be initiated. This test–retest information will be contributed to a central site for transcription and analysis, and data will be subsequently openly curated.
Conclusion
The goal of the working group is to create recommendations for field-wide standards in methods, analysis, and reporting of spoken discourse outcomes, as has been done across other related disciplines (e.g., Consolidated Standards of Reporting Trials, Enhancing the Quality and Transparency of Health Research, Committee on Best Practice in Data Analysis and Sharing). Additionally, the creation of a database through our multisite collaboration will allow the identification of psychometrically sound outcome measures and norms that can be used by clinicians and researchers to assess spoken discourse abilities in aphasia.
Get full access to this article
View all available purchase options and get full access to this article.
References
Armstrong, E. (2000). Aphasic discourse analysis: The story so far. Aphasiology, 14(9), 875–892.
Beeke, S., Maxim, J., & Wilkinson, R. (2007). Using conversation analysis to assess and treat people with aphasia. Seminars in Speech and Language, 28(2), 136–147.
Boyle, M. (2014). Test–retest stability of word retrieval in aphasic discourse. Journal of Speech, Language, and Hearing Research, 57(3), 966–978.
Boyle, M. (2015). Stability of word-retrieval errors with the AphasiaBank stimuli. American Journal of Speech-Language Pathology, 24(4), S953–S960.
Brady, M. C., Kelly, H., Godwin, J., Enderby, P., & Campbell, P. (2016). Speech and language therapy for aphasia following stroke. Cochrane Database of Systematic Reviews, 6, CD000425.
Brookshire, R. H. (1983). Subject description and generality of results in experiments with aphasic adults. Journal of Speech and Hearing Disorders, 48(4), 342–346.
Brookshire, R. H., & Nicholas, L. E. (1994). Speech sample size and test–retest stability of connected speech measures for adults with aphasia. Journal of Speech and Hearing Research, 37(2), 399–407.
Bryant, L., Ferguson, A., & Spencer, E. (2016). Linguistic analysis of discourse in aphasia: A review of the literature. Clinical Linguistics & Phonetics, 30(7), 489–518.
Capilouto, G. J., Wright, H. H., & Wagovich, S. A. (2006). Reliability of main event measurement in the discourse of individuals with aphasia. Aphasiology, 20(2–4), 205–216.
Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6(4), 284–290.
Clinical Centre for Research Excellence (CCRE) in Aphasia Rehabilitation. (2014). Aphasia rehabilitation best practice statements. Comprehensive supplement to the Australian aphasia rehabilitation pathway. CCRE in Aphasia Rehabilitation.
Damico, J. S., Oelschlaeger, M., & Simmons-Mackie, N. (1999). Qualitative methods in aphasia research: Conversation analysis. Aphasiology, 13(9–11), 667–679.
Dietz, A., & Boyle, M. (2018). Discourse measurement in aphasia research: Have we reached the tipping point? Aphasiology, 32(4), 459–464.
Donoghue, D., Physiotherapy Research and Older People (PROP) group., & Stokes, E. K. (2009). How much change is true change? The minimum detectable change of the Berg Balance Scale in elderly people. Journal of Rehabilitation Medicine, 41(5), 343–346.
Dutta, M., Murray, L., & Stark, B. C. (2019). Standardizing assessment of spoken discourse in aphasia: Directions for future research. Paper presented at the Clinical Aphasiology Conference, Whitefish, MT, United States.
Fergadiotis, G., Wright, H. H., & Capilouto, G. J. (2011). Productive vocabulary across discourse types. Aphasiology, 25(10), 1261–1278.
Finestack, L. H., Payesteh, B., Disher, J. R., & Julien, H. M. (2014). Reporting child language sampling procedures. Journal of Speech, Language, and Hearing Research, 57(6), 2274–2279.
Fitzpatrick, R., Davey, C., Buxton, M., & Jones, D. (1998). Evaluating patient-based outcome measures for use in clinical trials: A review. Health Technology Assessment, 2(14), 1–73.
Fromm, D., Forbes, M., Holland, A., Dalton, S. G., Richardson, J., & MacWhinney, B. (2017). Discourse characteristics in aphasia beyond the Western Aphasia Battery cutoff. American Journal of Speech-Language Pathology, 26(3), 762–768.
Gagnon, D. A., Schwartz, M. F., Martin, N., Dell, G. S., & Saffran, E. M. (1997). The origins of formal paraphasias in aphasics' picture naming. Brain and Language, 59(3), 450–472.
Galski, T., Tompkins, C., & Johnston, M. V. (1998). Competence in discourse as a measure of social integration and quality of life in persons with traumatic brain injury. Brain Injury, 12(9), 769–782.
Gearing, R. E., El-Bassel, N., Ghesquiere, A., Baldwin, S., Gillies, J., & Ngeow, E. (2011). Major ingredients of fidelity: A review and scientific guide to improving quality of intervention research implementation. Clinical Psychology Review, 31(1), 79–88.
Goodglass, H. (1993). Understanding aphasia. Academic Press.
Kim, H., Kintz, S., Zelnosky, K., & Wright, H. H. (2019). Measuring word retrieval in narrative discourse: Core lexicon in aphasia. International Journal of Language & Communication Disorders, 54(1), 62–78.
Kintz, S., & Wright, H. H. (2018). Discourse measurement in aphasia research. Aphasiology, 32(4), 472–474.
Kong, A. P.-H. (2009). The use of main concept analysis to measure discourse production in Cantonese-speaking persons with aphasia: A preliminary report. Journal of Communication Disorders, 42(6), 442–464.
MacWhinney, B. (2000). The CHILDES project: Tools for analyzing talk (3rd ed.). Erlbaum.
MacWhinney, B., Fromm, D., Forbes, M., & Holland, A. (2011). AphasiaBank: Methods for studying discourse. Aphasiology, 25(11), 1286–1307.
McNeil, M. R., Doyle, P. J., Fossett, T. R. D., Park, G. H., & Goda, A. J. (2001). Reliability and concurrent validity of the information unit scoring metric for the story retelling procedure. Aphasiology, 15(10–11), 991–1006.
McNeil, M. R., Doyle, P. J., Park, G. H., Fossett, T. R. D., & Brodsky, M. B. (2002). Increasing the sensitivity of the story retell procedure for the discrimination of normal elderly subjects from persons with aphasia. Aphasiology, 16(8), 815–822.
Meek, P. M., Nail, L. M., Barsevick, A., Schwartz, A. L., Stephen, S., Whitmer, K., Beck, S. L., Jones, L. S., & Walker, B. L. (2000). Psychometric testing of fatigue instruments for use with cancer patients. Nursing Research, 49(4), 181–190.
Murray, L. L., & Clark, H. M. (2015). Neurogenic disorders of language and cognition: Evidence-based clinical practice. Pro-Ed.
Nicholas, L. E., & Brookshire, R. H. (1993). A system for quantifying the informativeness and efficiency of the connected speech of adults with aphasia. Journal of Speech and Hearing Research, 36(2), 338–350.
Nicholas, L. E., & Brookshire, R. H. (1995). Presence, completeness, and accuracy of main concepts in the connected speech of non-brain-damaged adults and adults with aphasia. Journal of Speech and Hearing Research, 38(1), 145–156.
Nichols, T. E., Das, S., Eickhoff, S. B., Evans, A. C., Glatard, T., Hanke, M., Kriegeskorte, N., Milham, M. P., Poldrack, R. A., Poline, J.-B., Proal, E., Thirion, B., Van Essen, D. C., White, T., & Yeo, B. T. T. (2017). Best practices in data analysis and sharing in neuroimaging using MRI. Nature Neuroscience, 20(3), 299–303.
Pritchard, M., Hilari, K., Cocks, N., & Dipper, L. (2017). Reviewing the quality of discourse information measures in aphasia. International Journal of Language & Communication Disorders, 52(6), 689–732.
Pritchard, M., Hilari, K., Cocks, N., & Dipper, L. (2018). Psychometric properties of discourse measures in aphasia: Acceptability, reliability, and validity. International Journal of Language & Communication Disorders, 53(6), 1078–1093.
Richardson, J. D., Hudspeth Dalton, S. G., Shafer, J., & Patterson, J. (2016). Assessment fidelity in aphasia research. American Journal of Speech-Language Pathology, 25(4S), S788–S797.
Sim, P., Power, E., & Togher, L. (2013). Describing conversations between individuals with traumatic brain injury (TBI) and communication partners following communication partner training: Using exchange structure analysis. Brain Injury, 27(6), 717–742.
Squires, J. E., Estabrooks, C. A., O'Rourke, H. M., Gustavsson, P., Newburn-Cook, C. V., & Wallin, L. (2011). A systematic review of the psychometric properties of self-report research utilization measures used in healthcare. Implementation Science, 6(1), 83.
Stark, B. C. (2019). A comparison of three discourse elicitation methods in aphasia and age-matched adults: Implications for language assessment and outcome. American Journal of Speech-Language Pathology, 28(3), 1067–1083.
Tate, R. L. (2010). A compendium of tests, scales, and questionnaires: The practitioner's guide to measuring outcomes after acquired brain impairment. Psychology Press.
Ulatowska, H. K., Allard, L., Reyes, B. A., Ford, J., & Chapman, S. (1992). Conversational discourse in aphasia. Aphasiology, 6(3), 325–330.
Wallace, S. J., Worrall, L., Rose, T., & Le Dorze, G. (2017). Which treatment outcomes are most important to aphasia clinicians and managers? An international e-Delphi consensus study. Aphasiology, 31(6), 643–673.
Wallace, S. J., Worrall, L., Rose, T., Le Dorze, G., Breitenstein, C., Hilari, K., Babbitt, E., Bose, A., Brady, M., Cherney, L. R., Copland, D., Cruice, M., Enderby, P., Hersh, D., Howe, T., Kelly, H., Kiran, S., Laska, A.-C., Marshall, J., … Webster, J. (2019). A core outcome set for aphasia treatment research: The ROMA consensus statement. International Journal of Stroke, 14(2), 180–185.
Walter, S. D., Eliasziw, M., & Donner, A. (1998). Sample size and optimal designs for reliability studies. Statistics in Medicine, 17(1), 101–110.
Weir, J. P. (2005). Quantifying test–retest reliability using the intraclass correlation coefficient and the SEM. Journal of Strength and Conditioning Research, 19(1), 231–240.
Winstein, C. J., Stein, J., Arena, R., Bates, B., Cherney, L. R., Cramer, S. C., Deruyter, F., Eng, J. J., Fisher, B., Harvey, R. L., Lang, C. E., Mackay-Lyons, M., Ottenbacher, K. J., Pugh, S., Reeves, M. J., Richards, L. G., Stiers, W., & Zorowitz, R. D. (2016). Guidelines for adult stroke rehabilitation and recovery. Stroke, 47(6), e98–e169.
World Health Organization. (2018). International classification of functioning, disability and health. http://www.who.int/classifications/icf/en/
Wright, H. H., & Capilouto, G. J. (2009). Manipulating task instructions to change narrative discourse performance. Aphasiology, 23(10), 1295–1308.
Information & Authors
Information
Published In
American Journal of Speech-Language Pathology
Volume 30 • Number 1S • February 2021
Pages: 491-502
PubMed: 32585117
Copyright
Copyright © 2020 American Speech-Language-Hearing Association.
History
- Received: Sep 12, 2019
- Revised: Nov 21, 2019
- Accepted: Jan 3, 2020
- Published online: Jun 25, 2020
- Published in issue: Feb 11, 2021
Authors
Metrics & Citations
Metrics
Article Metrics
View all metricsCitations
Download article citation data for:
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
Citing Literature
- Junhua Ding, Daniel Mirman, Data-driven classification of narrative speech characteristics in stroke aphasia distinguishes neurological and strategic contributions, Cortex, 10.1016/j.cortex.2025.03.006, 186, (61-73), (2025).
- Carmit Altman, Mira Goral, Towards an integrated narrative analysis in nonfluent aphasia, Aphasiology, 10.1080/02687038.2025.2479793, (1-25), (2025).
- Guillaume Duboisdindien, Catherine T. Bolly, Heterogeneity of verbal and gestural functions in spoken discourse with MCI, Journal of Neurolinguistics, 10.1016/j.jneuroling.2024.101227, 73, (101227), (2025).
- Manaswita Dutta, Bijoyaa Mohapatra, Expanding the scope: multimodal dimensions in aphasia discourse analysis—preliminary findings, Frontiers in Human Neuroscience, 10.3389/fnhum.2024.1419311, 18, (2024).
- Robert Cavanaugh, Michael Walsh Dickey, William D. Hula, Davida Fromm, Jennifer Golovin, Julie Wambaugh, Gerasimos Fergadiotis, William S. Evans, Determinants of Multilevel Discourse Outcomes in Anomia Treatment for Aphasia, Journal of Speech, Language, and Hearing Research, 10.1044/2024_JSLHR-24-00030, 67, 9, (3094-3112), (2024).
- Charalambos Themistocleous, Open Brain AI and language assessment, Frontiers in Human Neuroscience, 10.3389/fnhum.2024.1421435, 18, (2024).
- Reem S. W. Alyahya, The development of a novel, standardized, norm‐referenced Arabic Discourse Assessment Tool (ADAT), including an examination of psychometric properties of discourse measures in aphasia, International Journal of Language & Communication Disorders, 10.1111/1460-6984.13083, 59, 5, (2103-2117), (2024).
- Marianne Casilio, Anna V Kasdan, Katherine Bryan, Kiiya Shibata, Sarah M Schneck, Deborah F Levy, Jillian L Entrup, Caitlin Onuscheck, Michael de Riesthal, Stephen M Wilson, Four dimensions of naturalistic language production in aphasia after stroke, Brain, 10.1093/brain/awae195, 148, 1, (291-312), (2024).
- Lauryn Zipse, Jeanne Gallée, Stefanie Shattuck-Hufnagel, A targeted review of prosodic production in agrammatic aphasia, Neuropsychological Rehabilitation, 10.1080/09602011.2024.2362243, 35, 4, (863-903), (2024).
- Jacquie Kurland, Anna Liu, Vishnupriya Varadharaju, Polly Stokes, Robert Cavanaugh, Reliability of the Brief Assessment of Transactional Success in Communication in Aphasia, Aphasiology, 10.1080/02687038.2024.2351029, 39, 3, (363-384), (2024).
- See more
View Options
Sign In Options
ASHA member? If so, log in with your ASHA website credentials for full access.
Member Login