Abstract
Purpose
The COVID-19 pandemic has drastically increased the use of telehealth. Prior studies of telehealth clinical swallowing evaluations provide positive evidence for telemanagement of swallowing. However, the reliability of these measures in clinical practice, as opposed to well-controlled research conditions, remains unknown. This study aimed to investigate the reliability of outcome measures derived from clinical swallowing tele-evaluations in real-world clinical practice (e.g., variability in devices and Internet connectivity, lack of in-person clinician assistance, or remote patient/caregiver training).
Method
Seven raters asynchronously judged clinical swallowing tele-evaluations of 12 movement disorders patients. Outcomes included the Timed Water Swallow Test (TWST), Test of Masticating and Swallowing Solids (TOMASS), and common observations of oral intake. Statistical analyses were performed to examine inter- and intrarater reliability, as well as qualitative analyses exploring patient and clinician-specific factors impacting reliability.
Results
Forty-four trials were included for reliability analyses. All rater dyads demonstrated “good” to “excellent” interrater reliability for measures of the TWST (intraclass correlation coefficients [ICCs] ≥ .93) and observations of oral intake (≥ 77% agreement). The majority of TOMASS outcomes demonstrated “good” to “excellent” interrater reliability (ICCs ≥ .84), with the exception of the number of bites (ICCs = .43–.99) and swallows (ICCs = .21–.85). Immediate and delayed intrarater reliability were “excellent” for most raters across all tasks, ranging between ICCs of .63 and 1.00. Exploratory factors potentially impacting reliability included infrequent instances of suboptimal video quality, reduced camera stability, camera distance, and obstruction of the patient's mouth during tasks.
Conclusions
Subjective observations of oral intake and objective measures taken from the TWST and the TOMASS can be reliably measured via telehealth in clinical practice. Our results provide support for the feasibility and reliability of telehealth for outpatient clinical swallowing evaluations during COVID-19 and beyond.
Supplemental Material
References
- American Speech-Language-Hearing Association. (2020). Telepractice (practice portal). https://www.asha.org/Practice-Portal/Professional-Issues/Telepractice/
- American Telemedicine Association. (2014). Core operational guidelines for telehealth services involving provider–patient interactions. https://southwesttrc.org/sites/default/files/resources/standards/ATA%20Core%20Guidelines.pdf
-
Athukorala, R. P., Jones, R. D., Sella, O., & Huckabee, M.-L. (2014). Skill training for swallowing rehabilitation in patients with Parkinson's disease.Archives of Physical Medicine and Rehabilitation, 95(7), 1374–1382. https://doi.org/10.1016/j.apmr.2014.03.001 -
Blue, R., Yang, A. I., Zhou, C., De Ravin, E., Teng, C. W., Arguelles, G. R., Huang, V., Wathen, C., Miranda, S. P., Marcotte, P., Malhotra, N. R., Welch, W. C., & Lee, J. Y. K. (2020). Telemedicine in the era of coronavirus disease 2019 (COVID-19): A neurosurgical perspective.World Neurosurgery, 139, 549–557. https://doi.org/10.1016/j.wneu.2020.05.066 -
Burns, C. L., Ward, E. C., Gray, A., Baker, L., Cowie, B., Winter, N., Rusch, R., Saxon, R., Barnes, S., & Turvey, J. (2019). Implementation of speech pathology telepractice services for clinical swallowing assessment: An evaluation of service outcomes, costs and consumer satisfaction.Journal of Telemedicine and Telecare, 25(9), 545–551. https://doi.org/10.1177/1357633X19873248 -
Burns, C. L., Ward, E. C., Hill, A. J., Kularatna, S., Byrnes, J., & Kenny, L. M. (2017). Randomized controlled trial of a multisite speech pathology telepractice service providing swallowing and communication intervention to patients with head and neck cancer: Evaluation of service outcomes.Head & Neck, 39(5), 932–939. https://doi.org/10.1002/hed.24706 -
Cichero, J. A. Y., Lam, P., Steele, C. M., Hanson, B., Chen, J., Dantas, R. O., Duivestein, J., Kayashita, J., Lecko, C., Murray, J., Pillay, M., Riquelme, L., & Stanschus, S. (2017). Development of International terminology and definitions for texture-modified foods and thickened fluids used in dysphagia management: The IDDSI framework.Dysphagia, 32(2), 293–314. https://doi.org/10.1007/s00455-016-9758-y -
Cook, D. A., & Beckman, T. J. (2006). Current concepts in validity and reliability for psychometric instruments: Theory and application.The American Journal of Medicine, 119(2), 166.e7–166.e16. https://doi.org/10.1016/j.amjmed.2005.10.036 -
Frank, U., van denEngel-Hoek, L., Nogueira, D., Schindler, A., Adams, S., Curry, M., & Huckabee, M.-L. (2019). International standardisation of the Test of Masticating and Swallowing Solids in children.Journal of Oral Rehabilitation, 46(2), 161–169. https://doi.org/10.1111/joor.12728 -
Frieden, T. R., & Lee, C. T. (2020). Identifying and interrupting superspreading events—Implications for control of severe acute respiratory syndrome coronavirus 2.Emerging Infectious Diseases, 26(6), 1059–1066. https://doi.org/10.3201/eid2606.200495 -
Gough, F., Budhrani, S., Cohn, E., Dappen, A., Leenknecht, C., Lewis, B., Mulligan, D. A., Randall, D., Rheuban, K., Roberts, L., Shanahan, T. J., Webster, K., Krupinski, E. A., Bashshur, R., & Bernard, J. (2015). ATA practice guidelines for live, on-demand primary and urgent care.Telemedicine and e-Health, 21(3), 233–241. https://doi.org/10.1089/tmj.2015.0008 -
Harris, P. A., Taylor, R., Thielke, R., Payne, J., Gonzalez, N., & Conde, J. G. (2009). Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics support.Journal of Biomedical Informatics, 42(2), 377–381. https://doi.org/10.1016/j.jbi.2008.08.010 -
Huckabee, M. L., McIntosh, T., Fuller, L., Curry, M., Thomas, P., Walshe, M., McCague, E., Battel, I., Nogueira, D., Frank, U., van denEngel-Hoek, L., & Sella-Weiss, O. (2018). The Test of Masticating and Swallowing Solids (TOMASS): Reliability, validity and international normative data.International Journal of Language & Communication Disorders, 53(1), 144–156. https://doi.org/10.1111/1460-6984.12332 -
Hughes, T. A. T., & Wiles, C. M. (1996). Clinical measurement of swallowing in health and in neurogenic dysphagia.QJM: An International Journal of Medicine, 89(2), 109–116. https://doi.org/10.1093/qjmed/89.2.109 -
Kantarcigil, C., Sheppard, J. J., Gordon, A. M., Friel, K. M., & Malandraki, G. A. (2016). A telehealth approach to conducting clinical swallowing evaluations in children with cerebral palsy.Research in Developmental Disabilities, 55, 207–217. https://doi.org/10.1016/j.ridd.2016.04.008 -
Koo, T. K., & Li, M. Y. (2016). A Guideline of selecting and reporting intraclass correlation coefficients for reliability research.Journal of Chiropractic Medicine, 15(2), 155–163. https://doi.org/10.1016/j.jcm.2016.02.012 -
Lamvik-Gozdzikowska, K., Guiu Hernandez, E., Apperley, O., McIntosh, T., & Huckabee, M.-L. (2019). Quantitative assessment of oral phase efficiency: Validation of the Test of Masticating and Swallowing Solids (TOMASS).International Journal of Language & Communication Disorders, 54(3), 444–450. https://doi.org/10.1111/1460-6984.12450 -
Malandraki, G. A. (2020). Telehealth recommendations for dysphagia management during COVID-19. Purdue I-Eat Research Lab. https://www.purdue.edu/i-eatlab/telehealth-recommendations-for-dysphagia-management-during-covid-19/ -
Malandraki, G. A., Markaki, V., Georgopoulos, V. C., Bauer, J. L., Kalogeropoulos, I., & Nanas, S. (2013). An international pilot study of asynchronous teleconsultation for oropharyngeal dysphagia.Journal of Telemedicine and Telecare, 19(2), 75–79. https://doi.org/10.1177/1357633x12474963 -
Malandraki, G. A., McCullough, G., He, X., McWeeny, E., & Perlman, A. L. (2011). Teledynamic evaluation of oropharyngeal swallowing.Journal of Speech, Language, and Hearing Research, 54(6), 1497–1505. https://doi.org/10.1044/1092-4388(2011/10-0284) -
Malandraki, G. A., Roth, M., & Sheppard, J. J. (2014). Telepractice for pediatric dysphagia: A case study.International Journal of Telerehabilitation, 6(1), 3–16. https://doi.org/10.5195/IJT.2014.6135 -
Mann, D. M., Chen, J., Chunara, R., Testa, P. A., & Nov, O. (2020). COVID-19 transforms health care through telemedicine: Evidence from the field.Journal of the American Medical Informatics Association, 27(7), 1132–1135. https://doi.org/10.1093/jamia/ocaa072 -
McEachern, W., Kirk, A., Morgan, D. G., Crossley, M., & Henry, C. (2008). Reliability of the MMSE administered in-person and by telehealth.Canadian Journal of Neurological Sciences/Journal Canadien Des Sciences Neurologiques, 35(5), 643–646. https://doi.org/10.1017/S0317167100009458 -
McHorney, C. A., Bricker, D. E., Kramer, A. E., Rosenbek, J. C., Robbins, J., Chignell, K. A., Logemann, J. A., & Clarke, C. (2000). The SWAL-QOL outcomes tool for oropharyngeal dysphagia in adults: I. Conceptual foundation and item development.Dysphagia, 15(3), 115–121. https://doi.org/10.1007/s004550010012 -
McHorney, C. A., Robbins, J., Lomax, K., Rosenbek, J. C., Chignell, K., Kramer, A. E., & Earl Bricker, D. E. (2002). The SWAL-QOL and SWAL-CARE outcomes tool for oropharyngeal dysphagia in adults: III. Documentation of reliability and validity.Dysphagia, 17(2), 97–114. https://doi.org/10.1007/s00455-001-0109-1 -
Miles, A., Connor, N. P., Desai, R. V., Jadcherla, S., Allen, J., Brodsky, M., Garand, K. L., Malandraki, G. A., McCulloch, T. M., Moss, M., Murray, J., Pulia, M., Riquelme, L. F., & Langmore, S. E. (2020). Dysphagia care across the continuum: A multidisciplinary Dysphagia Research Society taskforce report of service-delivery during the COVID-19 global pandemic.Dysphagia. https://doi.org/10.1007/s00455-020-10153-8 -
Morrell, K., Hyers, M., Stuchiner, T., Lucas, L., Schwartz, K., Mako, J., Spinelli, K. J., & Yanase, L. (2017). Telehealth stroke dysphagia evaluation is safe and effective.Cerebrovascular Diseases, 44(3–4), 225–231. https://doi.org/10.1159/000478107 -
Nathadwarawala, K. M., Nicklin, J., & Wiles, C. M. (1992). A timed test of swallowing capacity for neurological patients.Journal of Neurology, Neurosurgery & Psychiatry, 55(9), 822–825. https://doi.org/10.1136/jnnp.55.9.822 -
Pasnick, S., Carlos, W. G., Dela Cruz, C. S., Gross, J. E., Garrison, G., & Jamil, S. (2020). SARS-CoV-2 transmission and the risk of aerosol generating procedures.American Journal of Respiratory and Critical Care Medicine, 202(4), 13–14. https://doi.org/10.1164/rccm.2020C11 - R Core Team. (2018). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/
-
Sharma, S., Ward, E. C., Burns, C., Theodoros, D., & Russell, T. (2011). Assessing swallowing disorders online: A pilot telerehabilitation study.Telemedicine and e-Health, 17(9), 688–695. https://doi.org/10.1089/tmj.2011.0034 -
Stadnytskyi, V., Bax, C. E., Bax, A., & Anfinrud, P. (2020). The airborne lifetime of small speech droplets and their potential importance in SARS-CoV-2 transmission.Proceedings of the National Academy of Sciences of the United States of America, 117(22), 11875–11877. https://doi.org/10.1073/pnas.2006874117 -
Ward, E. C., Burns, C. L., Theodoros, D. G., & Russell, T. G. (2013). Evaluation of a clinical service model for dysphagia assessment via telerehabilitation.International Journal of Telemedicine and Applications, 2013, Article ID 918526. https://doi.org/10.1155/2013/918526 -
Ward, E. C., Burns, C. L., Theodoros, D. G., & Russell, T. G. (2014). Impact of dysphagia severity on clinical decision making via telerehabilitation.Telemedicine and e-Health, 20(4), 296–303. https://doi.org/10.1089/tmj.2013.0198 -
Ward, E. C., Crombie, J., Trickey, M., Hill, A., Theodoros, D., & Russell, T. (2009). Assessment of communication and swallowing post-laryngectomy: A telerehabilitation trial.Journal of Telemedicine and Telecare, 15(5), 232–237. https://doi.org/10.1258/jtt.2009.081204 -
Ward, E. C., Sharma, S., Burns, C., Theodoros, D., & Russell, T. (2012). Validity of conducting clinical dysphagia assessments for patients with normal to mild cognitive impairment via telerehabilitation.Dysphagia, 27(4), 460–472. https://doi.org/10.1007/s00455-011-9390-9 -
Weidner, K., & Lowman, J. (2020). Telepractice for adult speech-language pathology services: A systematic review.Perspectives of the ASHA Special Interest Groups, 5(1), 326–338. https://doi.org/10.1044/2019_PERSP-19-00146 -
Workman, A. D., Jafari, A., Welling, D. B., Varvares, M. A., Gray, S. T., Holbrook, E. H., Scangas, G. A., Xiao, R., Carter, B. S., Curry, W. T., & Bleier, B. S. (2020). Airborne aerosol generation during endonasal procedures in the era of COVID-19: Risks and recommendations.Otolaryngology—Head & Neck Surgery, 163(3), 465–470. https://doi.org/10.1177/0194599820931805 - Zoom Video Communications, Inc.. (2016). Security guide. https://d24cgw3uvb9a9h.cloudfront.net/static/81625/doc/Zoom-SecurityWhite-Paper.pdf.
Add to your Mendeley library
