Abstract
Purpose
The Consensus Auditory-Perceptual Evaluation of Voice (CAPE-V) was developed to provide a protocol and form for clinicians to use when assessing the voice quality of adults with voice disorders (Kempster, Gerratt, Verdolini Abbott, Barkmeier-Kramer, & Hillman, 2009). This study examined the reliability and the empirical validity of the CAPE-V when used by experienced voice clinicians judging normal and disordered voices.
Method
The validity of the CAPE-V was examined in 2 ways. First, we compared judgments made by 21 raters of 22 normal and 37 disordered voices using the CAPE-V and the GRBAS (grade, roughness, breathiness, asthenia, strain; see Hirano, 1981) scales. Second, we compared our raters' judgments of overall severity to a priori consensus judgments of severity for the 59 voices.
Results
Intrarater reliability coefficients for the CAPE-V ranged from .82 for breathiness to .35 for strain; interrater reliability ranged from .76 for overall severity to .28 for pitch.
Conclusions
Although both CAPE-V and GRBAS reliability coefficients varied across raters and parameters, this study reports slightly improved rater reliability using the CAPE-V to make perceptual judgments of voice quality in comparison to the GRBAS scale. The results provide evidence for the empirical (concurrent) validity of the CAPE-V.
References
-
Awan, S., & Lawson, L. (2009). The effect of anchor modality on the reliability of vocal severity ratings.Journal of Voice, 23, 341–352. -
Bangayan, P., Long, C., Alwan, A. A., Kreiman, J., & Gerratt, B. (1997). Analysis bysynthesis of pathological voices using the Klatt synthesizer.Speech Communication, 22, 343–368. -
Behrman, A. (2005). Common practices of voice therapists in the evaluation of patients.Journal of Voice, 19, 454–469. -
Carding, P. N., Wilson, J. A., MacKenzie, K., & Deary, I. J. (2009). Measuring voice outcomes: State of the science review.Journal of Laryngology and Otology, 123, 823–829. -
Carmines, E. G., & Zeller, R. A. (1979). Reliability and validity assessment.InM. S. Lewis-Beck (Ed.), Quantitative applications in the social science (pp. 9–27). Thousand Oaks, CA: Sage. -
Chan, K., & Yiu, E. (2006). A comparison of two perceptual voice evaluation programs for naïve listeners.Journal of Voice, 20, 229–241. -
Cook, D. A., & Beckman, T. J. (2006). Current concepts in validity and reliability for psychometric instruments: Theory and application.American Journal of Medicine, 119, 166.e7–166.e16. -
Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests.Psychological Bulletin, 52, 281–302. -
De Bodt, M. S., Wuyts, F. L., Van de Heyning, P. H., & Croux, C. (1997). Test-retest study of the GRBAS scale: Influence of experience and professional background on perceptual rating of voice quality.Journal of Voice, 11, 74–80. -
Dejonckere, P. H., Obbens, C., de Moor, G. M., & Wieneke, G. H. (1993). Perceptual evaluation of dysphonia: Reliability and relevance.Folia Phoniatrica et Logopaedica, 45, 76–83. -
de Krom, G. (1994). Consistency and reliability of voice quality ratings for different types of speech fragments.Journal of Speech and Hearing Research, 37, 985–1000. -
Delyiski, D. D., Shaw, H. S., & Evans, M. K. (2005). Influence of sampling rate on accuracy and reliability of acoustic voice analysis.Logopedics, Phoniatrics and Vocology, 30(2)55–62. -
DeVon, H. A., Block, M. E., Moyle-Wright, P., Ernst, D. M., Hayden, S. J., Lazzara, D. J., & Kostas-Polston, E. (2007). A psychometric toolbox for testing validity and reliability.Journal of Nursing Scholarship, 39, 155–164. -
Eadie, T. L., & Baylor, C. R. (2006). The effect of perceptual training on inexperienced listeners' judgments of dysphonic voice.Journal of Voice, 20, 527–544. -
Gerratt, B., & Kreiman, J. (2001). Measuring vocal quality with speech synthesis.The Journal of the Acoustical Society of America, 110, 2560–2566. -
Gorodetsky, R., Amir, G., & Yarom, R. (1992). Effect of ionizing radiation on neuromuscular junctions in mouse tongues.International Journal of Radiation Biology, 61, 539–544. -
Harshbarger, T. R. (1977). Introductory statistics: A decision map. New York, NY: Macmillan. -
Hirano, M. (1981). Clinical examination of voice. New York, NY: Springer-Verlag. -
Karnell, M., Melton, S., Childes, J., Coleman, T., Dailey, S., & Hoffman, H. (2007). Reliability of clinician-based (GRBAS and CAPE-V) and patient-based (V-RQOL and IPVI) documentation of voice disorders.Journal of Voice, 21, 576–590. -
Kelchner, L. N., Brehm, S. B., Weinrich, B., Middendorf, J., deAlarcon, A., Levin, L., & Elluru, R. (2010). Perceptual evaluation of severe pediatric voice disorders: Rater reliability using the Consensus Auditory Perceptual Evaluation of Voice.Journal of Voice, 24, 441–449. doi:10.1016/j.jvoice.2008.09.004 -
Kelly, P. A., O’Malley, K. J., Kallen, M. A., & Ford, M. E. (2005). Integrating validity theory with use of measurement instruments in clinical settings.Health Services Research, 40, 1605–1619. -
Kempster, G. B., Gerratt, B. R., Verdolini Abbott, K., Barkmeier-Kramer, J., & Hillman, R. E. (2009). Consensus Auditory-Perceptual Evaluation of Voice: Development of a standardized clinical protocol.American Journal of Speech-Language Pathology, 18, 124–132. -
Kent, R. (1996). Hearing and believing: Some limits to the auditory-perceptual assessment of speech and voice disorders.American Journal of Speech-Language Pathology, 5(3)7–23. -
Kreiman, J., & Gerratt, B. (1996). Perceptual structure of pathologic voice quality.The Journal of the Acoustical Society of America, 100, 1787–1795. -
Kreiman, J., & Gerratt, B. (1998). Validity of rating scale measures of voice quality.The Journal of the Acoustical Society of America, 104, 1598–1608. -
Kreiman, J., & Gerratt, B. (2000). Sources of listener disagreement in voice quality assessment.The Journal of the Acoustical Society of America, 108, 1867–1876. -
Kreiman, J., Gerratt, B. R., & Ito, M. (2007). When and why listeners disagree in voice quality assessment tasks.The Journal of the Acoustical Society of America, 122, 2354–2364. -
Kreiman, J., Gerratt, B. R., Kempster, G. B., Erman, A., & Berke, G. S. (1993). Perceptual evaluation of voice quality: Review, tutorial, and a framework for future research.Journal of Speech and Hearing Research, 36, 21–40. -
Lazarus, C. L. (2009). Effects of chemoradiotherapy on voice and swallowing.Current Opinion in Otolaryngology & Head and Neck Surgery, 17, 172–178. -
Marks, L., & Algom, D. (1998). Psychophysical scaling.InM. H. Birnbaum (Ed.), Measurement, judgment, and decision making (pp. 81–178). San Diego, CA: Academic Press. -
Sechrest, L. (2005). Validity of measures is no simple matter.Health Services Research, 40, 1584–1604. -
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin. -
Shrivastav, R., Sapienza, C. M., & Nandur, V. (2005). Application of psychometric theory to the measurement of voice quality using rating scales.Journal of Speech, Language, and Hearing Research, 48, 323–335. -
Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability.Psychological Bulletin, 86, 420–428. -
Stevens, S. S. (1975). Psychophysics. New York, NY: Wiley. -
Verdolini, K., Rosen, C. A., & Branski, R. C. (2006). Classification manual for voice disorders—I. Mahwah, NJ: Erlbaum. -
Webb, A. L., Carding, P. N., Deary, I. J., MacKenzie, K., Steen, N., & Wilson, J. A. (2004). The reliability of three perceptual evaluation scales for dysphonia.European Archives of Otorhinolaryngology, 261, 429–434. -
Winer, B. J. (1971). Statistical principles in experimental design. New York, NY: McGraw-Hill. -
Zraick, R. I., Wendel, K. W., & Smith-Olinde, L. (2005). The effect of speaking task on perceptual judgment of the severity of dysphonic voice.Journal of Voice, 19, 574–581.