Additional Details Regarding the ASPEKT Videofluoroscopy Rating Method (Analysis of Swallowing Physiology: Events, Kinematics and Timing) Used for the Study “Reference Values for Healthy Swallowing Across the Range From Thin to Extremely Thick Liquids”
The ASPEKT method has been developed in the Swallowing Rehabilitation Research Laboratory at the Toronto Rehabilitation Institute, University Health Network, as a standard operating procedure for objective rating of videofluoroscopies (videofluoroscopic swallowing study [VFSS]) for research and involves the following steps:
1. VFSS Recording Review and Clipping
Prior to rating each VFSS recording, each full-length recording is reviewed to identify the time codes associated with onset/offset of the x-ray for each sip or bolus contained in the recording. The boundaries identified are then used to splice the original full-length recording into smaller video clips, each containing the swallows associated with a single bolus. These boundaries are entered into a spreadsheet, which is then passed to MATLAB for video clipping. The spliced bolus-level video clips, with no audio track, are labeled with a random file number, with the master key retained in a file on the lab research server. Bolus level video clips are generated in sets of 150, which are referred to as batches of videos.
2. Video Rating Assignment
Each bolus level video clip is randomly assigned to two raters. Raters have previously completed a training program and demonstrated competency in all required rating procedures.
3. Computer Setup and Software
Raters are provided a computer with Windows 7 (or later) operating system and two monitors (one for video viewing and another for data entry). The 64-bit ImageJ software (National Institutes of Health,
https://imagej.nih.gov) is used to review each bolus-level video clip. ImageJ allows the users to view the videos in real time as well as frame by frame (forward frame advancement as well as backward, if this is helpful). Raters are given the freedom to review the video clips as many times as they wish. Raters are instructed to avoid ImageJ contrast adjustment and enhancement tools.
4. Overview of the Parameters Rated
Figure A1, below, provides an overview of the different parameters that can be collected using the ASPEKT method.
5. Counting the Number of Swallows for Each Bolus (ASPEKT Method Step 2)
a. The rater is asked to review the entire bolus-level video clip and to identify the number of swallow(s). The following components must be present in order to consider a “swallow” to have occurred: (a) at least one of laryngeal elevation, hyoid excursion, and/or pharyngeal constriction and (b) upper esophageal sphincter opening (UESO). The term subswallows is used to refer to individual swallows when there is more than one swallow per bolus.
b. Subswallows are further qualitatively classified as initial (the first subswallow in the series), piecemeal (a higher order subswallow in which additional material is transported from the oral cavity), or clearing (a higher order swallow of pharyngeal residue with no additional material added from the oral cavity) swallows.
6. Rating Penetration–Aspiration (ASPEKT Method Step 3)
a. Swallow Level Results
1 = Material does not enter the airway.
2 = Material enters the airway, remains above the vocal folds, and is ejected from the airway.
3 = Material enters the airway, remains above the vocal folds, and is not ejected from the airway.
4 = Material enters the airway, contacts the vocal folds, and is ejected from the airway.
5 = Material enters the airway, contacts the vocal folds, and is not ejected from the airway.
6 = Material enters the airway, passes below the vocal folds, and is ejected into the larynx or out of the airway.
7 = Material enters the airway, passes below the vocal folds, and is not ejected from the trachea despite effort.
8 = Material enters the airway and passes below the vocal folds, and no effort is made to eject.
Raters are instructed that they should only score new material entering the airway on each subswallow. In the case of scores of 3, 5, 6, 7, or 8 (indicating airway invasion without ejection out of the laryngeal vestibule), the frame of bolus entry into the laryngeal vestibule is documented to enable subsequent evaluation of the timing of airway invasion relative to laryngeal vestibule closure (LVC).
b. Amount of Material Entering the Airway
The amount of material entering the airway is coded subjectively as 0 (none), 1 (trace), or 2 (more than trace).
c. Bolus-Level Summary Results for Penetration and Aspiration
Bolus-level results are derived based on the highest PAS score seen across the series of subswallows for that bolus. In addition, if a previous PAS event is noted to evolve across subsequent, higher order subswallows for the same bolus (i.e., worsen or recover to a higher position in the airway), this is noted in the rating comments.
7. Event Timing (ASPEKT Method Step 4)
In order to facilitate the calculation of timing measures, raters are asked to record the frame numbers at which a series of key events occurs. The list of events is chosen on a study-by-study basis, and there is room for additional events to be added to the master list in the future. For the current study, the following operational definitions were used to define the events of interest:
a. Bolus passing mandible (BPM): the first frame where the leading edge of the bolus touches or crosses the shadow of the ramus of mandible. In cases where the bolus was considered to have escaped prematurely from the mouth into the pharynx, the first frame showing bolus material at or below the ramus of mandible was counted as the BPM frame. When a double mandible shadow was seen on the lateral view image, the lower edge of the more superior ramus was used as the landmark.
b.
Onset of the hyoid burst (HYB): the first anterior–superior “jump” of the hyoid that is associated with a swallow. This event has previously been referred to using the terminology
onset of maximal hyoid excursion or
onset of the pharyngeal response (
Robbins, Hamilton, Lof, & Kempster, 1992).
c. LVC: the first frame showing contact between the arytenoid process and the inferior surface of the epiglottis. In cases where there is no contact, the frame of maximum approximation of the arytenoid process to the inferior surface of the epiglottis is used, and the term laryngeal vestibule approximation is used instead of LVC.
d.
UESO: the first frame where the leading edge of the bolus (or, in rare cases, air) passes through the upper esophageal sphincter (UES). The UES is a narrow segment or region that typically lies between C4 and C6; the narrowest opening seen between C4 and C6 during a swallow is marked as the location of the sphincter (
Leonard, Kendall, & McKenzie, 2004). In addition, recognizing that the UES moves superiorly during the swallow (
Kahrilas, Logemann, Lin, & Ergun, 1992), the narrowest portion may be located above C4. The superior boundary of the tracheal air column can be used as a guide to decide where the location of the UES is during pharyngeal shortening. The specific location chosen for measurement is judged subjectively by the rater, and subsequent interrater agreement comparisons serve to flag cases where the chosen location may differ across raters and require review.
e. Maximum UES distension (UESMax): the frame where the UESO has the widest width (i.e., diameter), judged perpendicular to the cervical spine on a lateral-view fluoroscopy image.
f. Maximum pharyngeal constriction (MPC): the earliest frame showing maximum obliteration of the space in the pharynx. This event must occur before the upper pharynx begins to relax and before the tracheal air column begins to descend.
g. UES closure (UESC): the first frame where the UES achieves closure behind the bolus tail. This does not require closure of the entire UES segment, simply closure at a single point along the segment.
h. LVCOff: the first frame where there is visible opening (white space) of the laryngeal vestibule. This requires some separation of the tissues or of the arytenoids from the inferior surface of the epiglottis, but complete opening is not required. The leaf of the epiglottis may still be in a downward position. This event cannot be identified in cases of incomplete LVC.
i. Swallow rest: the terminal event of each swallow, identified as the first frame showing the pyriform sinuses at their lowest position, relative to the spine, prior to any hyoid burst or laryngeal elevation for a subsequent subswallow. For the terminal subswallow, this event is further defined as occurring within 30 frames (approximately 1 s) of UESC, prior to any nonswallow events such as coughing, talking, or UES reopening.
8. Judging the Completeness of LVC (ASPEKT Method Step 4c)
The frame of LVC (or laryngeal vestibule approximation) is reviewed, and the rater judges whether closure is complete (or incomplete). A rating of “complete” requires a seal between the epiglottis and the arytenoids, leaving no visible airspace.
9. Ordinal Ratings of Bolus Location on Key Event Frames (ASPEKT Method Step 5)
The location of the leading edge of the bolus is recorded on key frames during the pharyngeal swallow. For this study, bolus location was tracked on the frames of HYB and LVC.
a. Bolus location at swallow onset (i.e., on the frame of HYB)
On the frame of HYB, the scoring convention recommended in the MBSImp was used (
Martin-Harris et al., 2008): A score of 0 was assigned when the leading edge of the bolus head was in the region of the posterior angle of the ramus and back of the tongue, a score of 1 was assigned when the bolus head had reached the pit of the valleculae, a score of 2 was given when the bolus head was at the posterior laryngeal surface of the epiglottis, a score of 3 was given when the bolus head was in the pyriform sinus (i.e., inferior to the arytenoids), and a score of 4 was given when there was no appreciable swallow initiation at any bolus location.
b. Bolus location at LVC
For scores of bolus location at the time of LVC, the scale was extended as follows: 0 = bolus head in the oral cavity or at the posterior angle of ramus, 1 = bolus head at the vallecular pit, 2 = bolus head at the posterior laryngeal surface of epiglottis, 3 = bolus head at the level of the pyriform sinuses, 4 = bolus head in the UES, and 5 = no appreciable swallow initiation.
10. Pixel-Based Tracing of Structural Movement of Area (ASPEKT Method Steps 6–8)
ImageJ allows for the measurement of structural position, movement, or area on radiographic images. In the ASPEKT method, pixel-based tracing is performed to capture spatial information regarding hyoid position and movement, pharyngeal area, and UESO. All of these measures are made in a coordinate system for which the C2–C4 cervical spine serves as the
y-axis, with the origin located at the anterior inferior corner of C4, and for which the
x-axis is derived perpendicular to the
y-axis (see
Figure A2).
In addition, all pixel-based measures in the ASPEKT are derived in anatomically scaled units (
Molfenter & Steele, 2014;
Perlman, Vandaele, & Otterbacher, 1995). Derivation in millimeters would be possible, if a radio-opaque scalar reference of known size was placed on the patient in the field of view. A coin has been used by previous researchers for this purpose (
Logemann, 1986). However, studies have shown that height differences between men and women explain a significant amount of sex-based variation in hyoid movement measured in millimeters, suggesting that structural movements in swallowing may differ based on the length and size of the pharynx (
Molfenter & Steele, 2014). For this reason, anatomical scalars are used in the ASPEKT method. The preferred scalar is a straight line running from the anterior–inferior corner of the C2 vertebra down to the anterior–inferior corner of the C4 vertebra. In cases where the cervical spine is altered through degeneration, abnormal curvature, or the presence of hardware, measurement of the anterior–posterior length of the vocal folds is recommended as an alternative. This alternative measure, sometimes called
tracheal width, has been used in imaging studies of primate vocal tract evolution and speech production as a reference scalar (
Badin, Bailly, & Revert, 2002;
Boe et al., 2017;
Captier et al., 2013;
Serrurier & Badin, 2008) and has been shown to have a similar relationship to participant height as cervical spine scalars (
Molfenter & Steele, 2014). Explorations across a large number of videofluoroscopies in our lab suggest that the vocal fold length measure corresponds closely to half of the C2–C4 cervical spine scalar.
a.Measures of Hyoid Position (ASPEKT Method Step 6)
Hyoid position is measured as distance from the anterior–inferior corner of C4. Measurements are taken in the anterior (
X) and superior (
Y) planes. The
XY hypotenuse position is then derived using the Pythagorean theorem. This procedure can be performed on any frame of interest. For the current study, hyoid position was tracked frame by frame, beginning five frames prior to the HYB frame until approximately five frames after the beginning of hyoid descent from peak position. A MATLAB algorithm was then used to search through the series of hyoid position measures to confirm the frames of peak position in each plane (
X,
Y, and
XY). In the case of a plateau in hyoid movement at its peak position, the first frame at peak position was used. In the example shown in
Figure A3, the yellow triangle shows the three planes in which hyoid peak position is measured (as distance from the anterior–inferior corner of C4), relative to the green dashed line, which represents the C2–C4 reference scalar (i.e., one cervical unit). The position of the hyoid in this image is 1.23 cervical units (i.e., 123% of the scalar) anterior to the C4 origin along the
x-axis, 0.88 cervical units superior to the C4 origin along the
y-axis, and 1.55 cervical units away from the C4 origin along the
XY hypotenuse. Additional instructions regarding the measurement of hyoid movement using ImageJ, together with a spreadsheet for calculating anatomically scaled measures, can be found at
http://steeleswallowinglab.ca/srrl/best-practice/hyoid-movement/.
b.Measures of Hyoid Displacement and Kinematics (ASPEKT Method Step 6c)
For the current study, the hyoid parameter of interest was peak hyoid position, as described above. Measurement of hyoid displacement (i.e., change in position between two frames of interest) is also possible. In addition, the hyoid position histories that are captured through frame-by-frame tracing can be used to identify durations of hyoid movement between key frames and to calculate velocities and peak velocities (
Nagy et al., 2013;
Nagy, Molfenter, Peladeau-Pigeon, Stokely, & Steele, 2014).
c.Measures of UESO (ASPEKT Method Step 7a)
The ImageJ line tool is used to trace the diameter of UESO on the UESMax frame. In the example shown in
Figure A4, the yellow line represents the UES diameter measure, and the green dashed line represents the C2–C4 reference scalar (i.e., one cervical unit). The UES diameter line measures 0.32 cervical units in length or 32% of the reference scalar.
d.Measures of Pharyngeal Area (ASPEKT Method Steps 7b and 7c)
For the current study, it was of interest to measure the area of the pharynx at rest and on the frame of MPC. Area measures made using the ImageJ tool are expressed relative to the squared C2–C4 reference area (
Stokely, Peladeau-Pigeon, Leigh, Molfenter, & Steele, 2015). The boundaries of the pharynx are defined to include all space above the UES, below the top of C2, posterior to the arytenoids, base of the tongue, and pharyngeal surface of the epiglottis and anterior to the posterior pharyngeal wall. This is illustrated in
Figure A5, in which the red dashed square shows the squared C2–C4 reference scalar and the yellow outline shows the boundaries of the area of the pharynx. In this example, the pharyngeal area measure is 103% of the size of the squared reference area. It should be noted that the boundaries of the pharynx in the ASPEKT measurement approach differ from the boundaries used by
Leonard, Rees, Belafsky, and Allen (2011) by excluding the nasopharynx and the laryngeal vestibule.
Our convention is to use the anatomically normalized pharyngeal area measure on the MPC frame to represent the degree of pharyngeal constriction seen during a swallow. Calculation of a pharyngeal constriction ratio, by dividing the pharyngeal area at constriction by the pharyngeal area at rest, is also possible (
Leonard et al., 2011). It should be noted that the frame selected for measurement of the pharyngeal area at rest in the ASPEKT method is the swallow rest frame at the end of the swallow, whereas the frame used by Leonard et al. is a static frame at the beginning of the videofluoroscopy procedure, in which a 1-ml thin liquid bolus is held in the mouth. As shown in
Figure A6, the traceable unobliterated area of the pharynx, shown by the yellow outline, has an area that measures 4% of the red dashed square reference area and would yield a pharyngeal constriction ratio measure of 3.8% compared to the pharyngeal area measured in Figure A5.
11. Measures of Residue Severity (ASPEKT Method Step 8)
Postswallow residue can be measured in a variety of ways. Ordinal ratings, using 3- or 4-point scales, are commonly used in the literature (
Eisenhuber et al., 2002;
Robbins et al., 2007) but show limited sensitivity to change in treatment outcome studies (
Logemann et al., 2009;
Robbins et al., 2007;
Steele et al., 2013). Other authors have proposed measurements of residue for which the percentage of the bolus remaining in the pharynx is judged perceptually (
Rademaker, Pauloski, Logemann, & Shanahan, 1994) or measured relative to a frame showing the complete bolus in the pharynx (
Leonard, 2017). For the current study, and as part of the ASPEKT method, residue severity was measured on the “swallow rest” frame in a manner similar to the pixel-based measures of area described above. Residue can be measured in various locations. For the current study, residue was measured separately in the valleculae, in the pyriform sinuses, and elsewhere within the pharynx. Pixel-based tracing allows for the calculation of lateral-view residue area relative to the lateral-view area of the spatial housing, expressed as percent full. By convention, the upper boundaries for tracing the spatial housing of the valleculae and the pyriform sinuses are defined as the apex of the epiglottic leaf and the apex of the arytenoid process, respectively, with these lines drawn perpendicular to the spine. Alternatively, residue area can be expressed relative to the squared C2–C4 reference scalar. Finally, these different approaches can be combined to calculate the Normalized Residue Ratio Scale (NRRS) measure (
Pearson, Molfenter, Smith, & Steele, 2012). Additional instructions regarding the NRRS and residue measurement using ImageJ, together with a spreadsheet for calculating anatomically scaled measures, can be found at
http://steeleswallowinglab.ca/srrl/best-practice/nrrs-residue/.
The NRRS measure does not provide a conceptually interpretable unit and does not easily allow for the summation of total pharyngeal residue, given that the area of the spatial housing for residue that falls outside the valleculae and pyriform sinuses is not easily defined. Therefore, in order to appreciate the severity of total pharyngeal residue, we recommend expressing the sum of all residue areas relative to the squared C2–C4 reference scalar.
Residue measurement is illustrated in
Figures A7,
A8, and
A9. In Figure A7, the frame of swallow rest is shown, with residue seen in both the valleculae and pyriform sinuses. In Figure A8, the ImageJ tracings for residue measurement in the valleculae and pyriform sinuses are shown. The residue in each space is outlined in yellow, and the spatial housing areas are shown in white. The squared C2–C4 reference scalar is shown by the red dashed lines. In this example, the ratio of the vallecular residue to the vallecular housing area provides a measure of 64% full, which translates to 19% of the squared C2–C4 reference scalar and an NRRSv score of 0.68. The ratio of the pyriform sinus residue to its housing area yields a measure of 74% full, translating to 25% of the squared C2–C4 reference scalar and an NRRSp score of 1.04. In addition to residue in these spaces, Figure A7 shows some residue in the pharynx, between the valleculae and pyriform sinuses. This “extra” residue is traced in Figure A9 and occupies 5% of the C2–C4
2 reference scalar. The total pharyngeal residue area can be expressed as the sum of the three measured areas (vallecular, pyriform, and extra residue), translating to 49% of the C2–C4
2 reference scalar.
12. Interrater Agreement
Many studies in the dysphagia literature have reported poor interrater agreement for videofluoroscopy rating. In order for ASPEKT method ratings to be useful for research or in the clinical identification of swallowing pathophysiology, it is critical that good interrater and intrarater agreement can be established. Checking agreement is a key part of the ASPEKT method and occurs at two key time points in the process. First, agreement for ratings of the number of swallows, penetration–aspiration, and event timing is inspected at the end of the event identification phase (Step 4). This ensures that the frames that are carried forward for ordinal or pixel-based ratings are confirmed and that these subsequent measurements are made on identical frames by different raters. After a set of initial ratings has been made independently by two raters, an Excel macro program is run to inspect them for agreement and identify cases that require discrepancy resolution. Strict criteria are used to handle discrepancies between raters. Any difference (of any magnitude) in ratings of the number of swallows per bolus, PAS scores, LVC (complete/incomplete), or bolus location at swallow onset is sent to a consensus meeting for resolution. For timing measures, any difference of more than five frames between raters regarding the frame at which key events occurred is sent for resolution. Consensus meetings are attended by a minimum of three trained raters and involve review, repeat measurement, and debate regarding the discrepant ratings until consensus is achieved. In cases where ratings are in close enough agreement to not require resolution, a priori rules guide selection of the frame of record. For event timing, where discrepancies are five frames or less, the earlier frame is used as the frame of record for the timing of penetration–aspiration events, BPM, HYB, LVC, and UESO, whereas the later frame is used as the frame of record for MPC, UESMax, UESC, LVCoff, and swallow rest. Once discrepancies are resolved, the sequence of events can be derived and timing intervals between events are calculated.
In the current study, thresholds for differences in pixel-based measurement that required resolution were set on a parameter-by-parameter basis, using the 95% confidence interval upper boundary of rater differences from a previous data set. The thresholds were as follows: normalized UES diameter differences > 0.08, normalized measures of pharyngeal area at rest > 0.2, normalized measures of MPC > 0.16, and NRRS measures > 0.1 for the valleculae and > 0.22 for the pyriform sinuses. Each of these measures is further explained below. Where rater differences did not require resolution, the smaller of the two rating values was taken as the rating of record. In the absence of a historical data set for a rating team, upon which to base the decision about the need for discrepancy resolution of pixel-based measures, we recommend reviewing cases for which the ratio of the larger value to the smaller value exceeds a value of 1.6.
For the current study, interrater agreement for the initial ratings (i.e., before discrepancy resolution) was strong for all types of measures:
a. For the number of swallows per bolus, absolute agreement was achieved in 97.9% of cases, with a Fleiss κ of .938 (95% CI [.893, .983]).
b. For PAS scores, absolute agreement was found in 94% of cases, with 4.5% of cases differing by 1 point on the 8-point Penetration–Aspiration Scale and 1% of cases differing by more than 1 point. The Fleiss κ score was .338, suggesting fair agreement prior to the resolution of discrepancies by consensus (95% CI [.296, .379]).
c. For the identification of frame numbers corresponding to key events in the swallowing sequence, a mean absolute difference of 1.6 frames was found (95% CI [1.5, 1.7]), with an intraclass coefficient of 1 (95% CI [1, 1]).
d. For the binary categorical rating of LVC being complete or incomplete, absolute agreement was seen for 96.6% of the boluses in the data set, with a Fleiss κ of .926 (95% CI [.882, .97]).
e. For the ordinal rating measures of bolus location, absolute agreement was seen in 72% of cases, with 20% of cases differing by one level and 8% of cases differing by more than one level and with a moderate Fleiss κ of .563 (95% CI [.544, .582]).
f. For pixel-based measures of distance or length, a mean difference of 4.3 pixels (95% CI [4.08, 4.57]) and an intraclass coefficient of .972 (95% CI [.97, .973]).
g. For pixel-based measures of area, a mean difference of 181.5 pixels2 (95% CI [174, 189]) and an intraclass coefficient of .965 (95% CI [.964, .967]).
Appendix References
Badin, P., Bailly, G., & Revert, L. (2002). Three-dimensional linear articulatory modeling of tongue, lips and face, based on MRI and video images. Journal of Phonetics, 30, 533–553.
Boe, L. J., Berthommier, F., Legou, T., Captier, G., Kemp, C., Sawallis, T. R., … Fagot, J. (2017). Evidence of a vocalic proto-system in the baboon (papio papio) suggests pre-hominin speech precursors. PLOS ONE, 12(1), e0169321.
Captier, G., Boe, L. J., Badin, P., Guihard-Costa, A. M., Canovas, F., & Larroche, J. C. (2013). Geometrical growth models of the fetal forebrain, cerebellum, brainstem and change of the cranial base angles during fetal period. Morphologie, 97(317), 38–47.
Eisenhuber, E., Schima, W., Schober, E., Pokieser, P., Stadler, A., Scharitzer, M., … Oschatz, E. (2002). Videofluoroscopic assessment of patients with dysphagia: Pharyngeal retention is a predictive factor for aspiration. American Journal of Roentgenology, 178(2), 393–398.
Kahrilas, P. J., Logemann, J. A., Lin, S., & Ergun, G. A. (1992). Pharyngeal clearance during swallowing: A combined manometric and videofluoroscopic study. Gastroenterology, 103(1), 128–136.
Leonard, R., Kendall, K., & McKenzie, S. (2004). UES opening and cricopharyngeal bar in nondysphagic elderly and nonelderly adults. Dysphagia, 19(3), 182–191.
Leonard, R., Rees, C. J., Belafsky, P., & Allen, J. (2011). Fluoroscopic surrogate for pharyngeal strength: The pharyngeal constriction ratio (PCR). Dysphagia, 26(1), 13–17.
Leonard, R. J. (2017). Two methods for quantifying pharyngeal residue on fluoroscopic swallow studies: Reliability assessment. Annals of Otolaryngology and Rhinology, 4(3), 1168.
Logemann, J. A. (1986). Manual for the videofluorographic study of swallowing. Austin, TX: Pro-Ed.
Logemann, J. A., Rademaker, A., Pauloski, B. R., Kelly, A., Stangl-Mcbreen, C., Antinoja, J., … Shaker, R. (2009). A randomized study comparing the Shaker exercise with traditional therapy: A preliminary study. Dysphagia, 24(4), 403–411.
Martin-Harris, B., Brodsky, M. B., Michel, Y., Castell, D. O., Schleicher, M., Sandidge, J., … Blair, J. (2008). MBS measurement tool for swallow impairment—MBSImp: Establishing a standard. Dysphagia, 23(4), 392–405.
Molfenter, S. M., & Steele, C. M. (2014). Use of an anatomical scalar to control for sex-based size differences in measures of hyoid excursion during swallowing. Journal of Speech, Language, and Hearing Research, 57(3), 768–778.
Nagy, A., Leigh, C., Hori, S. F., Molfenter, S. M., Shariff, T., & Steele, C. M. (2013). Timing differences between cued and noncued swallows in healthy young adults. Dysphagia, 28, 428–434.
Nagy, A., Molfenter, S. M., Peladeau-Pigeon, M., Stokely, S., & Steele, C. M. (2014). The effect of bolus volume on hyoid kinematics in healthy swallowing. Biomed Research International, 2014, Article ID 738971.
Pearson, W. G., Jr., Molfenter, S. M., Smith, Z. M., & Steele, C. M. (2012). Image-based measurement of post-swallow residue: The Normalized Residue Ratio Scale. Dysphagia, 28, 167–177.
Perlman, A. L., Vandaele, D. J., & Otterbacher, M. S. (1995). Quantitative assessment of hyoid bone displacement from video images during swallowing. Journal of Speech and Hearing Research, 38(3), 579–585.
Rademaker, A. W., Pauloski, B. R., Logemann, J. A., & Shanahan, T. K. (1994). Oropharyngeal swallow efficiency as a representative measure of swallowing function. Journal of Speech and Hearing Research, 37(2), 314–325.
Robbins, J., Hamilton, J. W., Lof, G. L., & Kempster, G. B. (1992). Oropharyngeal swallowing in normal adults of different ages. Gastroenterology, 103(3), 823–829.
Robbins, J., Kays, S. A., Gangnon, R. E., Hind, J. A., Hewitt, A. L., Gentry, L. R., & Taylor, A. J. (2007). The effects of lingual exercise in stroke patients with dysphagia. Archives of Physical Medicine and Rehabilitation, 88(2), 150–158.
Rosenbek, J. C., Robbins, J. A., Roecker, E. B., Coyle, J. L., & Wood, J. L. (1996). A penetration–aspiration scale. Dysphagia, 11(2), 93–98.
Serrurier, A., & Badin, P. (2008). A three-dimensional articulatory model of the velum and nasopharyngeal wall based on MRI and CT data. The Journal of the Acoustical Society of America, 123(4), 2335–2355.
Steele, C. M., Bailey, G. L., Polacco, R. E., Hori, S. F., Molfenter, S. M., Oshalla, M., & Yeates, E. M. (2013). Outcomes of tongue-pressure strength and accuracy training for dysphagia following acquired brain injury. International Journal of Speech-Language Pathology, 15(5), 492–502.
Stokely, S. L., Peladeau-Pigeon, M., Leigh, C., Molfenter, S. M., & Steele, C. M. (2015). The relationship between pharyngeal constriction and post-swallow residue. Dysphagia, 30(3), 349–456.