Purpose: Information on reliability of outcome measures used to assess the effectiveness of interventions in dysphagia rehabilitation is lacking, particularly when used by different research groups. Here, we report on reliability of the penetration- aspiration scale (PAS) and temporal and clearance measures, determined using videofluoroscopy. Method: Secondary analysis used videofluoroscopies from the Swallowing Treatment using Electrical Pharyngeal Stimulation trial in subacute stroke. PAS scores (719 scores from 18 participants) were evaluated and compared to the original PAS scores from the trial. Five conditions were assessed, including reliability for every swallow and overall mean of the worst PAS score. Operational rules for assessing temporal and clearance measures were also developed using the same data, and reliability of these rules was assessed. Reliability of component-level and derivative-level scores was assessed using the intraclass correlation coefficient (ICC) and weighted kappa. Results: Image quality was variable. Interrater reliability for the overall mean of the worst PAS score was excellent (ICC = 0.914, 95% confidence interval [CI] [0.853, 0.951]) but moderate for every swallow in the bolus (ICC = 0.743, 95% CI [0.708, 0.775]). Intrarater reliability for PAS was excellent (all conditions). Excellent reliability (both inter- and intrarater > 0.90) was seen for temporal measures of stage transition duration (ICC = 0.998, 95% CI [0.993, 0.999] and ICC = 0.995, 95% CI [0.987, 0.998], respectively) as well as initiation of laryngeal closure and pharyngeal transit time and all individual swallow events. Strong scores were obtained for some clearance measures; others were moderate or weak. Conclusions: Interrater reliability for PAS is acceptable but depends on how the PAS scores are handled in the analysis. Interrater reliability for most temporal measures was high, although some measures required additional training. No clearance measures had excellent reliability.