Cognitive flexibility deficits are a hallmark feature of autism spectrum disorder (ASD), but few evidence-based behavioral interventions have successfully addressed this treatment target. Outcome measurement selection may help account for previous findings. The probabilistic reversal learning task (PRL) is a measure of cognitive flexibility previously validated for use in ASD, but its use as an outcome measure has not yet been assessed. The current study examined the feasibility, reproducibility, and sensitivity of PRL in a within-subjects trial of "Regulating Together," a group-based intervention targeting emotion regulation. We demonstrated the PRL is highly feasible, showed test-retest reproducibility, and is sensitive to detect change following the intervention. Our findings demonstrate the PRL task may be a useful outcome measure of cognitive flexibility in future intervention trials in ASD.