A handful of models and measures currently exist for the purpose of assessing student capacity for leading social change. Ample research suggests that students of different racial and gender identity groups exhibit various understandings, motivations, and behaviors related to social engagement and transformation, yet few studies take into consideration the potential for diverse students to interpret social-change-related survey scale items differently. Using a critical quantitative approach, this study compares factor loading patterns of the Social Issues Advocacy Scale (SIAS; Nilsson, et al., 2011) across eight race/gender subgroups to test the extent to which the factor structure remains invariant. Findings suggest that intersections of race and gender do influence how scale items cluster together. This study lends support for critical quantitative research designs that examine social phenomena using a specific-group approach, and calls for scholars to consider the cultural validity of scales used to measure capacity for social justice leadership.