Allyship is increasingly viewed as a crucial practice to address social and economic inequities faced by marginalized groups. However, little research explores how dominant group members can behave as allies and what marginalized group individuals consider as valuable. Research shows that women faculty in male-dominated academic disciplines regularly experience gendered microaggressions and sexism. In this study, using qualitative narratives from 202 women and men faculty in male-dominated academic departments, we explore women's experiences as recipients of allyship and men's experiences engaging in allyship. Further, we compare narratives about and from men considered "good colleagues" versus "exceptional allies." The findings reveal that good colleagues engage in an array of interpersonally supportive allyship behaviors (e.g., active listening). In contrast, exceptional allies go beyond them by undertaking consequential and conspicuous actions on behalf of women (e.g., impression promotion), a set of behaviors we term as visible advocacy. By exploring the range of allyship behaviors engaged in by men and viewed by women as valuable, we highlight implications for literature on gender equity and allyship. This study carries practical implications for men who desire to become better allies to women.