The article explores 5th and 6th grade pupils' reflections on why pupils may refrain from intervening in bullying, despite understanding that bullying is wrong. The findings are based on focus group interviews conducted with 74 Swedish school pupils, who were asked for their perspectives on the various participant roles depicted in a bullying vignette. The findings were analysed using methods from constructivist grounded theory and through the theoretical lens of Goffman's concept of social stigma. The interviewees emphasised the implications of being positioned as the 'victim', including being socially stigmatised, isolated, denigrated and further bullied, and suggested that the fear of being 'singled out' would be a main concern for pupils, and hence the driving force behind why they may refrain from intervening in defence of a victimised peer. The study thus highlights the associated processes of social stigmatisation and the non-intervention of pupils in school bullying situations.