This paper aims to investigate the reasons some female mathematicians give to justify their choice to not work in academia after finishing their doctoral studies. Nine female mathematicians who finished a PhD in Sweden answered a written questionnaire. Through collective narrative analysis, two main tracks were identified. One narrative described the struggle with self-identity in a gendered structure which included implicit power, while the other was more positive about exposure to discrimination, and highlighted the desire to work with applied mathematics. Through deductive thematic analysis, the results show that the main obstacle raised was the difficulty of getting a job in academia after their doctoral studies, especially permanent positions, without support. Compared to previous research, the lack of family-oriented political policies was not considered a main problem. Instead, the reasons provided by the respondents are structural problems, such as access to post-doc positions, and the stress of having to get research grants, as well as cultural aspects within the structure, including implicit and explicit use of power.