Research has accumulated on the effects of teachers on student achievement, especially in mathematics. However, empirical evidence regarding the impact of formal teacher competence indicators presents a mixed picture. This study explores the influence of educational level, subject -- and grade-specific specialization, teaching experience, and professional development as indicators of formal teacher competence on student mathematics achievement in grade 4 in Sweden. Utilizing data from TIMSS 2019, the study employs confirmatory factor analysis and multilevel structural equation modeling to also test a latent model of formal teacher competence. Results reveal a positive relationship between formal teacher competence and students' mathematics achievement, even after controlling for students' socio-economic status and immigration background. Additionally, selection effects were found, suggesting that students in classrooms with a more advantaged composition had more competent teachers. The study underscores the need for an equitable distribution of qualified teachers across schools, highlighting implications for teacher specialization and allocation.