The South African Language in Education Policy (LiEP) advocates using African languages in the Foundation Phase (FP) which is the first phase of formal schooling in South Africa. This paper seeks to identify the challenges in implementing the language provisions prescribed in the LiEP in South Africa and within the region. Using the challenge-based learning (CBL) approach, this paper provides a glimpse into the challenges that 50 FP teachers based in six rural schools in South Africa face when using African languages as LoLT when teaching mathematics. Data collected through questionnaires, focus-group interviews, and observations in mathematics classrooms were used to illuminate these challenges. The results indicate that teachers experienced three main challenges: multilingual learners and learners who were taught in English in pre-school, English scripted lesson plans, and teacher training that was only provided in English. Importantly this paper identified a language disjuncture between policy and practice where the policy requires teachers to teach in African languages while the teaching material is written in English.