Purpose: The Malaysian government has allocated a large budget for "Orang Asli" primary school education via the Department of "Orang Asli" Development (JAKOA) to help improve the "Orang Asli" pupils' academic performance and in, Mathematics as well. Teachers face challenges in ensuring that "Orang Asli" pupils become competent learners of mathematics. Hence, there is a need to explore the teachers' perspectives on the teaching and learning of Mathematics in the "Orang Asli" schools. This study examined the teachers' perspectives on the issues and challenges of the teaching and learning of mathematics for the "Orang Asli" primary school pupils in Malaysia. Methodology: In this qualitative study, two "Orang Asli" primary schools were randomly selected from eight primary schools within an "Orang Asli" Settlement in the District of Sungai Siput. The data were collected via focus group discussions and interviews, which were carried out during the enculturation visits and were fully transcribed and thematically analysed. The research participants were purposively selected and comprised the school's administrators and mathematics teachers. Findings: The results of the data analysis show that there are two main themes, namely classroom challenges and school challenges, in determining mathematics competence among "Orang Asli" pupils. Classroom challenges consists of five sub-themes namely coverage of Mathematics syllabus, Mathematics teaching and learning resources, pupil engagement, language barrier, and Mathematics learning culture. School challenges consists of two sub-themes namely school attendance and discipline. To enhance mathematics competence among "Orang Asli" pupils, it is deemed important to implement programmes and community engagement. Significance: This study contributes towards the knowledge of teaching and learning of Mathematics for "Orang Asli" pupils. Teachers are recommended to allow "Orang Asli" pupils to take home their mathematics textbooks so that they can complete their homework at home. Teachers are highly recommended to conduct mathematics teaching and learning activities in groups for meaningful and engaging lessons.