In this paper, we explore the challenges experienced by a group of Primary 5 to 6 (age 12-14) students as they engaged in a series of problem-solving tasks through block-based programming. The challenges were analysed according to a taxonomy focusing on the presence of computational thinking (CT) elements in mathematics contexts: preparing problems, programming, create computational abstractions, as well as troubleshooting and debugging. Our results suggested that the challenges experienced by students were compounded by both having to learn the CT-based environment as well as to apply mathematical concepts and problem solving in that environment. Possible explanations for the observed challenges stemming from differences between CT and mathematical thinking are discussed in detail, along with suggestions towards improving the effectiveness of integrating CT into mathematics learning. This study provides evidence-based directions towards enriching mathematics education with computation.