We explore the effect of students' perceived task difficulty on the mathematics self-efficacy -- performance relationship. Specifically, we expand on previous reciprocal effects studies through including students' self-efficacy for different levels of task difficulty in an empirical investigation. We examined students' self-efficacy for easy, medium difficulty, and hard tasks and performance on a national mathematics test in a longitudinal study of 95 Norwegian students from grade 8 to grade 9. We found differential relationships between self-efficacy for different levels of task difficulty and national test performance. In support of the 'skill development' model, grade 8 national test performance predicted grade 9 self-efficacy for medium and hard, but not easy, tasks. While mastery experiences are likely to arise more easily on easier tasks, such experiences are likely to matter more on harder tasks. Our findings highlight the importance of supporting students' engagement with challenging tasks to strengthen both their performance and self-efficacy.