We investigated the relations between self-reported math anxiety, task difficulty, and pupil dilation in adults and very young children during math tasks of varying difficulty levels. While task difficulty significantly influenced pupillary responses in both groups, the association between self-reported math anxiety and pupil dilation differed across age cohorts. The children exhibited resilience to the effects of math anxiety, hinting at additional influential factors such as formal math education experiences shaping their relations to mathematics and their impact on cognitive processes over time. Contrary to expectations, no significant association between self-reported math anxiety and pupil dilation during task anticipation was found in either group. In adults, math anxiety influenced pupil dilation exclusively during the initial phase of task processing indicating heightened cognitive load, but this influence diminished during sustained task processing. Theoretical implications emphasize the need for exploring individual differences, cognitive strategies, and the developmental trajectory of math anxiety in very young children.