The processing of ordinally organized information is a characteristic of both serial-order working memory and numerical cognition. Serial positions of items presented within a list follow an ordinal organization when stored in working memory, whereas numbers are based on an ordinal structure stored in long-term memory. We tested the hypothesis that long-term numerical ordinal representations support the coding of temporary serial position information in working memory. In Experiment 1, learned word-number associations appeared to have a negative instead of a positive impact on immediate serial recall performance relative to control conditions. Experiments 2 showed that this effect was due to a stronger opportunity for learning associations of words to serial positions in the control lists as compared to the experimental lists. Experiment 3 showed that when controlling for these positional learning effects, there was no reliable effect of learned word-number associations on immediate serial recall performance. This study indicates that numerical codes do not play a major role in coding serial position information in working tasks. At the same time, the robust item-position learning effects demonstrate a contribution of long-term item-position associations to immediate memory for order.