The "consensual assessment technique" (CAT) is a reliable and valid method to measure (product) creativity and often considered "the" gold standard of creativity assessment. The reliability measure traditionally applied in CAT studies--inter-rater reliability--cannot capture time-sampling error, which is a particular relevant source of error for specific applications of the CAT. Therefore, the present study intended to investigate the test--retest reliability of CAT ratings. We asked raters (N = 61) for their creativity assessment of the same set of 90 fashion outfits at an initial rating session and a follow-up session either 2 or 4 weeks later. We found that mean product ratings--the actual focus of interest in the CAT--were highly stable over time, as evidenced by consistency and agreement ICCs clearly exceeding levels of .90. However, individual raters (partially) lacked temporal stability, indicating a drift in rater tendencies over time. Our findings support the CAT's reputation as a highly reliable measurement method, but question the temporal rating stability of the CAT's actual "measurement instrument," namely individual judges.