Perception is driven not only by current stimulation but also by previous sensory experience, which may serve as a perceptual prior for stimulus processing. A possible mechanism underlying this phenomenon is formalized in the internal reference model, which assumes that humans rely on an internal reference that updates continuously by integrating past and present stimulus representations. As a direct consequence of this process, discrimination sensitivity is higher when a constant standard precedes rather than follows a variable comparison in two-alternative forced-choice tasks. The present study exploited this Type B effect in order to examine whether an internal reference can be formed across stimuli varying within and across modalities. In a series of four experiments, task-irrelevant and/or task-relevant features either remained constant or varied from trial to trial. In duration discrimination, the Type B effect as a proxy of perceptual prior formation was not affected by variation of pitch or modality. However, in intensity discrimination, it was considerably reduced when stimulus modality and thus also the task-relevant feature (i.e., brightness and loudness) alternated rather than remained constant. These results indicate that temporal information can be isolated from stimulus-specific features and integrated across subsequent trials into a perceptual prior for stimulus judgment, whereas intensity information from different modalities seems to hamper this process. In general, our findings contribute to a growing body of research aimed at understanding the formation and content of perceptual priors.