When deciding whether to explore, agents must consider both their need for information and its cost. Do children recognize that exploration reflects a trade-off between action costs and expected information gain, inferring epistemic states accordingly? In two experiments, 4- and 5-year-olds (N = 144; of diverse race and ethnicity) judge that an agent who refuses to obtain low-cost information must have already known it, and an agent who incurs a greater cost to gain information must have a greater epistemic desire. Two control studies suggest that these findings cannot be explained by low-level associations between competence and knowledge. Our results suggest that preschoolers' theory of mind includes expectations about how costs interact with epistemic desires and states to produce exploratory action.