Does knowledge of other people's minds grow from concrete experience to abstract concepts? Cognitive scientists have hypothesized that infants' first-person experience, acting on their own goals, leads them to understand others' actions and goals. Indeed, classic developmental research suggests that before infants reach for objects, they do not see others' reaches as goal-directed. In five experiments (N = 117), we test an alternative hypothesis: Young infants view reaching as undertaken for a purpose but are open-minded about the specific goals that reaching actions are aimed to achieve. We first show that 3-month-old infants, who cannot reach for objects, lack the expectation that observed acts of reaching will be directed to objects rather than to places. Infants at the same age learned rapidly, however, that a specific agent's reaching action was directed either to an object or to a place, after seeing the agent reach for the same object regardless of where it was, or to the same place regardless of what was there. In a further experiment, 3-month-old infants did not demonstrate such inferences when they observed an actor engaging in passive movements. Thus, before infants have learned to reach and manipulate objects themselves, they infer that reaching actions are goal-directed, and they are open to learning that the goal of an action is either an object or a place.