Previous research shows that sensory and motor systems interact during perception, but how these connections among systems are created during development is unknown. The current work exposes young children to novel "verbs" and objects through either (a) actively exploring the objects or (b) by seeing an experimenter interact with the objects. Results demonstrate that the motor system is recruited during auditory perception only after learning involved self-generated interactions with objects. Action observation itself led to above-baseline activation in one motor region during visual perception, but was still significantly less active than after self-generated action. Therefore, in the developing brain, associations are built upon real-world interactions of body and environment, leading to sensori-motor representations of both objects and words.