One of the core tenets of cognitive metaphor theory is the claim that metaphors ground abstract knowledge in concrete, first-hand experience. In this paper, I argue that this grounding hypothesis contains some problematic conceptual ambiguities and, under many reasonable interpretations, empirical difficulties. I present evidence that there are foundational obstacles to defining a coherent and cognitively valid concept of "metaphor" and "concrete meaning," and some general problems with singling out certain domains of experience as more immediate than others. I conclude from these considerations that whatever the facts are about the comprehension of individual metaphors, the available evidence is incompatible with the notion of an underlying conceptual structure organized according to the immediacy of experience.