It has been 70 years since Donald Hebb published his formalized theory of synaptic adaptation during learning. Hebb's seminal work foreshadowed some of the great neuroscientific discoveries of the following decades, including the discovery of long-term potentiation and other lasting forms of synaptic plasticity, and more recently the residence of memories in synaptically connected neuronal assemblies. Our understanding of the processes underlying learning and memory has been dominated by the view that synapses are the principal site of information storage in the brain. This view has received substantial support from research in several model systems, with the vast majority of studies on the topic corroborating a role for synapses in memory storage. Yet, despite the neuroscience community's best efforts, we are still without conclusive proof that memories reside at synapses. Furthermore, an increasing number of non-synaptic mechanisms have emerged that are also capable of acting as memory substrates. In this review, we address the key findings from the synaptic plasticity literature that make these phenomena such attractive memory mechanisms. We then turn our attention to evidence that questions the reliance of memory exclusively on changes at the synapse and attempt to integrate these opposing views.