The association between physical fitness and age-related differences in cognition and brain structure has been studied fairly extensively during development and aging, yet comparatively less in young adulthood. The current study examined 1,195 young adults aged 22 to 36 (54% female; 67% Caucasian) to better understand associations between physical fitness--grip strength and submaximal cardiovascular endurance--and age-related differences in executive function (EF), memory, and average cortical thickness. EF, memory, and cortical thickness were negatively associated with age, and higher endurance was positively associated with EF and memory. Neither physical fitness measure associated with cortical thinning. To follow-up on these analyses, data from monozygotic (n = 149 pairs) and dizygotic (n = 93 pairs) twins were used to estimate the degree to which heritability versus environment might contribute to the observed associations between cognition and endurance. Environmental effects shared by monozygotic and dizygotic twins alike were estimated to account for roughly 50% of the correlation between endurance and cognition (EF and memory). Heritability and nonshared environmental effects were inconsistent across EF and memory. Overall, these findings suggest an association between cardiovascular endurance and age-related differences in cognition in young adulthood and that these associations may be independent of cortical thinning. Whereas there was consistent evidence for a moderate contribution of the shared environment, there was limited and inconsistent evidence for a role of heritability.