The relative impact of admissions factors and curricular measures on the first medical licensing exam (United States Medical Licensing Exam [USMLE] Step 1) scores is examined. The inclusion of first-year and second-year curricular measures nearly doubled the variance explained in Step 1 scores from the amount explained by the combination of preadmission demographic characteristics and admissions factors. In addition, the relationship between the Medical College Admissions Test (MCAT) and Step 1 scores becomes counterintuitive in models that include curricular measures, where students with the lowest combined admissions metrics (MCAT, grade-point average) score "higher", on average, than those with some of the highest admissions metrics. Overreliance on traditional metrics in admissions decisions can exclude students from medical school who have the ability to succeed.