This study explores the effect the U.S. National Science Foundation Research Experiences for Undergraduates (NSF REU) programs have on REU participants and other undergraduate student researchers and how their success was measured. The authors conducted interviews with 10 principal investigators (PIs) of REU programs and surveyed undergraduate researchers from NSF REU programs as well as non-NSF programs. Overall, the undergraduate researchers studied indicated that the skills with the greatest gain as a result of their program were their research skills, knowledge of graduate school, and presentation skills. PIs and undergraduate researchers agreed that presentations and posters are the most notable metrics of success in their programs. As a result of the findings, the authors present a definition of a successful undergraduate research program in the discussion.