Understanding the impact of undergraduate research experiences (UREs) and course-based undergraduate research experiences (CUREs) is crucial as universities debate the value of allocating scarce resources to these activities. We report on the Berkeley Undergraduate Research Evaluation Tools (BURET), designed to assess the learning outcomes of UREs and CUREs in chemistry and other sciences. To validate the tools, we administered BURET to 70 undergraduate students in the College of Chemistry and 19 students from other STEM fields, comparing the performance of students who had less than one year of undergraduate research to those with more than one year of research experience. Students wrote reflections and responded to interviews during poster presentations of their research project. BURET asks students to communicate the significance of their project, analyze their experimental design, interpret their data, and propose future research. Scoring rubrics reward students for integrating disciplinary evidence into their narratives. We found that the instruments yielded reliable scores, and the results clarified the impacts of undergraduate research, specifically characterizing the strengths and weaknesses of undergraduate researchers in chemistry at our institution. Students with at least a year of research experience were able to use disciplinary evidence more effectively than those with less than one year of experience. First-year students excelled at explaining the societal relevance of their work, but they incorporated only minimal discussion of prior research into their reflections and presentations. Students at all levels struggled to critique their own experimental design. These results have important implications for undergraduate learning, suggesting areas for faculty members, graduate student research mentors, and CURE or URE programs to improve undergraduate research experiences.