Critical reflection allows students to synthesize their learning and deepen their understanding of an experience (Ash & Clayton, 2009). A recommended reflection method is for students to write essays about their experiences. However, on a large scale, such reflection essays become difficult to analyze in a meaningful way. At Roanoke College, we have developed a system to automate the qualitative coding process using a software analysis tool (i.e., NVivo). This inductive approach allows us to identify patterns in student learning that indicate effective and ineffective aspects of applied learning experiences. It provides a more nuanced and rich approach to the analysis of student learning outcomes in order to ensure the quality of our programs through continuous improvement.