This project evaluated the effectiveness of a course design within an upper-level biology course that incorporated what prior scholarship of teaching and learning (SoTL) research has suggested to be best practices for developing critical thinking skills while also managing the grading load on the instructor. These efforts centered on the development of a clearly articulated subset of skills identified by the Critical Thinking Assessment Test (CAT) and also incorporated learning experiences designed to instill what we refer to as a "habit of critical investigation." In this study, we tested the hypothesis that a single semester of an aligned course utilizing active learning and multiple opportunities for practice and feedback would: (a) increase the extent to which students agreed with the importance of questioning the credibility of claims across the semester, (b) increase the frequency at which students reported personally questioning the credibility of claims across the semester, (c) increase the number of students reporting investigation techniques consistent with critical investigation across the semester and (d) result in significantly greater student performance on the CAT questions that assessed the sub-skills practiced in the course when compared to the performance of a representative group of senior students at our institution. We observed substantial and statistically significant gains in both the frequency at which students reported questioning claims and the degree to which their reported investigative actions were consistent with critical investigation. Furthermore, on the critical thinking sub-skills most aligned with what was practiced in the course, the experimental group significantly outperformed the comparison group.