Although the use of traditional just-in-time teaching techniques has long been viewed positively by students and instructors in undergraduate calculus courses, past studies in this area have not addressed gains in student achievement with respect to specific calculus topics. This paper investigates the latter by administering modified just-in-time review modules over college algebra topics in a first-semester calculus course at Kansas State University during Spring 2012. Analysis of student performance revealed significant gains in learning on the topics of limits and the chain rule for derivatives. Issues pertaining to the timing of the reviews, along with descriptions of the review assignments, analysis of student performance, and implications for classroom instruction are discussed.