This quasi-experimental study investigated the role of prior psychology knowledge and in-class retrieval activity in the testing effect. Undergraduate introductory psychology students (N = 53) from two classes at a small liberal arts college practiced retrieving information in class with multiple-choice quizzing and concept mapping. Prior psychology knowledge was measured using a 25-item multiple-choice pretest. Both students with high and low prior psychology knowledge had higher scores on examination material that was practiced in class with retrieval-based concept mapping compared to traditional multiple-choice quizzes and to no organized in-class retrieval activity at all. Only students with high prior psychology knowledge had higher scores on quizzed material compared to no organized in-class retrieval practice, and these scores were lower than those on material that was practiced with in-class concept mapping. In comparison to administering multiple-choice quiz questions, a more useful in-class activity might be to have students, especially those with less prior psychology knowledge, practice retrieving material through free recall and connection building activities such as a concept map.