Recently there has been a surge of interest in technology-aided teaching strategies such as the flipped classroom. Given the growing interest in these techniques, it is important to critically evaluate their effectiveness and to begin to examine factors that might shape how effective the flipped classroom will be in a given educational setting. Although most existing research on the flipped classroom suggests an advantage of the flipped approach over a more traditional lecture approach, most of this research has been conducted in a single educational setting at a time and in ways that preclude definitive conclusions about the relative effectiveness of the flipped approach. We present the results of a study that addressed many of these methodological limitations and compared the effectiveness of the flipped approach to a traditional lecture approach across two semesters in courses from two different disciplines, finance and psychology. We found that the effectiveness of the flipped versus the traditional approach varied across the two courses. In particular, in the psychology courses the flipped approach resulted in superior performance on quizzes administered immediately after exposure to course material, but resulted in similar performance on exams administered some time after initial exposure. In contrast, in the finance courses the flipped approach resulted in similar performance on immediately administered quizzes, but superior performance on later administered exams. These results contribute to the burgeoning literature on the flipped classroom both by addressing methodological limitations found in previous work and by identifying some of the specific conditions under which the flipped approach may be a good pedagogical choice.