Simulations are increasingly recognized for promoting active learning and student engagement and fostering career skills. Yet whether and how simulations facilitate learning, engagement, or both are still subjects of considerable debate. Many existing studies tend to be observational in design, constrained to a single semester, or reliant on student self-assessment to measure efficacy. We use an experimental approach to demonstrate the effects of classroom simulations on both self-assessment and objective academic performance at a large public university. Over the course of three years, we "treated" some sections of an Introduction to International Relations course (N = 244) with simulations while other "control" sections participated in debates and discussions. We compare self-assessed engagement and motivation from surveys, as well as performance on quizzes and exams in the same course (same instructor, materials, and textbook). Our results demonstrate that students "treated" with the simulations show sustained improvements in academic performance in both iterative (weekly quizzes) and long-term (final exam) contexts, controlling for the mode of delivery. These improvements are also found to be robust for both high and low-performing students in the class. However, we were surprised to find a disconnect between student perceptions of engagement and actual course performance: Self-assessed student engagement and motivation were poor predictors of subsequent academic performance.