The COVID-19 pandemic has enforced higher education institutions to adopt emergency remote teaching (ERT) as the substitution for traditional face-to-face (F2F) classes. A lot of concerns have been raised among education institutions, faculty, and students regarding the effectiveness of this sudden shift to online learning. This study aims to statistically investigate the impacts of such a transition on the academic performance of undergraduate students enrolled in the Financial Engineering course. A novel rank percentage measure is proposed and employed to compare the academic performance of around 500 students who attended the course during the four semesters, including the transitional disrupted semester by the pandemic, two consecutive online semesters, and the traditional face-to-face classroom. Our analysis emphasizes the significance of the differences between specific subgroups of the students. In particular, academically average to good students with cumulative GPAs greater than 2.90 have been negatively impacted by the transition to online learning, whereas the results for students with cumulative GPAs less than 2.90 are not very conclusive. Realizing the effects of such closures on the academic performance of students is considered important, since the results might have some merits for other courses and instructors. The template model can be transferred to other courses, and employed by the university administrators, specifically for developing policies in emergency circumstances that are not limited to pandemics.