Despite the importance of replication, it remains rare in the interactive learning research community. In this paper, we attempt to replicate recent quasi-experimental results suggesting that the ALEKS intelligent tutoring system is effective at improving student course outcomes in higher education (Mojarad et al., 2018). In this paper, we conduct a near replication, collecting a new data set of higher education students using ALEKS, at the same university as in that earlier paper. We investigate the robustness of the results found to the choice of quasi-experimental methodology. In the earlier work, the popular propensity score matching algorithm was used; a recent methodological paper challenges this method (King & Nielsen, 2019). We therefore investigate the impact of using another matching algorithm, inverse probability of treatment weighting (IPTW) instead of propensity score matching, and compare the results obtained by these two methods. We replicate the previous study: ALEKS is statistically significantly associated with better student course outcomes. The use of IPTW leads to the same qualitative result as PSM, but IPTW achieves superior matching, suggesting that this method should be preferred for future quasi-experiments within the interactive learning research community.