Since 2000, the PISA test implemented by OECD has become the prime benchmark for international comparisons in education. The 2015 PISA edition introduced methodological changes that altered the nature of its results. PISA made no longer valid non-reached items of the final part of the test, assuming that those unanswered questions were more a problem of motivation than skills. Latin American countries were the only ones that had a large proportion of non-reached items in the first PISA editions. The methodological change had a large impact in these countries and was miscommunicated by the OECD reports. This makes the first two PISA editions (2000 and 2003) no longer comparable with the latest ones (2006, 2009, 2012 and 2015). We analyze hypotheses for the Latin American exception in PISA, suggesting that the introduction of the new Computer-Based Assessment of PISA in 2015, combined with an increasing culture of evaluation and political influences damaged intertemporal comparisons of PISA. Although PISA is a low-stakes assessment for students, it has become high-stakes for governments. This triggered mechanisms to increase test-taking motivation, and students tended to answer more questions of the test, but at the same time, these changes made intertemporal comparisons flawed.