Using results from the Collegiate Learning Assessment (CLA) administered at Central Connecticut State University, a public Carnegie master's-larger programs university in the Northeast, this study demonstrates time on spent on the test, student motivation, and to a lesser extent the local institutional administration procedures represent problematic intervening variables in the measurement of student learning. Findings from successive administrations of the instrument reveal wide year-to-year variations in student performance related to time on test and motivation. Significant additional study of these factors should likely be prioritized ahead of adoption of accountability practices that rely upon low-stakes testing to measure student learning and demonstrate institutional effectiveness. Survey Results with Time on Test and Test Scores (F '08, Sp. '09 & F '09) are appended. (Contains 8 tables and 3 footnotes.)