Institutions of higher education have developed a host of interventions to increase the retention of first-year students whose academic performance is inadequate. This study evaluated the overall effectiveness of an intervention strategy designed for first-year students on academic probation at a mid-western research university. A quasi-experimental design was generated to identify probationary students' sentiments about various issues relating to their first-year experience. The one-parameter logistic (1PL) Rasch measurement model was utilized for data analysis. Results suggest the intervention strategy is very effective in meeting its goals and can significantly improve students' intellectual, social, and emotional experiences at college. This study contributes to the research literature in several key ways: an empirically-supported intervention strategy is identified; the modified instrument and study design serves as a useful evaluation model for other institutions/programs; and the evaluation process includes data analysis via the Rasch model. Additional areas of research are identified for future studies. (Contains 5 tables and 1 figure.)