When institutions introduce financial aid policies, researchers have an opportunity to study these policies within the framework of a natural experiment. For this methodological note, I use a natural experiment that occurred at the public university when institutional scholarship for all first-time freshmen meeting a minimum high school grade point average and test score requirements was introduced. Scholarship renewal was contingent on credit hour accumulation and grade point averages while in college. I compare students with the same high school grades and test scores before and after the new policy implementation to evaluate the mechanisms of the associations among policy, loss of merit aid eligibility or failure to maintain grades or to accumulate credit hours, and retention. I suggest that "mediation" describes the association between the new policy and retention of high achieving in-state students. After the policy was implemented, these students were more likely to maintain their grades and full-time enrollment to remain scholarship eligible and, therefore, had greater return rates. "Moderation" helps to explain the association between merit aid and retention of out-of-state students. The effect of failure to maintain grades or full-time enrollment on departure was stronger for out-of-state students once financial aid was at stake. Limitations of inferring causation from natural experiments are discussed.