Reminders are a popular feature in smartphone apps designed to promote desirable behaviors that are best performed regularly. But can they also promote students' regular studying? In the present study with 85 lower secondary school students aged 10-12, we combined a smartphone-based between- and within-person experimental manipulation with logfile data of a vocabulary learning app. Students were scheduled to receive reminders on 16 days during the 36-day intervention period. Findings suggest that reminders can be a double-edged sword. The within-person experimental manipulation allowed a comparison of study probability on days with and without reminders. Students were more likely to study on days they received a reminder compared to days when they did not receive a reminder. However, when compared to a control group that never received reminders, the effect was not due to students studying more frequently on days with reminders. Instead, they studied less frequently on days without reminders than students in the control group. This effect increased over the study period, with students becoming increasingly less likely to study on days without reminders. Taken together, these results suggest a detrimental side effect of reminders: students become overly reliant on them.