When making decisions, the amount of time remaining matters. When time horizons are long, exploring unknown options can inform later decisions, but when time horizons are short, exploiting known options should be prioritized. While adults and adolescents adapt their exploration in this way, it is unclear when such adaptation emerges and how individuals behave when time horizons are ambiguous, as in many real-life situations. We examined these questions by having 5- to 6-year-olds (N = 43), 11- to 12-year-olds (N = 40), and adult college students (N = 49) in the United States complete a Simplified Horizons Task under short, long, and ambiguous time horizons. Adaptation to time horizons increased with age: older children and adults explored more when horizons were long than when short, and while some younger children adapted to time horizons, younger children overall did not show strong evidence of adapting. Under ambiguous horizons, older children and adults preferred to exploit over explore, while younger children did not show this preference. Thus, adaptation to time horizons is evident by ages 11-12 and may begin to emerge around 5-6 years, and children decrease their tendencies to explore under short and ambiguous time horizons with development. This developmental shift may lead to less learning but more adaptive decision making.