Background: Since 2002, the Chinese Ministry of Education has conducted reform in the physical education (PE) curriculums of schools in China, with a focus on shifting from sports skills to regular participation in physical activity (PA) and promoting health. The aim of the study, therefore, is to examine the effects of school PE on the exercise habits of children and adolescents in China over time. Methods: Data based on 5941 observations of 3708 individuals aged 6 to 17 were collected from the China Health and Nutrition Survey (CHNS) for the period 2004 to 2015. The data were analyzed using the fixed-effect Logit model and the random-effect Tobit model. Results: The likelihood of exercising outside of school is 20.2% higher for students who have school PE than those who do not. Our study found that increasing the duration of PE at school by 100%, increases the duration of out-of-school PA by 22.3%. The variety of the types of sports schools offer encourages students to participate in out-of-school physical activity. The likelihood of students exercising outside of school increases by 5.6% when 1 more exercise type is provided in school PE. In addition, soccer, basketball, badminton, and volleyball education increases students' participation in after-school exercises. Soccer and basketball education, in particular, improves the duration of after-school PA. Conclusions: To form exercise habits in children and adolescents, we encourage the promotion of a variety of physical activities in schools, especially team sports such as soccer and basketball.