Small private online courses (SPOCs) have influenced teaching and learning in China's higher education. Learning management systems (LMSs) are important components in SPOCs. They can collect various data related to student behavior and support pedagogical interventions. This research used feature engineering and nearest neighbor smoothing models to predict the performance of students. Five learning behavior features were selected based on Spearman's rank correlation coefficients with students' final grades. Through testing with data from the fall semester of 2020, the model attained the highest ROC-AUC value of 0.9390. Based on these models, the researchers conducted an engagement intervention that displayed learning behavior dashboards to students in the fall of 2021. During the intervention, the course platform updated the dashboards and notified students weekly. This intervention was further investigated through a randomized controlled trial. The experimental results suggested that the intervention could improve students' learning behavior in terms of total study time, tutorial reading, and video viewing. In addition, this study used a modified dynamic key-value memory network model to depict a student's knowledge state and to calculate the probability of solving an exercise by mining numerous exercise records. Based on the predicted probability, instructors could recommend personalized exercises for each student. In the fall of 2021, the researchers also conducted a randomized controlled trial on this intervention, demonstrating that this personalized exercise recommendation could increase students' concept mastery. Experiments revealed that the proposed models and interventions had a positive effect on students' learning of course content.