The relationship between Information and Communication Technology (ICT) and academic performance is a controversial issue that has attracted increasing attention from administrators, policymakers, and researchers. The relationship between perceived ICT competence and the academic performance of rural students deserves particular attention. Although a small but growing body of research has examined the relationship between perceived ICT competence and student academic performance, few studies have viewed perceived ICT competence as a multilevel construct. This study aimed to fill this gap by examining the relationship between multilevel perceived ICT competence (i.e., student- and school-level perceived ICT competence) and student academic performance using a sample of 5530 students from 156 schools in rural China. Two-level hierarchical linear modeling results indicated that student- and school-level perceived ICT competence could predict academic performance. Furthermore, school-level perceived ICT competence could moderate the relationship between student-level ICT competence and academic outcomes. Specifically, the role of student-level perceived ICT competence showed heterogeneity across schools. Academic performance was strongly correlated with student-level perceived ICT competence in schools with a low level of perceived ICT competence; in contrast, this outcome was not observed in schools with a high level of perceived ICT competence. The findings suggest that administrators and policymakers in China should pay special attention to rural schools where perceived ICT competence is low and consider providing services for students in these schools to promote educational equity.