Concerns about the negative effects of digitalization on students' reading comprehension, empirically backed by recent meta-analyses, question the efficacy of digital tools in the language arts classroom. By analyzing data from 4 and 8th grade US students from NAEP 2017, we aimed to test the generalization of the negative association between screens use and reading comprehension test scores within language arts classrooms, and to identify teachers' practices to support comprehension, which could reduce such a negative relationship. We used data from 149,400 4th grade and 144,900 8th grade students to predict their reading comprehension scores based on their frequency of use of digital devices in the language arts class, as well as on the specific learning activities performed with such devices. Results revealed that amount of daily use of digital devices was negatively related to scores on a reading comprehension test. In addition, teachers' uses of digital tools to support students' reading comprehension showed positive relations for student use of digital devices for reading projects, and negative relations for activities addressing specific reading skills, such as building and practicing vocabulary. We discuss these results in light of our current understanding of the effects of digitalization on reading.