We examined the potential different effectiveness of an online homework system (IXL) and the traditional paper-and-pencil homework. A study involving 98 participants was conducted in a middle school (grade 8). We compared the post-test results from the online homework group with the traditional homework group. Both homework assignment effect (class-level) and homework completion effect (student-level) proposed by Trautwein (2007) were investigated. No significant difference was found among the students who were assigned different types of homework (class-level). We conclude that IXL is as effective as the traditional homework on students' learning. Meanwhile, not surprisingly, we revealed that students who complete the homework outperformed the students who did not (student-level). We suggest that teachers give students an option to do online or traditional homework based on their preference, as long as they complete the homework.