A growing number of educators expect that artificial intelligence (AI) will augment students' capacities and rapidly transform the teaching and learning practice. However, there is a lack of convincing evidence on the effects of Student-AI Collaboration (SAC) on a learning task's performance. A critical examination of the effects on students' learning performance is a crucial step in understanding the potential benefits of SAC on learning. Through a repeated measure experiment participated by 20 undergraduate students in South Korea, this study examined the effects of SAC on a public advertisement drawing task. The findings revealed that SAC significantly affects creativity in content, expressivity in expression, and public utility in effectiveness varied depending on students' attitude toward AI or on the level of drawing skill. Implications for the design of educational AI and AI literacy education are discussed.