This study adopted a quasi-experimental design with follow-up interview to develop a computer-based two-tier assessment (CBA) regarding the science topic of electric circuits and to evaluate the diagnostic power of the assessment. Three assessment formats (i.e., paper-and-pencil, static computer-based, and dynamic computer-based tests) using two-tier items were conducted on Grade 4 (n = 90) and Grade 5 (n = 86) students, respectively. One-way ANCOVA was conducted to investigate whether the different assessment formats affected these students' posttest scores on both the "phenomenon" and "reason" tiers, and confidence rating for an answer was assessed to diagnose the nature of students' responses (i.e., scientific answer, guessing, alternative conceptions, or knowledge deficiency). Follow-up interview was adopted to explore whether and how the various CBA representations influenced both graders' responses. Results showed that the CBA, in particular the dynamic representation format, allowed students who lacked prior knowledge (Grade 4) to easily understand the question stems. The various CBA representations also potentially encouraged students who already had learning experience (Grade 5) to enhance the metacognitive judgment of their responses. Therefore, CBA could reduce students' use of test-taking strategies and provide better diagnostic power for a two-tier instrument than the traditional paper-based version.