Comparability studies between CBT and PPT studied if there were score differences between these two versions. However, there were no consistent results across studies and time. The purpose of this study is to summarize the testing mode effects for K-12 mathematics tests by conducting a meta-analysis. 103 effect sizes of 32 studies were collected across 1987 to 2013. Results find: (1) The estimated mean effect size was -0.11 with the standard error of 0.015. (2) Three moderator variables, grade, item type and source of studies, statistically significantly influenced the mean effect size. (3) A meta-regression showed that the relationship between the mean effect size and time of studies was statistically significant. (4) Sensitivity analysis indicated students with disabilities might get more disadvantages on CBT.