This study investigated the effectiveness of a web-based assessment training program and whether explicitly guiding the participants toward the recommended learning sequence would improve their performance on assessing the underhand roll compared to participants that were given free choice of their learning sequence. Participants were 48 volunteer pre-service undergraduate kinesiology majors from two universities that were randomly assigned to two treatment groups. The data collected on the participants included pre and post assessment scores, as well as program usage data: the frequency and time spent using each of the training program options. Results revealed a non-significant treatment effect was found between the two treatment groups, but significant overall pre to post performance gain of 24.4 percent (ES = 0.772). Program usage data revealed that participants with higher competency scores significantly used the guided practice option more frequently and for more time and overall used the program for more time than participants with lower competency scores. (Contains 2 figures and 2 tables.)