Student evaluations of college teaching have been endorsed and criticized for as long as they have been used as part of important decision-making practices in higher education. With the growth of distance education, the need for alternative approaches for these assessments has increased. We were interested in the extent to which outcomes were comparable across in-class and on-line course evaluations. We conducted a randomized controlled trial across 7 colleges, 25 departments, and 41 instructors at a large urban research university in the southeastern part of the United States. The distribution of ratings across demographic and comparison groups was similar. Response rates were lower for students participating online; however, none of the scale score differences between groups exceeded an effect size 0.21 and the estimated benefits were large. We discuss the advantages and disadvantages of alternative approaches for evaluating instruction in the context of past, current, and future research and practice.