Reform efforts in statistics education emphasize the need for students to develop statistical thinking. Critical to this goal is a solid understanding of design in the process of collecting data, evaluating evidence, and drawing conclusions. We collected survey responses from over 700 college students at the start of an introductory statistics…

Institutional researchers are often tasked with assessing why college-wide initiatives succeed or fail. This can be a difficult task: researchers need to discriminate between multiple feasible explanations, work with limited data, and produce compelling narratives. Process tracing is a qualitative methodology that enables researchers to make valid…

This pan Canadian participatory action research project involved faculty and managers in twelve Canadian colleges and institutes, nine employers, and 1300 students and workers in examining exemplary practice in essential skills as a way of catalyzing system change. Multiple sources of data were used in the three year project and key findings…

Summative student evaluation of teaching (SET) is a contentious process, but given the increasing emphasis on quality and accountability, as well as national and international calls for centralised student feedback systems, is likely to become an inevitable aspect of teaching. This research aimed to clarify academics' attitudes to SET in a large…

Because it is often impractical or impossible to obtain school transcripts or records on subjects, many researchers rely on college students to accurately self-report their academic record as part of their data collection procedures. The purpose of this study is to investigate the validity and reliability of student self-reported academic…

Districts now have access to a wealth of new information that can help target students with appropriate supports and bring focus and coherence to college readiness efforts. However, the abundance of data has brought its own challenges. Schools and school systems are often overwhelmed with the amount of data available. The capacity of districts to…

While the significance of validation of data collection instruments in speech act research has been recognized and has attracted considerable interest, most validation studies employed a between-subjects design. In so doing, it is possible that the differences were caused by the group effects rather than different data collection techniques. This…

Student ratings of instruction are widely used as a basis for personnel decisions and faculty development recommendations. This digest discusses concerns about the validity of student ratings and presents a case for their use in teacher evaluation. There are several strong arguments for using student ratings to evaluate teachers. Students are in a…

This study determined the effects of two data collection techniques on response rate, response bias, response validity and respondent evaluation. Administration of a computer survey was compared to a machine readable paper survey. Cover letters were sent to 249 college sophomores who were randomly selected and proportionally stratified by intended…

The Cooperative Assessment of Experiential Learning (CAEL) Project involves Educational Testing Service and some 250 member institutions. The purpose of the project is to develop procedures and materials that will be useful in strengthening the assessment of experiential learning in higher education. This report describes extensive findings that…