Educational researchers often need to construct precise and reliable measurement scales of complex and varied representations such as participants' written work, videoed lesson segments and policy documents. Developing such scales using can be resource-intensive and time-consuming, and the outcomes are not always reliable. Here we present…

Phenomenography is an empirical approach to ascertain the qualitatively different ways in which individuals experience and understand aspects of the world around them. Originating in educational research (Marton, F., & Säljö, R. (1976). On qualitative differences in learning. 1- outcome and process. "British Journal of Educational…

Data analysis is a significant methodological component when conducting quantitative education studies. Guidelines for conducting data analyses in quantitative education studies are common but often underemphasize four important methodological components impacting the validity of inferences: quality of constructed measures, proper handling of…

Two studies were conducted for the development and validation of a multidimensional test to assess undergraduate students' mathematical thinking about derivative. The first study involved two phases: question generation and refinement of the Thinking-about-Derivative Test (TDT). The second study included four phases as follows: test…

As one of the most widely used creativity assessment tools, the Consensual Assessment Technique (CAT) has been praised as a valid tool to assess creativity. In Amabile's (1982) seminal work, the inter-rater reliability was defined as construct validity of the CAT. During the past three decades, researchers followed this definition and…

This instrumental case study explored how five newly appointed superintendents identified key stakeholders and built trust and social capital with stakeholders in their districts. Stakeholder, trust, and social capital theory were the lenses that guided this study. We utilized a pragmatic research design and thematic data analysis to interpret our…

This article revisits Paas and Van Merrienboer's (1993) measure of instructional efficiency, which can be applied by educational researchers to compare the effects of different instructional conditions on learning. This measure relied on performance and mental effort on the test, and as such gave an indication of the quality of learning outcomes.…

Grade point average (GPA) often correlates highly with variables of interest to educational researchers and thus offers the potential to greatly increase the statistical power of their research studies. Yet this variable is often underused in marketing education research studies. The reliability and validity of the GPA are closely examined here in…

A study was designed to determine whether the presence or absence of a classroom observer and the prior knowledge or lack of knowledge that an observation was to occur would affect the verbal behavior of teachers as measured by the Flanders System of Interaction Analysis. The variables were dichotomized yielding a 2 x 2 x 2 experimental design…

A study was designed to (1) compare two types of reliability in the observation of teachers' behavior, (2) explore the relationship between observer reliability and the validity of their systematic classroom observations, and (3) investigate the effects of training, observer beliefs, and the passage of time on reliability and validity estimates.…

Educational researchers have claimed that the quantitative and qualitative approaches to educational inquiry are, indeed, compatible. However, it would be unfortunate to discontinue this debate. The quantitative-qualitative debate began with the interpretive approach to social inquiry. Dilthey argued that since cultural/moral sciences differ from…

The methodology employed in 14 motivational orientation studies was reviewed. All studies used either the Education Participation Scale, the Continuing Learning Orientation Index or the Reasons for Educational Participation Scale. Issues discussed concerned factor scoring, factor analysis, rotation, scaling, reliability, and the extent to which…

Evaluation roles used in the Department of Research and Development in the Cincinnati Public Schools are identified and described. These include: project evaluator, local-school evaluator, independent program evaluator, external evaluator and external auditors. The merits of each evaluation role will be discussed as to its relationship with…

Records kept by teachers and data collected by trained observers were examined as alternatives for measuring allocated and engaged instructional time in elementary school classes. The teacher ratings of student engagement did not correlate with observer ratings of engagement, but a high correlation between teacher ratings of engagement and student…

In order to make policy decisions, educators must evaluate educational research proposals and projects. Findings immediately related to practice are often inadequate, omitting the theoretical establishment of principles by which we can explain and predict the phenomena of our world. These theoretical linkages to the practical world of…