A good test is one that has at least three qualities: reliability, or the precision with which a test measures what it is supposed to measure; validity, i.e., if the test really measures what it is supposed to measure; and practicality, or if the test, no matter how sound theoretically, is practicable in reality. These are the sine qua non for…

This study applied generalizability theory to investigate the interrater reliability of a team-scored electronic portfolio required for initial teacher certification. The sample consisted of 31 preservice teacher portfolios which were assigned to three groups of portfolio review teams. The review teams, which had received several rounds of…

Pattern generalising problems offer a very rich context for exploring relationships among quantities, expressing generality and representing the same relationship in different ways. Selecting appropriate tasks for students to work on in class is by no means a straightforward process, but there are ways to handle it. This article aims to explore…

The drive to quality-manage medical education has created a need for valid measurement instruments. Validity evidence includes the theoretical and contextual origin of items, choice of response processes, internal structure, and interrelationship of a measure's variables. This research set out to explore the validity and potential utility of an…

What are the most significant factors that motivate and inhibit faculty with regard to teaching in online environments? And what are the specific kinds of experiences that underlie and explain the importance of these factors? One goal of this study was to add to understanding of these issues, but the primary purpose of this study is determining…

We propose that the continuing controversies over the use of quasi-"F"-ratios in psycholinguistic research might be circumvented, if not resolved, by the judicious application of Generalizability Theory (GT) analyses. We argue that GT is a logical extension of the basic rationale behind repeated measures Analysis of Variance (ANOVA) and the…

Responds to the comments of LoSchiavo F. M. and Shatz M. A.; Webster G. D., Nichols A. L., and Schember T. O.; Stroebe W. and Nijstad B.; and Haeffel et al. on the author's original article regarding the assertion that American psychology focuses too narrowly on Americans while neglecting the other 95% of the world's population. The author…

Using generalizability theory to evaluate the reliability of child and adolescent measures enables researchers to enhance precision of measurement and consequently increase confidence in research findings. With an observer-rated measure of child self-regulation, we illustrate how multiple sources of error variance (e.g., raters, items) affect the…

The Mayer, Salovey, & Caruso Emotional Intelligence Test (MSCEIT) has been reported to provide reliable scores for the four-branch ability model of emotional intelligence [Mayer, J. D., Salovey, P., & Caruso, D. R. (2002). "Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT). User's manual." Toronto, Canada: Multi-Health…

Inclusion of students with disabilities in district-wide and state assessments is mandated by federal regulations, and teachers sometimes play an important role in rating these students' work. In this study, trained teachers rated student proficiency in performance assessments in language arts and mathematics in third, fifth, and ninth grades. The…

The consensual assessment technique (CAT) is a measurement tool for creativity research in which appropriate experts evaluate creative products [Amabile, T. M. (1996). "Creativity in context: Update to the social psychology of creativity." Boulder, CO: Westview]. However, the CAT is hampered by the time-consuming nature of the products (asking…

Presented at the national conference for the American Educational Research Association (AERA) in April 2009. This study examined the utility of scores on the SAT writing test, specifically examining the reliability of scores using generalizability and item response theories. The study also provides an overview of current predictive validity…

The present study used generalizability theory to investigate the dependability of systematic observations of students' problem behavior and on-task behavior in classrooms. The Direct Observation Form (McConaughy & Achenbach, 2009) was used with a sample of 24, 6- to-11-year-old children attending 18 different elementary schools. The participants…

The latent state-trait (LST) theory is an extension of the classical test theory that allows one to decompose a test score into a true trait, a true state residual, and an error component. For practical applications, the variances of these latent variables may be estimated with standard methods of structural equation modeling (SEM). These…

In this study, the authors wanted to examine whether success on items testing basic equivalence knowledge, such as the meaning of the equal sign and ability to solve problems such as 3 + 5 = 4 + _, predicted success on items testing more advanced algebraic thinking (i.e. principles of equality and solving equations that use letter variables). This…