There is something about the topic of validity that seems to provoke dissatisfaction in many of those who encounter it--a sense that something is not right, and that something needs to be done to sort it out. Paul E. Newton in his target essay does not attempt a radical reconstruction of the validity edifice. His position is that the "consensus…

As illuminated forcefully by Professor Newton's provocative analytical and historical excursion, as long as tests are employed to practical ends (prediction, selection, etc.) there is little cause for the metatheoretic angst that occasions rounds of papers on the topic of validity. But then, also, there seems little need, within this context of…

Paul E. Newton's article on the consensus definition of validity tackles a number of big issues and makes a number of strong claims. I agreed with much of what he said, and I disagreed with a number of his claims, but I found his article to be consistently interesting and thought provoking (whether I agreed or not). I will focus on three general…

This paper presents the authors' critique on "Clarifying the Consensus Definition of Validity" by Paul E. Newton (this issue). There are serious differences of opinion regarding the topic of validity. Newton is aware of these differences, as made clear by his choice of references and particularly his effort to respond to the various Borsboom…

The author responds to the recent work of Kaminski, Sloutsky, and Heckler (2008) and advances two major concerns about their research and its applicability to learning mathematics: a confounding variable that arises from the mathematical differences between the generic examples and concrete examples poses a threat to the construct validity of the…

What is the relation between the cognitive competence of a national population that nation's economic prosperity? Lynn and Vanhanen [Lynn, R. & Vanhanen, T. (2002). "IQ and the wealth of nations." Westport, CT: Praeger.] presented data pointing to an exceptionally strong relationship between IQ scores and Gross Domestic Product per…

This article presents the authors' response to the discussants of their article "A Suggested Change in Terminology and Emphasis Regarding Validity and Education" (this issue)--Susan E. Embretson, Joanna S. Gorin, Robert J. Mislevy, Pamela A. Moss, and Stephen G. Sireci. Their response is limited to a brief summarization of their position…

Discusses design and evaluation concepts and methodologies in the context of comprehensive system design (CSD) and considers the concept of error throughout. Topics addressed include six core concepts of CSD; design processes; evaluation concepts; evaluation processes, including criterion-related validity, construct validity, and systemic…

A theory-driven approach to validity is proposed. The central argument is that a model or theory should be formulated in a program evaluation and the modeling process should include the identification of potential threats to validity in research. (Author/LMO)

The validity of students' evaluations of university teacher effectiveness must be demonstrated through a construct validation approach. Empirical relations between students' evaluations of teaching effectiveness (such as the Students' Evaluation of Educational Quality instrument) and other indicators of effective teaching are examined. (SLD)

This paper focuses on issues of data trustworthiness in praxis-oriented empirical work, research openly committed to the building of a world in which we can all flourish. The central argument is that those exploring the possibilities for a change-enhancing advocacy paradigm for doing empirical research in the human sciences must begin to be more…

Six distinguishable aspects of construct validity are discussed as they apply to performance assessment, emphasizing content, substantive, structural, generalizability, external, and consequential aspects. Taken together, these aspects provide a way to address validity questions in score interpretation and use. (SLD)

NCATE (2002) requires the measurement of knowledge, skills, and dispositions as part of its accreditation requirements for teacher education programs (Standard 1) and the use of unit assessment systems to aggregate and analyse data with a view toward program improvement (Standard 2). Data must indicate that candidates meet professional, state, and…