Researchers strategically assess youth mental health by soliciting reports from multiple informants. Typically, these informants (e.g., parents, teachers, youth themselves) vary in the social contexts where they observe youth. Decades of research reveal that the most common data conditions produced with this approach consist of discrepancies…

In this paper, I discuss my engagement with a visual journal as a companion to creative research practice during my dissertation research. Grounded in arts based research methodologies; I explore visual journals in relationship to research, reflection and analytic processes. I begin with a discussion of the visual journal as an artifact of…

We explore new tools and methods for learning designers and researchers to characterize pedagogical approaches that are applied to the design of MOOCs. This paper makes three main contributions to literature on MOOC design and evaluation: (1) an Expanded Assessing MOOC Pedagogies instrument for use by learning designers and researchers within…

As one of the three Rs, "'rithmetic" has always been central to education and education research. By virtue of that centrality, research in mathematics education has often reflected and at times led trends in education research. This chapter provides some deep background on epistemological and other issues that shape current research,…

Evaluation in higher education is an evolving social practice; that is, it involves what people, institutions and broader systems do and say, how they do and say it, what they value, the effects of these practices and values, and how meanings are ascribed. The textual products (verbal, written, visual and gestural) that inform and are produced by,…

Primary research in education and social sciences is marked by a diversity of methods and perspectives. How can we accommodate and reflect such diversity at the level of synthesizing research? What are the critical methodological decisions in the process of a research synthesis, and how do these decisions open up certain possibilities, while…

This paper addresses an important problem that may really be a pseudoproblem perpetuated by a current artificial or even political need to isolate researchers, evaluators, and theorists and put them in conceptual-methodological boxes. In this paper, we argue that scholars of all kinds should focus on the nouns (i.e., research, evaluation, theory)…

This paper takes the form of a dialogue between the two authors, and is in two halves, the first half discursive and propositional, and the second half exemplifying the rhetorical, epistemological and metaphysical affordances of poetry in critically scrutinising the rhetoric, epistemology and metaphysics of educational management discourse. The…

Teaching on the philosophies of evaluation as part of a pilot introductory course of a new diploma programme jointly sponsored by the United Nations (UN) Evaluation Group and the United Nations Staff College, the author compared and contrasted the "rationalistic" paradigm with the "naturalistic" paradigm on the basis of ontology, epistemology, and…

The sense that replicability is an important aspect of empirical science led Killeen (2005a) to define "p[subscript rep]," the probability that a replication will result in an outcome in the same direction as that found in a current experiment. Since then, several authors have praised and criticized 'p[subscript rep]," culminating…

Researchers in the areas of psychology and education strive to understand the intersections among validity, educational measurement, and cognitive theory. Guided by a mixed model conceptual framework, this study investigates how respondents' opinions inform the validation argument. Validity evidence for a science assessment was collected through…

This article presents the author's address at the 2007 "Journal of Applied Quantitative Methods" ("JAQM") prize awarding festivity. The festivity was included in the opening of the 4th International Conference on Applied Statistics, November 22, 2008, Bucharest, Romania. In the address, the author reflects on three theses that…

The belief that randomized controlled trials provide a "gold standard" for evaluating therapeutic interventions is challenged. The need for research designs that produce valid and reliable information about interventions, which is clinically/educationally relevant and personally important, is discussed together with possibilities for broadening…

Evaluation research is conceived of as process of asking questions and finding answers. Discusses categorical theory, erotetic logic, pragmatics, and epistemology. (RB)

A group of defenses of qualitative evaluation methods is examined, based on a hard relativistic interpretation of the work of Thomas Kuhn. A promising defense of qualitative evaluation may be found in a soft-relativist interpretation of Kuhn's analysis of the nature of scientific discovery. (Author/LMO)