In this article I provide an overview of the methodological issues involved in measuring constructs relevant to character development and education. I begin with a nontechnical overview of the 3 fundamental psychometric properties of measurement: reliability, validity, and equivalence. Developing and evaluating measures to ensure evidence of all 3…

This article presents the results of a systematic review of methods that have been used to measure or assess metacognition in children aged 4-16 years over a 20-year period (1992-2012). It includes an overview of the types of tool and methods used linked with the ages of the participants targeted and how metacognition and associated concepts are…

Children and youth with deficits in social competence present substantial challenges for schools, teachers, parents and peers. These challenges cut across disciplinary, instructional and interpersonal domains and they frequently create chaotic home, school and classroom environments. Schools are charged with teaching an increasingly diverse…

In their study about the Dr. Fox lecture, Naftulin, Ware, and Donnelly (1973) claimed that an expressive speaker who delivered an attractive lecture devoid of any content could seduce students into believing that they had learned something significant. Over the decades, the study has been (and still is) cited hundreds of times and used by…

We systematically evaluated the peer-reviewed Rorschach validity literature for the 65 main variables in the popular Comprehensive System (CS). Across 53 meta-analyses examining variables against externally assessed criteria (e.g., observer ratings, psychiatric diagnosis), the mean validity was r = 0.27 (k = 770) as compared to r = 0.08 (k = 386)…

Meta-assessment, or the assessment of assessment, can provide meaningful information about the trustworthiness of an academic program's assessment results (Bresciani, Gardner, & Hickmott, 2009; Palomba & Banta, 1999; Suskie, 2009). Many institutions conduct meta-assessments for their academic programs (Fulcher, Swain, & Orem, 2012),…

The current review addresses the validity of published meta-analyses in education that determines the credibility and generalizability of study findings using a total of 56 meta-analyses published in education in the 2000s. Our objectives were to evaluate the current meta-analytic practices in education, identify methodological strengths and…

The authors have over the years tried to revise meta-analysis because it's basic premise is to add apples and oranges together and analyze. In other words, various data on the same subject are chosen using different samples, research strategies, and number properties. The findings are then homogenized and a statistical analysis is used (Snell, J.…

Ruscio, Seaman, D'Oriano, Stremlo, and Mahalchik (this issue) evaluate 22 bibliometric indicators, including conventional measures, like the number of publications, the "h" index, and many "h" index variants. To assess the quality of the indicators, their well-justified criteria encompass conceptual, empirical, and practical…

Although the student evaluation of teaching has been extensively researched, no general consensus has been reached about the validity of the process. One contentious issue has been the relationship between the evaluations and learning. If good instruction increases the amount of learning that takes place, then learning and the evaluations should…

A meta-analysis of published studies that reported correlations between self-ratings and informant ratings of well-being (life-satisfaction, happiness, positive affect, negative affect) was performed. The average self-informant correlation based on 44 independent samples and 81 correlations for a total of 8,897 participants was r = 0.42 [99%…

Good quantitative evidence does not require large, aggregate group designs. The authors describe ground-breaking work in managing the conceptual and practical demands in developing meta-analytic strategies for single subject designs in an effort to add to evidence-based practice. (Contains 2 figures.)

Purpose: This study assessed the psychometric properties of instruments used to measure constructs similar to stuttering-specific health-related quality of life. In the stuttering literature, most such instruments were originally intended to measure speakers' attitudes about, or reactions to, their stuttering. Method: Seventeen instruments were…

Reviewed validity evidence for systematic methods of evaluating training and experience (T&E) ratings in personnel selection. Meta-analytic summaries of data indicated that validity varied with type of T&E procedure used. Illinois job element and behavioral consistency methods demonstrated useful levels of validity with small corrected standard…

This project consisted of a meta-analysis of U.S. research published from 1980 to 2004 on the effect of specific science teaching strategies on student achievement. The six phases of the project included study acquisition, study coding, determination of intercoder objectivity, establishing criteria for inclusion of studies, computation of effect…