Chalmers recently published a critique of the use of ordinal a[alpha] proposed in Zumbo et al. as a measure of test reliability in certain research settings. In this response, we take up the task of refuting Chalmers' critique. We identify three broad misconceptions that characterize Chalmers' criticisms: (1) confusing assumptions with…

In response to an argument by Baird, Andrich, Hopfenbeck and Stobart (2017), Michael Kane states that there needs to be a better fit between educational assessment and learning theory. In line with this goal, Kane will examine how psychometric constraints might be loosened by relaxing some psychometric "rules" in some assessment…

The main points of Sijtsma and Green and Yang in Educational Measurement: Issues and Practice (34, 4) are that reliability, internal consistency, and unidimensionality are distinct and that Cronbach's alpha may be problematic. Neither of these assertions are at odds with Davenport, Davison, Liou, and Love in the same issue. However, many authors…

The focus article (Willoughby et al., 2014) (1) introduced the distinction between formative and reflective measurement and (2) proposed that performance-based executive function tasks may be better conceptualized from the perspective of formative rather than reflective measurement. This proposal stands in sharp contrast to conventional…

In this commentary, we offer a set of visual tools that can assist education researchers, especially those in the field of mathematics, in developing cohesiveness from a mixed methods perspective, commencing at a study's research questions and literature review, through its data collection and analysis, and finally to its results. This expounds…

The author previously published a paper discussing how to conduct an analysis based on a cluster sample. In that paper, the author outlined several widely adopted alternative approaches, and pointed out that such approaches are anyway not needed for population figures, and not possible for non-probability samples. Thus, the author queried the…

Notes that conventional practice in social work research is to recommend level of reliability close to 0.80 as minimum standard. Contends that needed reliability varies by situation: that in situations in which important decisions about individuals are being made, 0.90 provides better standard; whereas in descriptive survey research with large…

Innacurate sample data and uncertain estimates are defined as obstacles to assessing the suicide rate among college students. A standardization of research and reporting services is recommended. (JMF)

Much of educational administration research suffers from lack of conformity between data analysis and research design, inattention to the test/retest stability of measurement instruments, and the use of statistics for nominal or interval data with ordinal data, random-sample statistics with nonrandom samples, and unnecessarily complex statistical…

In a recent article in this journal, Lombard, Snyder-Duch, and Bracken (2002) surveyed 200 content analyses for their reporting of reliability tests, compared the virtues and drawbacks of five popular reliability measures, and proposed guidelines and standards for their use. Their discussion revealed that numerous misconceptions circulate in the…

This paper provides a review of two volumes of the "Journal of Industrial Teacher Education" and the "Journal of Vocational Education Research." One of the major findings of this review was that insufficient information was provided to make an informed judgment about internal validity. (SSH)

Research manuscripts recently published in the "Journal of Industrial Teacher Education" (JITE) and the "Journal of Vocational Education Research" (JVER) were reviewed to determine how effectively external validity had been handled. The main issue of concern when discussing validity is the generalizability of findings. (SSH)

Views intercoder reliability as a sampling problem for content analyses. Develops a formula for generating sample sizes needed to have valid reliability estimates. Suggests steps for reporting reliability. (TB)

Noting that improvement in rater reliability means eliminating differences among raters, this paper discusses ways to assess writing evaluator reliability and methods for achieving higher levels of interrater reliability. After showing that reliability can be improved two ways--by increasing the number of raters or measurements made, and by…

Although single-subject research techniques are valuable to counselors, analyzing change by graphs alone is open to major sources of error. Discusses these sources and explores the issues of serial dependency, unreliability of graphs, interjudge disagreement on graphed data, and the use of time-series statistics with relevance to the counselor in…