This study explored the relationship between accuracy of and confidence in performance of 114 prospective primary school teachers in answering diagnostic questions on potential difference in parallel electric circuits. The participants were required to indicate their confidence in their answers for each question. Bias and calibration indices were…

Multiple-choice-questions are common in medical examinations, but guessing biases assessment results. Confidence-based-testing (CBT) integrates indicated confidence levels. It has been suggested that correctness of and confidence in an answer together indicate knowledge levels thus determining the quality of a resulting decision. We used a CBT…

One of the key feedback questions is "where to next?" and this article provides some directions as to where to next for research based on a review of the five articles in this special issue. The directions relate to the critical importance of calibration, the multidimensionality of calibration, the relation of calibration to self-regulation…

This study adopted a quasi-experimental design with follow-up interview to develop a computer-based two-tier assessment (CBA) regarding the science topic of electric circuits and to evaluate the diagnostic power of the assessment. Three assessment formats (i.e., paper-and-pencil, static computer-based, and dynamic computer-based tests) using…

Traditional knowledge assessments rely on multiple-choice type questions that only report a right or wrong answer. The reliance within the education system on this technique infers that a student who provides a correct answer purely through guesswork possesses knowledge equivalent to a student who actually knows the correct answer. A more complete…

In this replication study, we examined the effect of educators' participation in a practice-based professional learning series for designing, implementing, and evaluating Functional Assessment-based Interventions (FABI; Umbreit, Liaupsin, Ferro, & Lane, 2007) on their actual knowledge and perceived knowledge, confidence, and usefulness of…

A single experiment is reported in which introductory psychology students were administered a multiple-choice test on psychology with either 4 (n = 78) or 5 alternatives (n = 92) prior to any lectures being delivered. Two answers were generated for each question: a small answer consisting of their favorite alternative, and a large answer…

The purpose of this study was to explore Pre-service Elementary Science Teachers' (PSTs) conceptual understanding of Particulate Nature of Matter (PNM) through a three-tier diagnostic test. Participants were 215 PSTs from Turkey. Data consisted of participants' responses to the Particulate Nature of Matter Test (PNMT). The PNMT consists of…

This paper presents the Agresti & Coull "Adjusted Wald" method for computing confidence intervals and margins of error for common proportion estimates. The presented method is easily implementable by business students and practitioners and provides more accurate estimates of proportions particularly in extreme samples and small…

This paper describes the use of existing confidence and performance data to provide feedback by first demonstrating the data's fit to a simple linear model. The paper continues by showing how the model's use as a benchmark provides feedback to allow current or future students to infer either the difficulty or the degree of under or over…

Contradictory results have been reported regarding the accuracy of various methods used to assess student learning in higher education. The current study examined student learning outcomes across a multi-section and mult-iinstructor psychology research course with both indirect and direct assessments in a sample of 67 undergraduate students. The…

Testing Theory-based Quantitative Predictions (TTQP) represents an alternative to traditional Null Hypothesis Significance Testing (NHST) procedures and is more appropriate for theory testing. The theory generates explicit effect size predictions and these effect size estimates, with related confidence intervals, are used to test the predictions.…

Many models of recognition are derived from models originally applied to perception tasks, which assume that decisions from trial to trial are independent. While the independence assumption is violated for many perception tasks, we present the results of several experiments intended to relate memory and perception by exploring sequential…

We introduce a specific type of item for knowledge tests, confidence-weighted true-false (CTF) items, and review experiences of its application in psychology courses. A CTF item is a statement about the learning content to which students respond whether the statement is true or false, and they rate their confidence level. Previous studies using…

This study explored students' ability to evaluate their learning from a multimedia inquiry unit about the causes of global climate change. Participants were 90 sixth grade students from four science classrooms. Students were provided with a text describing the causes of climate change as well as graphs showing average global temperature changes.…