ERIC Number: ED323225
Record Type: Non-Journal
Publication Date: 1985
Pages: 16
Abstractor: N/A
ISBN: N/A
ISSN: N/A
EISSN: N/A
The Effect of Examinee Pass/Fail Information on the Level of Passing Score Set by Expert Judges.
Saunders, Joseph C.; Linton, Dan L.
Florida Journal of Educational Research, v27 n1 p5-18 Fall 1985
The effect on passing scores of providing judges with pass/fail data for the various possible cut-off score levels was studied. Passing scores produced by individual judges using a modified version of the procedure of W. H. Angoff (1971) were compared with the same judges' passing scores as revised after considering information on the number of examinees who would pass and fail at various cut-score levels in several testing situations. The 7 instruments, each with a corresponding committee of experts, included: (1) 4 basic skills examinations in reading and mathematics for students in grades 6 and 8; (2) 2 tests in reading and mathematics for college underclassmen; and (3) a 51-item observation checklist that is used to assess the classroom performance of beginning public school teachers. In all, 161 judges participated. Knowledge of examinee pass/fail rates did not have a consistent effect on the average level of passing scores set by the groups of expert judges. The observed reduction in the variability of the experts' passing scores, together with the high negative correlations between initial cut scores and the magnitudes of subsequent revisions, suggested that judges with more extreme Angoff passing scores tended to shift toward more typical/moderate positions. Information about examinee pass/fail rates may serve as a reality check in the evaluation process. Five data tables are included. (SLD)
Descriptors: Academic Standards, Achievement Tests, Classroom Observation Techniques, College Students, Cutting Scores, Elementary School Students, Elementary Secondary Education, Evaluation Methods, Evaluators, Higher Education, Knowledge Level, Mathematics Tests, Pass Fail Grading, Public School Teachers, Reading Tests, Scoring, Standard Setting (Scoring)
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A