ERIC Number: EJ1024934
Record Type: Journal
Publication Date: 2014
Pages: 18
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0895-7347
EISSN: N/A
Requiring a Consistent Unit of Scale between the Responses of Students and Judges in Standard Setting
Humphry, Stephen; Heldsinger, Sandra; Andrich, David
Applied Measurement in Education, v27 n1 p1-18 2014
One of the best-known methods for setting a benchmark standard on a test is that of Angoff and its modifications. When scored dichotomously, judges estimate the probability that a benchmark student has of answering each item correctly. As in most methods of standard setting, it is assumed implicitly that the unit of the latent scale of the responses of the judges is the same as that of the responses of the students. In an empirical example using a modified Angoff approach, this article shows that this assumption is not correct. In particular, using the dichotomous Rasch model, the article shows that the unit of the latent scale for the students is smaller than that of the judges. Accordingly, to obtain an unbiased estimate of a standard, the article shows that it is necessary to place the judges' responses onto a latent scale with the same unit as that of the students. Implications for standard setting procedures are considered, including understanding some observed difficulties reported in the literature in applying Angoff's standard setting procedure.
Descriptors: Foreign Countries, Standard Setting (Scoring), Judges, Item Response Theory, Computation, Probability, Reading Tests, Comparative Analysis, Grade 7, Students, Benchmarking
Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site: http://www.tandf.co.uk/journals
Publication Type: Journal Articles; Reports - Research
Education Level: Grade 7; Elementary Education; Middle Schools; Junior High Schools; Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Australia
Grant or Contract Numbers: N/A