NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: EJ1202778
Record Type: Journal
Publication Date: 2018-Dec
Pages: 19
Abstractor: As Provided
ISBN: N/A
ISSN: EISSN-2330-8516
EISSN: N/A
Available Date: N/A
Anchored Graphical Representations: A Graphical Alternative to Traditional Just Qualified Candidate Descriptors for Licensure Tests. Research Report. ETS RR-18-40
Kannan, Priya; Tannenbaum, Richard J.; Hebert, Delano
ETS Research Report Series, Dec 2018
A well-constructed just qualified candidate (JQC) description is needed to arrive at a reasonable passing score for licensure tests. Traditionally, such descriptions consist of a list of knowledge and skill statements without sufficient context to internalize its intended meaning, allowing the standard-setting panelists to make idiosyncratic interpretations. In a series of studies, we evaluated an alternative JQC description called anchored graphical representation (AGR). AGRs are intended to provide both text and visual organizers by each tested domain to contextualize the meaning of the JQC. In Study 1, 22 mathematics educators participated in a mock standard-setting study and were randomly assigned to conditions in which they used either the AGR-based or the text-based definitions in making standard-setting judgments. In Study 2, 17 social studies educators were randomly assigned to either the AGR-based or text-based condition. While the overall passing score did not noticeably vary between the conditions, consistent with our expectations, results from both studies showed that the domain passing scores for the AGR groups reflected the relative importance of each test domain for the JQC and that the inter-panelist variability was smaller for the AGR groups. Collectively, our results indicate that use of the AGR resulted in standard-setting ratings that were more aligned with the panelists' differential expectations of JQCs' performance for each domain, and the results add to the procedural validity evidence supporting the reasonableness of standard-setting recommendations.
Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb@ets.org; Web site: https://www.ets.org/research/policy_research_reports/ets
Publication Type: Journal Articles; Reports - Research; Tests/Questionnaires
Education Level: Elementary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: New Jersey
Identifiers - Assessments and Surveys: Praxis Series
Grant or Contract Numbers: N/A
Author Affiliations: N/A