NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: EJ1273106
Record Type: Journal
Publication Date: 2020
Pages: 12
Abstractor: As Provided
ISBN: N/A
ISSN: EISSN-1554-5210
EISSN: N/A
The Use of Open-Ended Questions in Large-Scale Tests for Selection: Generalizability and Dependability
Atilgan, Hakan; Demir, Elif Kübra; Ogretmen, Tuncay; Basokcu, Tahsin Oguz
International Journal of Progressive Education, v16 n5 p216-227 2020
It has become a critical question what the reliability level would be when open-ended questions are used in large-scale selection tests. One of the aims of the present study is to determine what the reliability would be in the event that the answers given by test-takers are scored by experts when open-ended short answer questions are used in large-scale selection tests. On the other hand, another aim of the study is to reveal how reliability changes upon changing the number of items and raters and what the required number of items and raters is to reach a sufficient degree of reliability. The study group consisted of 443 8th grade students from three secondary schools located in three different towns of the city of Izmir. These students were given a test including 20 open-ended short answer questions which was developed within the scope of the study. Students' answers were rated by four experienced teachers independently of one another. In the analyses, G theory's fully crossed two-facet design p "x" i "x" r with students (p), items (I) and raters (r). The analyses found "Ep"[superscript 2] and [phi]=0,855 and it was concluded that well-educated raters in rating open-ended short answer questions can achieve consistent scoring at an adequate level.
International Association of Educators. Available from: PEN Academic Publishing. e-mail: secretary@inased.org; e-mail: info@penpublishing.net; Web site: http://www.inased.org/ijpe.htm; Web site: http://ijpe.penpublishing.net/
Publication Type: Journal Articles; Reports - Research
Education Level: Secondary Education; Elementary Education; Grade 8; Junior High Schools; Middle Schools
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Turkey
Grant or Contract Numbers: N/A