ERIC Number: ED609149
Record Type: Non-Journal
Publication Date: 2020-Jul
Pages: 57
Abstractor: ERIC
ISBN: N/A
ISSN: N/A
EISSN: N/A
A Comparison Study of NAEP-TEL and ICILS
Wang, Yan; Murphy, Kevin B.
National Center for Education Statistics
In 2018, the National Center for Education Statistics (NCES) administered two assessments--the National Assessment of Educational Progress (NAEP) Technology and Engineering Literacy (TEL) assessment and the International Computer and Information Literacy Study (ICILS)--to two separate nationally representative samples of 8th-grade students in the United States. TEL was administered in January-March to 15,400 students, while ICILS was administered in March-May to 9,000 students. Because of some similarities in content and item formats, NCES commissioned a study to explore the similarities and differences between the two assessment frameworks as well as the characteristics of the assessment items. The results of the comparison study showed that the framework targets and design features underlying the scenario-based tasks (SBTs) used in both assessments have many similarities. Both assessments have a focus on information and communication technologies, and both are digital-based assessments that use interactive features in item design. However, there are differences between the two assessments in framework scope and item characteristics. About half of the TEL framework targets are beyond the scope of ICILS; TEL evaluates students' knowledge and skills in analyzing general systems, while ICILS limits its evaluation to computer systems. ICILS uses some response modes (e.g., flowcharts) that are not used in TEL. Forty-two percent of the ICILS items do not allow students to go back to the previous item to change their answers, compared with 85 percent of the TEL SBT items. About two-thirds of ICILS items assess procedural knowledge, while only about one-third of TEL SBT items do that.
Descriptors: National Competency Tests, International Assessment, Computer Literacy, Information Literacy, Technological Literacy, Grade 8, Test Items, Test Content, Computer Assisted Testing, Comparative Analysis, Test Format
National Center for Education Statistics. Available from: ED Pubs. P.O. Box 1398, Jessup, MD 20794-1398. Tel: 877-433-7827; Web site: http://nces.ed.gov/
Publication Type: Reports - Research; Numerical/Quantitative Data
Education Level: Elementary Education; Grade 8; Junior High Schools; Middle Schools; Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: National Center for Education Statistics (ED); American Institutes for Research (AIR)
Identifiers - Assessments and Surveys: National Assessment of Educational Progress
IES Funded: Yes
Grant or Contract Numbers: EDIES12D0002