NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
ERIC Number: ED582118
Record Type: Non-Journal
Publication Date: 2014
Pages: 8
Abstractor: ERIC
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
A Case Study of an International Performance-Based Assessment of Critical Thinking Skills
Wolf, Raffaela; Zahner, Doris; Kostoris, Fiorella; Benjamin, Roger
Council for Aid to Education
The measurement of higher-order competencies within a tertiary education system across countries presents methodological challenges due to differences in educational systems, socio-economic factors, and perceptions as to which constructs should be assessed (Blömeke, Zlatkin-Troitschanskaia, Kuhn, & Fege, 2013). According to Hart Research Associates (2009), there is substantial merit in assessing twenty-first century skills such as critical thinking and writing since about 78% of academic institutions in the United States have established crossdiscipline learning outcomes, so called meta domains (Porter, McMaken, Hwang, & Yang, 2011), that all undergraduate students should possess upon graduation. Furthermore, changing skill demands of graduating students have been observed around the world since the 1990s (Levy & Murname, 2004). Meeting the demands of today's world requires a shift in assessment strategies to measure the skills now prized in a complex global environment. More specifically, assessments that only foster the recall of factual knowledge have been on the decline, whereas assessments that evoke higher-order cognitive skills have seen an accelerating demand in the twenty-first century. As an example, CAE (the Council for Aid to Education) has been developing assessments that target higher-order skills. The Collegiate Learning Assessment-plus (CLA+) is a measure that emulates critical-thinking and writing skills. In late 2012, the Agenzia Nazionale di Valutazione del Sistema Universitario e della Ricerca (ANVUR) approached CAE proposing a research study to test the feasibility of adapting, translating, and administering CLA+ to higher education students in Italy. The purpose of this feasibility study was twofold. The first purpose was to see if it was possible to assess Italian students' higher-order skills as outlined in Table 1. The second purpose was to see if the Italian students' performance was comparable to their American counterparts. CLA+ is a mixed-format type assessment; thus this paper presents the results from the feasibility study as a case study of the successful adaption, translation, and administration of CLA+ in 12 Italian institutions. A discussion is provided regarding how different biases may be addressed within an international context. A second analysis examined whether students from Italy and the US ascribe the same meanings to different item formats (PT and SRQs) thus addressing the issue of measurement equivalence and the feasibility of cross-cultural score comparisons. Results are interpreted within a validity framework.
Council for Aid to Education. 215 Lexington Avenue 16th Floor, New York, NY 10016. Tel: 212-661-5800; Fax: 212-661-9766; e-mail: clateam@cae.org; Web site: http://cae.org
Publication Type: Reports - Research
Education Level: Higher Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Council for Aid to Education (CAE)
Identifiers - Location: Italy; United States
Grant or Contract Numbers: N/A
Author Affiliations: N/A