ERIC Number: EJ800330
Record Type: Journal
Publication Date: 2008-Jul
Pages: 19
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0895-7347
EISSN: N/A
Application of Generalizability Theory to Concept Map Assessment Research
Yin, Yue; Shavelson, Richard J.
Applied Measurement in Education, v21 n3 p273-291 Jul 2008
In the first part of this article, the use of Generalizability (G) theory in examining the dependability of concept map assessment scores and designing a concept map assessment for a particular practical application is discussed. In the second part, the application of G theory is demonstrated by comparing the technical qualities of two frequently used mapping techniques: construct-a-map with created linking phrases (C) and construct-a-map with selected linking phrases (S). Some measurement facets that influence concept-map scores are explored and how to optimize different concept mapping techniques by varying the conditions for different facets is shown. It is found that C and S are not technically equivalent. The G coefficients for S are larger than those for C under the same condition. Furthermore, a decision(D) study shows that fewer items (propositions) would be needed for S than C to reach desired level of G coefficients if only one occasion could be afforded. On the other hand, C seems to reveal students' understanding about different concepts than S better. For practical purposes, one might prefer S because it is easier to score and produces higher reliability. However, this efficiency comes at the cost of validity. We would trade off validity and reliability for efficiency by including more propositions in C map. (Contains 1 figure and 2 tables.)
Descriptors: Generalizability Theory, Concept Mapping, Validity, Reliability, Interrater Reliability, Grade 8
Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site: http://www.tandf.co.uk/journals/default.html
Publication Type: Journal Articles; Reports - Research
Education Level: Grade 8
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: California
Grant or Contract Numbers: N/A