This article provides a conceptual framework for considering both the theoretical and methodological factors that underpin the successful validation of a competency framework. Drawing on educational assessment literature, this article argues that a valid competency framework relates to an interpretive judgement of the credibility of the claims…

This paper addresses a relatively unexplored area in the field of learning analytics: how analytics are taken up and used as part of teaching and learning processes. Initial steps are taken towards developing design knowledge for this "middle space," with a focus on students as analytics users. First, a core set of challenges for…

With the widespread use of learning analytics (LA) tools, there is a need to explore how these technologies can be used to enhance teaching and learning. Little research has been conducted on what human processes are necessary to facilitate meaningful adoption of LA. The research problem is that there is a lack of evidence-based guidance on how…

We are using stealth assessment, embedded in "Plants vs. Zombies 2," to measure middle-school students' problem solving skills. This project started by developing a problem solving competency model based on a thorough review of the literature. Next, we identified relevant in-game indicators that would provide evidence about students'…

Information and communication technology (ICT)-enhanced research methods such as educational data mining (EDM) have allowed researchers to effectively model a broad range of constructs pertaining to the student, moving from traditional assessments of knowledge to assessment of engagement, meta-cognition, strategy and affect. The automated…

When validating assessment models built with data mining, generalization is typically tested at the student-level, where models are tested on new students. This approach, though, may fail to find cases where model performance suffers if other aspects of those cases relevant to prediction are not well represented. We explore this here by testing if…

This article focuses on school self-evaluations (SSE). It addresses whether SSE meets quality indicators and whether differences can be found between the quality judgments of school principals and inspectors. Data stem from two complementary data collections: population data of school inspections and a survey of a representative sample of school…

[Slides] presented at the Annual Meeting of National Council on Measurement in Education (NCME) in San Diego, CA in April 2009. This presentation describes how the vehicles for gathering student evidence--task models and test specifications--are developed.

This article provides a typology of reasons for conducting mixed-methods research in special education. The mixed-methods research process is described along with the role of the rationale and purpose of study. The reasons given in the literature for utilizing mixed-methods research are explicated, and the limitations of these reason frameworks…

Suggests two complementary frameworks for providing validity evidence for rating data. One conceptualizes raters as data collection instruments that should be subject to traditional procedures for establishing validity evidence; the other evaluates studies using raters from an experimental design perspective allowing the assessment of the study's…

The development of a valid and reliable method of assessing recreational patterns was the purpose of this study. Recreational activities were classified using activity clusters and the setting for the activity. This information yielded a two-dimensional matrix for viewing activities. The first dimension of the matrix was the recreational activity…

Inferences made in integrative research reviews are as important to the validity of behavioral science knowledge as are those in primary research. The research review is conceptualized as a scientific inquiry involving five stages paralleling those of primary research. Problem formulation is the stage when variables are defined conceptually and…

Article is in the form of a major paper followed by responses of five art educators. (MB)

A conceptual model of quality of life, developed at the Centre for Health Promotion at the University of Toronto (Canada), and associated instrumentation for collecting data from persons with developmental disabilities are presented. Results from a preliminary study with 41 participants support the reliability and validity of the model's…