ERIC Number: EJ1130331
Record Type: Journal
Publication Date: 2017
Pages: 11
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0818-8068
EISSN: N/A
When Rating Systems Do Not Rate: Evaluating ERA's Performance
Henman, Paul; Brown, Scott D.; Dennis, Simon
Australian Universities' Review, v59 n1 p58-68 2017
In 2015, the Australian Government's Excellence in Research for Australia (ERA) assessment of research quality declined to rate 1.5 per cent of submissions from universities. The public debate focused on practices of gaming or "coding errors" within university submissions as the reason for this outcome. The issue was about the in/appropriate allocation of research activities to Fields of Research. This paper argues that such practices are only part of the explanation. With the support of statistical modelling, unrated outcomes are shown to have also arisen from particular evaluation practices within the discipline of Psychology and the associated Medical and Health Sciences Research Evaluation Committee. Given the high stakes nature of unrated outcomes and that the evaluation process breaches public administration principles by being not appealable nor appropriately transparent, the paper concludes with recommendations for the strengthening ERA policy and procedures to enhance trust in future ERA processes.
Descriptors: Rating Scales, Foreign Countries, Universities, Achievement Rating, Evaluation Methods, Research, Public Agencies, Statistical Analysis, Bayesian Statistics, Probability
National Tertiary Education Union. PO Box 1323, South Melbourne 3205, Australia. Tel: +61-3-92541910; Fax: +61-3-92541915; e-mail: editor@aur.org.au; Web site: http://www.aur.org.au
Publication Type: Journal Articles; Reports - Evaluative
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Australia
Grant or Contract Numbers: N/A