ERIC Number: EJ993912
Record Type: Journal
Publication Date: 2012
Pages: 3
Abstractor: ERIC
ISBN: N/A
ISSN: ISSN-1536-6367
EISSN: N/A
Measuring What?
Porter, Theodore M.
Measurement: Interdisciplinary Research and Perspectives, v10 n3 p167-169 2012
Ruscio et al. (Ruscio, Seaman, D'Oriano, Stremlo, & Mahalchik, this issue) write of a thing with which scientists and scholars are all too familiar, the assessment of published research and of its authors. The author was startled to discover how little the agenda of the paper seems to engage with factors one relies on for salary and promotion decisions. Scholarly impact and productivity are indeed paramount considerations, but to the extent one chooses to quantify these qualities, one normally relies on two categories of numbers: (1) numbers of publications; and (2) numbers of citations. Ruscio et al. undertake to compress this duality into a single measure of "scholarly impact." Ruscio et al. set about defining measures that will be free of the arbitrariness of subjective judgment. They have let their investigation be shaped by the quest for measures that can be reined as much as possible by rules. A subjective assessment, they imply, requires interpretation and, hence, is vulnerable to bias, while an objective one cannot be biased if it can be scored by machine. While acknowledging that quantitative measures have a lesser role in personnel decisions than more subjective ones, they are determined to expel every trace of subjectivity from the quantitative side of the analysis. This kind of objectivity, when reason is reduced to an algorithm, can stand in the way of truthful knowledge. The author contends that the focus paper is vitiated by an overwhelming concern to expel human judgment--not from the evaluation process, which, so far, is unrealistic, but from the measures.
Descriptors: Evaluation Criteria, Data Analysis, Evaluative Thinking, Bias, Measurement, Outcome Measures, Scholarship, Bibliometrics, Citations (References), Evaluation Methods, Evaluation Problems, Validity, Measurement Techniques, Periodicals, Productivity, Researchers, Scientific Research, College Faculty, Faculty Promotion, Faculty Publishing, Faculty Evaluation, Citation Indexes, Professional Recognition, Information Technology
Psychology Press. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site: http://www.tandf.co.uk/journals
Publication Type: Journal Articles; Opinion Papers
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A