The peer review process used to select manuscripts for publication in academic journals is important to the individual researcher and the scientific community, but it has been the topic of little research. This article analyzes the reviewers overall recommendation, ratings on specific subscales, and written critiques. Reviewer reliability is…

Agreement between two independent reviews of each of 278 manuscripts was compared on an overall recommendation and on specific rating items. Agreement between reviewers on separate dimensions, the unweighted sum of the dimensions, and various weighted sums was no better than that for the overall recommendation itself. (SLD)