Background: The introduction of the Australian Research Training Scheme has been a strong reason for assuring the quality of the research higher degree (RHD) experience; if students experience poor supervision, an unsupportive climate, and inadequate infrastructure, prior research suggests RHD students will be less likely to complete their degree,…

The National Student Survey (NSS) of course experience satisfaction is sent to final year students of all higher education institutions in England, Northern Ireland, Scotland and Wales (N = 171,290 and 157,342 students in 2005 and 2006) and results are made publicly available. The present investigation assesses the reliability and appropriate use…

Peer review is a gatekeeper, the final arbiter of what is valued in academia, but it has been criticized in relation to traditional psychological research criteria of reliability, validity, generalizability, and potential biases. Despite a considerable literature, there is surprisingly little sound peer-review research examining these criteria or…

The reliability, long-term stability, and generalizability of student ratings of teacher effectiveness are discussed. The Students' Evaluation of Educational Quality (SEEQ) instrument is examined from these perspectives. The multidimensionality of student response to such evaluation instruments must be recognized. (SLD)

The peer review process used to select manuscripts for publication in academic journals is important to the individual researcher and the scientific community, but it has been the topic of little research. This article analyzes the reviewers overall recommendation, ratings on specific subscales, and written critiques. Reviewer reliability is…

Reanalyzed data from the Self-Description Questionnaire II (SDQII) from an earlier study of 376 gifted students, and a new study of 837 gifted students to explore alternative approaches to missing data and data normalization. Results strongly support the construct validity of responses to the SDQII by gifted adolescents. (SLD)

Evaluated external assessor ratings of the quality of the research team and the proposed research for proposals submitted to the Australian Research Council large grants program. Data from 313 evaluations show that the reliability of the assessments varied systematically with the number of assessors and the score considered. Discusses implication…

Over 1,000 undergraduate and graduate business administration majors from 100 different classes provided course and instructor evaluations at the end of each class, and again at least one year after program completion. Results showed large and statistically significant correlations between end-of-term and retrospective ratings. (Author/GDC)

The construct validity of responses to the Tennessee Self Concept Scale (TSCS) was evaluated in two sets of analyses. First, exploratory and confirmatory factor analyses, and an analysis of variance model adapted from multitrait-multimethod (MTMM) research, were used to examine the internal structure of the TSCS responses. Second, MTMM analyses…

In five studies, the applicability of two North American instruments for student evaluation of teacher performance was assessed at the University of Western Sydney (Macarthur, Australia). Results of a multitrait-multimethod analysis indicate the instruments are applicable at this institution and across diverse educational settings. (Author/MSE)

A paradigm is described for studying the applicability of two North American instruments (the Students' Evaluation of Educational Quality and Frey's Endeavor) to other countries. Factor analyses of studies in Australia, Spain, and Papua New Guinea support the generality of the instruments' factors to the educational contexts of other countries.…

The Self-Description Questionnaire III (SDQ III) was designed to measure 13 factors of self-concept. The reliabilities of the factors were high, and correlations among them were low. Academic achievement measures in language and mathematics were correlated with self-concepts in the same areas but not with other self-concept factors. (Author/DWH)

Test-retest data (six-month interval) collected on preadolescent student self-concepts, teachers' ratings of student self-concepts, and academic ability provided strong support for the construct validity of self-concept. The multiple dimensions of self-concept showed a logical and consistent pattern of relationships with a variety of criteria.…

Developed and tested the psychometric properties of the 38-item Self Description Questionnaire for Preschoolers. Found that the scale was reliable for young children, first-order and higher order confirmatory factor analysis models fit the data, and factor correlations were mostly moderate. Achievement test scores correlated moderately with…

Agreement between two independent reviews of each of 278 manuscripts was compared on an overall recommendation and on specific rating items. Agreement between reviewers on separate dimensions, the unweighted sum of the dimensions, and various weighted sums was no better than that for the overall recommendation itself. (SLD)