Applied psychologists who provide services in nations composed of multiple and varied cultural groups face certain ethical dilemmas that would not arise in more homogeneous societies. These ethical dilemmas revolve around the concept of population validity. Population validity refers to the generalizability of research findings across different…

Three procedures for evaluating the sampling specificity of results are reviewed. These procedures are Tukey's jacknife technique, Efron's bootstrap technique, and cross-validation methods. The jacknife technique uses different subsamples derived from the original total data set to provide empirical estimates of the generalizability of effect…

A procedure was developed to analyze data with missing observations by extracting data from a sparsely filled data matrix into analyzable smaller subsets of data. This subdividing method, based on the conceptual framework of meta-analysis, was accomplished by creating data sets that exhibit structural designs and then pooling variance components…

The purpose of this study was to investigate the relative appropriateness of several procedures for estimating reliability and standard errors of measurement of complex reading comprehension tests. Seven generalizability theory models were conceptualized by incorporating one or several factors of items, passages, themes, contents, and types of…

Generalizabilty theory was used to examine the dependability of anchoring labels of Likert-type scales. In samples of 173 graduate students and 108 graduate students completing attitude measures, variance components associated with labeling were found to be minute, contributing little to observed score variance. Possible explanations are…

This book presents a comprehensive overview of univariate and multivariate generalizability theory, a psychometric model that provides a powerful approach to the analysis of errors of measurement through the use of random-effects and mixed-model analysis of variance. (SLD)

A 2-facet measurement model is identified, and its coefficient of generalizability (CG) is examined. Three other multifaceted measurement models and their CGs are identified. An empirical investigation of all four procedures is conducted using data from a study of the psychopathology of 71 prison inmates. (SLD)

Develops basic concepts of generalizability theory using hypothetical study of reliability of scores of vocational interest inventory. Asserts that generalizability theory makes it possible to consider all possible sources of unreliability simultaneously and to design optimal decision-making study. (NB)

Items previously described as "negatively worded" are redefined as "connotatively inconsistent" because this term has a broader base for generalization. Using generalizability theory with a sample of 102 graduate students, the study showed that connotatively consistent and reversed connotatively inconsistent items were not…

This rejoinder to criticism of a previous article defends the Columbus Group definition of giftedness and talent by comparing and contrasting this definition with Gagne's Differentiated Model of Giftedness and Talent. Thresholds of giftedness and talent, including generalizability and clinical experience that support the Columbus Group definition,…

Evaluated Angoff ratings from eight different occupational licensing examinations through generalizability theory to estimate the optimal number of raters. Results indicate that approximately 10 to 15 raters is an optimal target range. (SLD)

This study evaluated a "substantively driven" method for scoring NAEP writing assessments automatically. The study used variations of an existing commercial program, e-rater[R], to compare the performance of three approaches to automated essay scoring: a "brute-empirical" approach in which variables are selected and weighted solely according to…

Arguments are presented for conceptualizing validity within a postmodern approach. Validity, reliability, and generalizability have been a holy trinity of social science research, and standard definitions of validity have been taken from criteria developed for psychometric tests. From a postmodern point of view, validity is sometimes discarded as…

Generalizability or invariance procedures have been known for over three decades. Through the years, these procedures have not been widely discussed or employed. One reason for the lack of use is that most of the articles on invariance procedures have been mathematically oriented. The mathematical orientation of research articles and the lack of…

Generalizability theory was used in this study of the Group Embedded Figures Test, which was administered to undergraduates in a mathematics course. Results indicated that the test has desirable psychometric characteristics, including test and item difficulty and item discrimination coefficients. (Author/GDC)