Background: Youth programs often rely on self-reported data without clear evidence as to the accuracy of these reports. Although the validity of self-reporting has been confirmed among some high school and college age students, one area that is absent from extant literature is a serious investigation among younger children. Moreover, there is…

The sophisticated analytical techniques available to institutional researchers give them an array of procedures to estimate a causal effect using observational data. But as many quantitative researchers have discovered, access to a wider selection of statistical tools does not necessarily ensure construction of a better analytical model. Moreover,…

This report summarizes the complex research literature on quantitative methods for assessing how impacts of educational interventions on instructional practices and student learning differ across students, educators, and schools. It also provides technical guidance about the use and interpretation of these methods. The research topics addressed…

Districts now have access to a wealth of new information that can help target students with appropriate supports and bring focus and coherence to college readiness efforts. However, the abundance of data has brought its own challenges. Schools and school systems are often overwhelmed with the amount of data available. The capacity of districts to…

This guide is designed to help researchers make sure that their research reports include enough information about study measures so that readers can assess the quality of the study's methods and results. The guide also provides examples of write-ups about measures and suggests resources for learning more about these topics. The guide assumes…

Characterising the relationship between participants' scores on two different questionnaires is a common problem in educational research. The complement of the statistic known as Wilks' lambda measures the amount of variance shared between the scores obtained by the same group of participants on two sets of variables. (1 - lambda) is symmetric, in…

Some researchers have argued that discrepant broad index scores invalidate IQs, but others have questioned the fundamental logic of that argument. To resolve this debate, the present study used a nationally representative sample of children (N = 1,200) who were matched individually for IQ. Children with significantly uneven broad index score…

Issues regarding confidence and tolerance intervals are discussed within the context of educational measurement. Conceptual distinctions are drawn between these two types of intervals; and examples, under various error and true score models, are used to compare such intervals. It is shown that there tend to be only small differences in tolerance…

Two strategies for assessing item bias are discussed: methods comparing item difficulties unconditional on ability and methods comparing probabilities of response conditional on ability. Results suggest that the iterative logit method is an improvement on the noniterative one and is efficient in detecting biased and unbiased items. (Author/DWH)

This paper compares five growth measures using the High School and Beyond database. The measures are: (1) simple gain (posttest minus pretest); (2) difference between group means; (3) percentage of students who scored higher on the posttest than the pretest; (4) percentage of items missed on the pretest which are subsequently answered correctly on…

Two methods of 'equating' tests using item response theory (IRT) are compared, one using true scores, the other using the estimated distribution of observed scores. On the data studied, they yield almost indistinguishable results. This is a reassuring result for users of IRT equating methods. (Author)

Median test scores on the Comprehensive Tests of Basic Skills (CTBS) for many District of Columbia public schools declined substantially in 1990, although this decline was not evident in reports from school officials. These declines occurred at elementary, junior high, and senior high school levels, in grades 6, 9, and 11, respectively. They also…

In latent trait theory the latent space, or space of the hypothetical construct, is usually represented by some unidimensional or multi-dimensional continuum of real numbers. Like the latent space, the item response can either be treated as a discrete variable or as a continuous variable. Latent trait theory relates the item response to the latent…

The purpose of this study was to examine the effect that large within-examinee item difficulty variability had on estimates of the proportion of consistent classification of examinees into mastery categories over two test administrations. The classification consistency estimate was based on a single test administration from an estimation procedure…

A procedure for obtaining maximum likelihood trait estimates from number-correct (NC) scores for the three-parameter logistic model is presented. It produces an NC score to trait estimate conversion table. Analyses in the estimated true score metric confirm the conclusions made in the trait metric. (Author/DWH)