Because of the increased popularity of computerized adaptive testing (CAT), many admissions tests, as well as certification and licensure examinations, have been transformed from their paper-and-pencil versions to computerized adaptive versions. A major difference between paper-and-pencil tests and CAT from an examinee's point of view is that in…

This paper argues that special procedures for constructing assessment tools containing performance assessment tasks are unnecessary and that current test methodology can easily be generalized to complex performance assessment tasks without destroying the desirable characteristics of those tasks. Reasonable statistical requirements for sound…

This study provided initial validity evidence for multidimensional measures of coaching competency derived from the Coaching Competency Scale (CCS). Data were collected from intercollegiate men's (n = 8) and women's (n = 13) soccer and women's ice hockey teams (n = 11). The total number of athletes was 585. Within teams, a multidimensional…

Definition of the issues to the use of latent trait models, specifically one- and three-parameter logistic models, in conjunction with multi-level achievement batteries, forms the basis of this paper. Research results related to these issues are also documented in an attempt to provide a rational basis for model selection. The application of the…

Describes tailored testing (in which a computer selects appropriate items from an item bank while an examinee is taking a test) and shows it to be superior to paper-and-pencil tests in such areas as reliability, security, and appropriateness of items. (IRT)

Residency educators are identifying approaches to measure resident competence. Portfolios are well suited since they require work already completed as part of patient care where competency must be demonstrated. This paper describes assessment of the reliability and validity of portfolios in a psychiatry residency program. This was a…

A live tailored testing study was conducted to compare the results of using either the one-parameter logistic model or the three-parameter logistic model to measure the performance of college students on multiple choice vocabulary items. The results of the study showed the three-parameter tailored testing procedure to be superior to the…

The overall purpose of this research was to compare a maximum likelihood based tailored testing procedure to a Bayesian tailored testing procedure. The results indicated that both tailored testing procedures produced equally reliable ability estimates. Also an analysis of test length indicated that reasonable ability estimates could be obtained…

An example application of portfolio assessment was developed and the model and estimates of reliability derived from the literature were then used to estimate the characteristics of an operational large-scale portfolio assessment program. Costs were estimated to put results in a realistic context. (SLD)

Five comparisons were made relative to the quality of estimates of ability parameters and item calibrations obtained from the one-parameter and three-parameter logistic models. The results indicate: (1) The three-parameter model fit the test data better in all cases than did the one-parameter model. For simulation data sets, multi-factor data were…

This paper has shown the advantages of tailored testing over traditional methods, the need for latent trait theory in the application of tailored testing, and some of the results available to show the quality of estimates obtained by tailored testing procedures. Since the tailored testing procedures overcome many of the problems with traditional…

The reliability and validity of a tailored testing procedure based on the simple logistic model was determined for an achievement test in statistics and measurement. The test was administered on a CRT terminal to students from graduate and undergraduate measurement courses. Equivalent form reliability over a one-week interval was found to be 0.595…

An application of the two-paramenter logistic (Rasch) model to tailored testing is presented. The model is discussed along with the maximum likelihood estimation of the ability parameters given the response pattern and easiness parameter estimates for the items. The technique has been programmed for use with an interactive computer terminal. Use…

This report describes a study comparing the classification results obtained from a one-parameter and three-parameter logistic based tailored testing procedure used in conjunction with Wald's sequential probability ratio test (SPRT). Eighty-eight college students were classified into four grade categories using achievement test results obtained…

A study was conducted to compare tailored testing procedures based on a Bayesian ability estimation technique and on a maximum likelihood ability estimation technique. The Bayesian tailored testing procedure selected items so as to minimize the posterior variance of the ability estimate distribution, while the maximum likelihood tailored testing…