The Fresno test of competence in evidence based medicine appears to
be an impressive development.(1) Ramos et. al. assert that it has content
reliability, high inter-rater and internal reliability, good
discriminatory ability, construct validity and avoids floor and ceiling
effects. We believe that the authors should be more circumspect in their
claims.
Supporting data from only two raters was reported and they were
intimately involved in the development of the test. We cannot assume that
other raters will achieve comparable levels of inter-rater reliability,
internal reliability, discriminatory ability, absence of floor and ceiling
effects or acquire the ability to distinguish experts and novices. Levels
of inter-rater reliability, internal consistency and discrimination are
intimately dependent on the population which has taken the test and are
all likely to be lower with a more homogenous group of evidence based
medicine learners. Furthermore ‘the test presently has only one set of
clinical vignettes and one set of numeric examples’.(1) It cannot be
assumed that any of these key attributes can be maintained in subsequent
versions of the test which will need to be developed since the original
tested version and its scoring rubrics have been published.(2) Finally,
the authors state that the ‘best use of the Fresno test is to measure
change in knowledge after instruction’ yet do not present any sensitivity
data to support this statement.
In short, Ramos et. al. cannot yet reassure educators and learners
that the Fresno test is indeed a ‘simple, reliable and valid tool for
assessing knowledge and skill in … evidence-based medicine’.(1)
References
1. Ramos KD, Schafer S, Tracz SM. Validation of the Fresno test of
competence in evidence based medicine. BMJ 2003;326:319-21.
Rapid Response:
The Fresno test
The Fresno test of competence in evidence based medicine appears to
be an impressive development.(1) Ramos et. al. assert that it has content
reliability, high inter-rater and internal reliability, good
discriminatory ability, construct validity and avoids floor and ceiling
effects. We believe that the authors should be more circumspect in their
claims.
Supporting data from only two raters was reported and they were
intimately involved in the development of the test. We cannot assume that
other raters will achieve comparable levels of inter-rater reliability,
internal reliability, discriminatory ability, absence of floor and ceiling
effects or acquire the ability to distinguish experts and novices. Levels
of inter-rater reliability, internal consistency and discrimination are
intimately dependent on the population which has taken the test and are
all likely to be lower with a more homogenous group of evidence based
medicine learners. Furthermore ‘the test presently has only one set of
clinical vignettes and one set of numeric examples’.(1) It cannot be
assumed that any of these key attributes can be maintained in subsequent
versions of the test which will need to be developed since the original
tested version and its scoring rubrics have been published.(2) Finally,
the authors state that the ‘best use of the Fresno test is to measure
change in knowledge after instruction’ yet do not present any sensitivity
data to support this statement.
In short, Ramos et. al. cannot yet reassure educators and learners
that the Fresno test is indeed a ‘simple, reliable and valid tool for
assessing knowledge and skill in … evidence-based medicine’.(1)
References
1. Ramos KD, Schafer S, Tracz SM. Validation of the Fresno test of
competence in evidence based medicine. BMJ 2003;326:319-21.
2. Ramos, K. D., Schafer, S., and Tracz, S. M. Test and validation
process details. http://bmj.com/cgi/content/full/326/7384/319/DC1. (date
accessed 14-2-2003)
Competing interests:
None declared
Competing interests: No competing interests