NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1422975
Record Type: Journal
Publication Date: 2024
Pages: 13
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-1536-6367
EISSN: EISSN-1536-6359
Visualizing Agreement: Bland-Altman Plots as a Supplement to Inter-Rater Reliability Indices
Brogan L. Barr; Virginia V. W. McIntosh; Eileen F. Britt; Jennifer Jordan; Janet D. Carter
Measurement: Interdisciplinary Research and Perspectives, v22 n2 p175-187 2024
Even when raters demonstrate agreement in the use of a measure, limited score variability or violation of often-ignored statistical assumptions can result in lower reliability estimates than intuitively expected. This article uses data drawn from two randomized controlled trials of schema therapy and cognitive behavioral therapy for the treatment of major depression and binge eating disorder or bulimia nervosa (N = 212) to illustrate the limits of the intraclass coefficient. Randomly selected therapy sessions were rated for therapeutic alliance quality by independent observers using the well-validated Vanderbilt Psychotherapy Process Scale and Vanderbilt Therapeutic Alliance Scale. Scores on subscales related to therapist behavior were restricted, indicating consistent alliance-supportive actions. Inter-rater reliability estimates were low despite high agreement between raters, however. The use of Bland-Altman plots to visualize agreement and data spread is suggested as a useful tool for researchers, consistent with the ideal of exploring reliability from a number of perspectives.
Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: Researchers
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: New Zealand
Grant or Contract Numbers: N/A