NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
ERIC Number: ED624293
Record Type: Non-Journal
Publication Date: 2021
Pages: 30
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
Accurate Models vs Accurate Estimates: A Simulation Study of Bayesian Single-Case Experimental Designs
Batley, Prathiba Natesan; Hedges, Larry V.
Grantee Submission, Paper presented at the Texas Universities' Educational Statisticians and Psychometricians Annual Meeting (2019) and at the Annual Meeting of the American Educational Research Association (2019)
Although statistical practices to evaluate intervention effects in SCEDs have gained prominence in the recent times, models are yet to incorporate and investigate all their analytic complexities. Most of these statistical models incorporate slopes and autocorrelations both of which contribute to trend in the data. The question that arises is whether in SCED data that show trend, there is indeterminacy between estimating slope and autocorrelation because both contribute to trend and the data have limited number of observations. Using Monte Carlo simulation, we compared the performance of four Bayesian change-point models: (a) intercepts only (IO), (b) slopes but no autocorrelations (SI), (c) autocorrelations but no slopes (NS), and (d) both autocorrelations and slopes (SA). Weakly informative priors were used to remain agnostic about the parameters. Coverage rates showed that for the SA model either the slope effect size or the autocorrelation credible interval almost always erroneously contained 0 and the Type II errors were prohibitively large. Considering the 0-coverage and coverage rates of slope effect size, intercept effect size, mean relative bias, and second phase intercept relative bias, the SI model outperformed all other models. Therefore, it is recommended that researchers favour the SI model over the other three models. Research studies that develop slope effect sizes for SCEDs should consider the performance of the statistic by taking into account coverage and 0-coverage rates. These helped uncover patterns that were not realized in other simulation studies. We underline the need for investigating the use of informative priors in SCEDs. [This paper was published in "Behavior Research Methods."]
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Audience: N/A
Language: English
Sponsor: Institute of Education Sciences (ED)
Authoring Institution: N/A
IES Funded: Yes
Grant or Contract Numbers: R305D170041
Author Affiliations: N/A