NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
ERIC Number: ED627066
Record Type: Non-Journal
Publication Date: 2022-Jul-25
Pages: 4
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
More Powerful A/B Testing Using Auxiliary Data and Deep Learning
Adam C. Sales; Ethan Prihar; Johann Gagnon-Bartsch; Ashish Gurung; Neil T. Heffernan
Grantee Submission, Paper presented at Artificial Intelligence in Education (Durham, United Kingdom, Jul 27-31, 2022)
Randomized A/B tests allow causal estimation without confounding but are often under-powered. This paper uses a new dataset, including over 250 randomized comparisons conducted in an online learning platform, to illustrate a method combining data from A/B tests with log data from users who were not in the experiment. Inference remains exact and unbiased without additional assumptions, regardless of the deep-learning model's quality. In this dataset, incorporating auxiliary data improves precision consistently and, in some cases, substantially.
Publication Type: Speeches/Meeting Papers; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: Institute of Education Sciences (ED)
Authoring Institution: N/A
IES Funded: Yes
Grant or Contract Numbers: R305D210031