ERIC Number: EJ1327603
Record Type: Journal
Publication Date: 2022-Apr
Pages: 13
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0266-4909
EISSN: N/A
Assessing Fidelity of Implementation to a Technology-Mediated Early Intervention Using Process Data
Helsabeck, Nathan P.; Justice, Laura M.; Logan, Jessica A. R.
Journal of Computer Assisted Learning, v38 n2 p409-421 Apr 2022
Background: Process data, data generated by a user's interaction with a web-based application, is an emerging tool in educational research. The current study explores using process data as a measure of implementation fidelity to a randomized control trial (RCT) of the Read It Again Mobile (RIA-M) curricular supplement. Objectives: To determine the extent to which teachers implemented RIA-M and to assess the utility of using process data in the assessment of fidelity. Methods: The RCT involved 30 pre-kindergarten classrooms with a sample of n = 216 students. RIA-M provides a curricular supplement which teachers may incorporate into classroom instruction and is delivered via a tablet computer. Pre and post literacy assessments are used to determine treatment effect. Process data, produced from teacher interactions with the tablet, and classroom observations are used to assess fidelity. Results and Conclusions: Our findings indicate no difference between treatment and control students in the RCT. Yet, we find that process data provides unique fidelity information concerning treatment exposure, adherence, and quality of program delivery. Specifically, process data indicated that teachers did not demonstrate the same level of fidelity that was captured in classroom observations. This finding provides some evidence for the absence of an intervention effect. Major Takeaways: The current study improves our understanding of how web-based interventions may be assessed for implementation fidelity using process data. Further, process data offers a potentially reliable and scalable measure of fidelity for other web-based educational interventions.
Descriptors: Fidelity, Program Implementation, Early Intervention, Handheld Devices, Preschool Children, Preschool Curriculum, Supplementary Education, Reading Instruction, Data Use, Preschool Teachers, Web Based Instruction, Educational Technology
Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://bibliotheek.ehb.be:2191/en-us
Publication Type: Journal Articles; Reports - Research
Education Level: Early Childhood Education; Preschool Education
Audience: N/A
Language: English
Sponsor: Institute of Education Sciences (ED)
Authoring Institution: N/A
IES Funded: Yes
Grant or Contract Numbers: R305A150274