Links between achievement, executive functions, and self-regulated learning
Summary
Student self-regulated learning (SRL) is theorized to draw upon cognitive resources such as executive functions (EF) in support of planning, monitoring, and control processes in the service of academic goals. Prior work has demonstrated connections between direct measures of EF and reports of regulation behaviors, but this has not been frequently extended using an SRL framework to classroom behaviors and resulting school achievement. We find relations between inhibition and shifting elements of EF and teacher reports of SRL and links between both and student achievement on standardized tests and classroom grades in mathematics and language arts. We also find that links between EF and math achievement are partially mediated through SRL. Our results suggest that aspects of EF can support or may be a bottleneck for SRL and thus academic achievement, and as such, they have implications for cognitive and educational interventions.
1 INTRODUCTION
Schooling requires a variety of regulatory tasks: Students must be agents of their own learning, formulating plans and directing and managing their internal and external resources (Bandura, 2001; Zimmerman, 1990). Within the education literature, this is often characterized as self-regulated learning (SRL), a process by which students set goals, monitor their progress toward those goals, and adjust as necessary (Pintrich, 2000; Zimmerman, 2000). Self-regulation of learning is complex and cognitively demanding: As learners engage in these processes, their ability to direct and focus attention may be necessary to perform actions such as representing and choosing among competing goals and strategies (Miller & Cohen, 2001). The cognitive skills supporting these processes can be thought of as executive functions (EF): The ability to focus and shift attention, inhibit distractors, and update information in working memory (Miyake, Friedman, Rettinger, Shah, & Hegarty, 2001). EF has itself been found to predict academic achievement (e.g., Blair & Razza, 2007; Duncan et al., 2007); however, the shared associations between EF, SRL, and academic achievement are not well understood. The current paper aims to shed light on these associations within a sample of elementary students and investigates the shared and unique contributions of EF and SRL to academic achievement, testing a mediational hypothesis in which EF operates through SRL to affect achievement.
2 THE LINK BETWEEN EXECUTIVE FUNCTIONS AND ACHIEVEMENT
EF is an essential component of higher thought processes, crucial for the regulation of behavior in complex situations, such as school environments (Diamond, 2013). Many consider EF to be a general cognitive ability that is used across academic domains and into adulthood—one that predicts diverse outcomes such as better mental and physical health (e.g., Blair, 2002; Khurana et al., 2012; Ochsner, Silvers, & Buhle, 2012), school success (e.g., Diamond, 2013; Duncan et al., 2007; St Clair-Thompson & Gathercole, 2006), and participation in less violence and crime (e.g., Hancock, Tapscott, & Hoaken, 2010). Among school-aged children, stronger associations have been found between EF and mathematics than between EF and language arts (e.g., Brock, Rimm-Kaufman, Nathanson, & Grimm, 2009; Gathercole, Pickering, Knight, & Stegmann, 2004), with the EF and language arts association likely to be strongest as children are beginning to read and before reading becomes an automatized process (Blair & Razza, 2007). However, these differences in associations are not universally accepted—a recent meta-analysis found no differences in unadjusted associations between EF and reading and mathematics (Jacob & Parkinson, 2015). In other subjects, such as advanced mathematics, science, or social studies, EF may become more important as children age and schoolwork becomes more complicated, requiring that students manage the demands of multiple tasks in complex activities (Agostino, Johnson, & Pascual-Leone, 2010; Chein & Schneider, 2012; Best, Miller, & Naglieri, 2011; Raghubar, Barnes, & Hecht, 2010).
Although a general three factor structure for EF is often proposed (Miyake et al., 2001), there is evidence that the measurable structure of EF changes across development: starting as a single factor (e.g., Wiebe, Espy, & Charak, 2008) and differentiating as children age (e.g., Lehto, Juujärvi, Kooistra, & Pulkkinen, 2003). Differences in operationalization of EF (by factor structure or specific measures) may influences heterogeneity in associations between EF and achievement (Jacob & Parkinson, 2015). Within mathematics, findings are mixed as to whether inhibition, shifting, or working memory updating predict achievement, individually or together (Bull & Lee, 2014).
Exploring the links between EF and academic performance is a burgeoning area of research; however, studies often focus on young children and/or researcher-administered measures of achievement. The current study contributes to this line of work by relating aspects of EF, namely, inhibition, shifting, and working memory, to four real-world measures of elementary school achievement in both mathematics and English/language arts (ELA). Additionally, we investigate SRL as a potential mechanism through which EF may operate to improve achievement in these domains.
3 THE LINK BETWEEN SRL AND ACHIEVEMENT
Links between SRL and academic achievement have been demonstrated across populations and domains (e.g., Fuchs et al., 2003; Pintrich & de Groot, 1990; Wolters & Pintrich, 1998). However, there are open questions as to whether skill in SRL is domain specific and whether certain SRL behaviors are more beneficial for work in particular academic subjects (Alexander, Dinsmore, Parkinson, & Winters, 2011). Additionally, there are open questions as to when students develop competence in SRL and when and how these competencies translate to learning gains—younger elementary students, especially those below age seven (about second grade), may not have developed all the requisite competencies necessary for SRL (Paris & Newman, 1990; cf. Perry, 1998). There is some evidence that SRL behaviors in elementary school students are associated with achievement (e.g., Howse, Lange, Farran, & Boyles, 2003), and although much of the earlier work framed within an SRL perspective was with older students, there is an increasing focus on the use and improvement of SRL in elementary/primary school children (Dignath & Büttner, 2008; Zeidner, Boekaerts, & Pintrich, 2000). The current study focuses on third-grade students as those who may be at the threshold of SRL development and use (Zimmerman, 1990). As noted in Paris and Newman (1990), this SRL development and use may be limited by competencies underlying SRL, such as accurate self-appraisals. We view EF, in particular, the ability to direct attention and draw on working memory, as one such underlying competency.
4 THE LINK BETWEEN EXECUTIVE FUNCTIONS, SRL, AND SELF-REGULATORY BEHAVIORS
Although SRL research varies as to the specific model of SRL adopted, all models include the elements of planning; regulatory mechanisms, such as monitoring; and evaluation and adjustment (e.g., Efklides, 2011; Pintrich & Zusho, 2002; Winne & Hadwin, 1998; Zimmerman, 2000). The specific model of SRL adopted within this study is that of Pintrich (2000), where learners regulate their cognition, motivation, behaviors, and context in pursuit of their goals. In particular, this study focuses on student regulation of behavior as it is perceived by their classroom teacher—Pintrich's focus on behavior and the cognitive capacities that support and limit student regulation of behavior within a system of SRL is thus complementary.
Implicit in the Pintrich model and other models of SRL are assumptions that certain cognitive processes are necessary for effective regulation. It has been explicitly noted that the processes of SRL heavily tax working memory resources and that limits in such resources may hinder efforts to regulate learning (Pintrich & Zusho, 2002; Winne, 1995). Pintrich and Zusho (2002) point out that working memory resources may directly constrain SRL, but also note that variations in attention and other executive control processes may also influence effective use of SRL. Despite the theoretical assumption that EF underlie SRL processes, there have been few empirical tests of this relationship with elementary-aged children.
Mostly situated outside of the SRL literature, a diverse body of work has focused on the self-regulatory behaviors that can be characterized as self-control (but are also termed behavioral regulation, self-discipline, hot EF, and effortful control; Duckworth, Gendler, & Gross, 2014; Zelazo & Carlson, 2012). These behaviors include the abilities to regulate attention and resist impulses in real-life situations or on laboratory-style behavioral tasks that include an emotional, risk, or reward component. The former is often measured with student surveys or parent/teacher questionnaires (e.g., Behavior Rating Inventory of Executive Function—Gioia, Espy, & Isquith, 2003; Children's Behavior Questionnaire—Putnam & Rothbart, 2006), and the latter is exemplified by tasks such as the delay of gratification task as in Mischel, Ebbesen, and Raskoff Zeiss (1972). Much of this work is done with young children, in preschool or kindergarten, and has shown positive correlations between EF measured as performance-based tasks and self-control measured through teacher or parent reports (e.g., Blair & Razza, 2007) or behavioral tasks (e.g., Brock et al., 2009). Some of this work finds unique associations between EF and achievement and between self-control and achievement (e.g., Waber, Gerber, Turcios, Wagner, & Ford, 2006), but other work fails to find such links, finding instead that self-regulation/hot EF do not predict achievement when considered with performance-based EF tasks (e.g., Brock et al., 2009; Willoughby, Kupersmidt, Voegler-Lee, & Bryant, 2011).
Self-control can be distinguished from SRL. Duckworth and colleagues (2014) describe SRL as a subset of self-control, specifically involving regulation in the academic domain, but they also note that SRL, as traditionally defined (Efklides, 2001; Zimmerman, 2000), can be broader than self-control, as SRL includes strategy generation and use. Zimmerman and Kitsantas (2014) empirically distinguish between self-control and SRL, testing the unique contribution of each to academic achievement. They found that, although correlated, the two aspects of regulation could be differentiated, and that only SRL uniquely predicted achievement.
Both Duckworth and colleagues (2014) and Zimmerman and Kitsantas (2014) recognize commonalities of regulation across contexts, such as metacognition. Recent work in metacognition has found that certain aspects of EF correlate with metacognitive monitoring and control (e.g., Bryce, Whitebread, & Szűcs, 2015; Roebers, Cimeli, Röthlisberger, & Neuenschwander, 2012). Roebers (2017) proposes an integration of EF and metacognition, noting that both play a role in self-regulation and SRL and calls for continued investigation into their links. Most of this research on metacognition has been limited to monitoring and control and may not capture the volitional aspects noted as important to SRL in Duckworth et al. (2014). As one exception, Follmer and Sperling (2016) tested how direct measures of EF related to self-reported SRL in college students, finding that the association was mediated through a self-report measure of metacognition. Other work considering EF and metacognition has not tested such a mediational hypothesis.
Limited prior work has focused on EF and classroom behaviors that likely draw on SRL (but were not directly defined as SRL). Gathercole, Lamont, and Alloway (2006) observed three first grade students identified as having EF (working memory) deficits as they completed normal classroom tasks. The authors noted that the students appeared to struggle with common classroom demands such as remembering instructions and keeping track of their progress. Gathercole et al. later expanded upon this work to develop and test a more classroom-oriented assessment of the central executive (Gathercole, Durling, Evans, Jeffcock, & Stone, 2008). Within their 2008 study, those students who performed poorly on direct measures of working memory also performed poorly on the classroom-oriented measure. Also looking at classroom behaviors, Brock et al. (2009) found moderate associations between measures of EF and a composite of learning-related behaviors consisting of teacher ratings of self-directed learning, distractibility (reversed), work habits, and self-control.
Extending research on EF, self-regulation, and achievement, the current study frames regulation in terms of SRL, a construct that is situated within a line of educational research largely distinct from the self-concept and behavioral regulation work previously linked with EF (Duckworth et al., 2014) and, as supported in Zimmerman and Kitsantas (2014), is likely more related to achievement. Other recent work has begun to explore EF within an SRL framework. Namely, Dias and Seabra (2017) found that an intervention in first grade focusing on EF could improve both achievement and some SRL behaviors reported by the students' second grade teachers. Cirino et al. (2017) also reported on an EF intervention with fourth graders, but one that did not appear to have effects on SRL. They noted generally weak relations between their measures of EF and SRL, calling for more work to elucidate these relations among school-aged children, especially in light of the large focus of prior work on preschoolers (p. 15).
Regardless of the framework, few studies have directly tested a mediational hypothesis in which EF operates through regulation to affect achievement. As an exception, Brock et al. (2009) tested such a hypothesis with their sample of kindergarteners, but concluded that learning-related behaviors did not account for the relation between an amalgamated measure of EF and a researcher-administered measure of achievement. Although their study provided valuable information on the relations between EF and achievement in kindergarteners, expansion to investigation of these associations with classroom measures of achievement may lend ecological validity. Additionally, these associations are likely to change as children age—the young age of the students in Brock et al. (2009) may have limited their opportunity to engage in regulatory behavior in the classroom or changed the nature of the associations between EF and academic performance (Best, Miller, & Jones, 2009; Gathercole et al., 2008). Two studies with older students may be more instructive: Gardner-Neblett, DeCoster, and Hamre (2014) used self-reliance as their measure of SRL-like skills. They found that self-reliance mediated the association between early childhood attention and adolescent performance on a researcher-administered standardized test for math but not language. Neuenschwander, Röthlisberger, Cimeli, and Roebers (2012) tested whether EF operated through learning-related behaviors to improve achievement on standardized tests and grades in a sample of Swiss kindergarten and young elementary students. The authors found that EF's association with grades was partially mediated through learning-related behaviors, whereas the association between EF and test scores did not operate through learning-related behaviors. However, this specific pattern of findings may have been due to measurement: The authors used a researcher-administered standardized test, 1 not directly relevant to the classroom behaviors on which the students were assessed.
The current study builds on prior work both within and outside of the SRL framework to better understand how EF and achievement may be linked through SRL in elementary school-aged children. As achievement outcomes, we use both classroom grades and state-administered standardized tests with real consequences for students and teachers. Additionally, we examine the separate contributions of two components of EF distinguishable within early elementary school: inhibition/shifting and working memory (see Lee, Bull, & Ho, 2013). Specifically, we ask, Does SRL mediate the association between EF and achievement? and Do these associations vary depending on domain of achievement and the choice of measure for both EF and achievement?
In considering the conceptual overlap between EF and SRL, we expect moderate associations between the measures of EF and SRL. The students in our study are in third grade—a time when reading instruction shifts from learning to read to reading to learn (Hernandez, 2011). Therefore, students may not rely as heavily on EF when completing reading tasks (see Blair & Razza, 2007). Given this time-period, we expect that, among our sample, the EF-mathematics link may be stronger than the EF-ELA link. Regarding the mediational question, we hypothesize that the older age of our students and more academically relevant measure of achievement will result in mediational associations for both grades and standardized tests in contrast to the previous research on kindergarteners or using researcher-created achievement tests.
5 METHOD
5.1 Sample and procedure
The sample for this study comes from a larger project on the effectiveness of a digital mathematics program within 52 Southern California schools (see Rutherford et al., 2014, for details on the evaluation project). All students in grades two, three, and five within 18 of the project schools were sent consent forms asking for parent permission to engage in individual testing. Grade four was excluded because, due to the project design, there was no treatment/control variation within this grade and a major goal of individual testing was evaluation related. The regional school district handled consent procedures and provided the research team with rosters of all students with consent. Although the exact response rate is unknown, the regional district provided an estimated response rate of approximately 60%. The provided rosters were randomly ordered within each classroom to establish a testing order.
In the spring of 2011, research teams of three to five trained undergraduate and graduate students visited participating elementary schools for two days of testing. Teams were able to test between six and 12 students at a time using individual Windows-based netbook computers setup in a school library or empty classroom. The students were pulled from class according to the random order assigned prior to the school visit. At prearranged times, students were escorted from class as a group to the testing room. Written assent was obtained, and students completed the netbook assessments. All computer testing was conducted in English. Although a number of the participants were classified as English language learners (ELL) by their districts, all participating students were able to converse in English and read and understand the assent form and testing instructions. Nonetheless, we added ELL status as a covariate in the analyses (see below).
The following school year, a smaller subsample was randomly selected among the third graders who had testing data from their second grade year. Teacher ratings of SRL behaviors and teacher-reported classroom grades were collected for this subsample, which was limited in size so as to not place an undue burden on project teachers. The current study reports on this subsample and includes 211 third graders across 45 teachers and 18 schools. This sample is 49% male, 82% Hispanic, 52% ELL, and 79% eligible for the national free/reduced lunch program. At the time of state testing for the study posttest, the average age was 9.04 years. The sample demographics reflect those within the study schools, which were 52% male, 85% Hispanic, 56% ELL, and 81% eligible for free/reduced lunch. Descriptive statistics for the study variables are reported in Table 1 and compare across the current study sample and the larger project samples.
| Mean | SD | Min | Max | Count | |
|---|---|---|---|---|---|
| Total sample in 18 project schools | |||||
| 2nd grade math CST | 378.86 | 81.22 | 150.00 | 600.00 | 1,289 |
| 2nd grade ELA CST | 348.82 | 60.87 | 198.00 | 600.00 | 1,294 |
| 3rd grade math CST | 392.89 | 80.88 | 150.00 | 600.00 | 1,448 |
| 3rd grade ELA CST | 333.02 | 58.83 | 153.00 | 600.00 | 1,448 |
| Individually tested students within 18 schools | |||||
| 2nd grade math CST | 391.67a | 77.84 | 197.00 | 600.00 | 331 |
| 2nd grade ELA CST | 357.62a | 58.65 | 217.00 | 525.00 | 332 |
| 3rd grade math CST | 403.41a | 76.89 | 211.00 | 600.00 | 352 |
| 3rd grade ELA CST | 340.13a | 54.76 | 191.00 | 468.00 | 352 |
| Hearts and Flowers | 0.73 | 0.19 | 0.00 | 1.00 | 350 |
| Backward digit span | 3.37 | 0.57 | 3.00 | 5.00 | 304 |
| The study sample: individually tested with SRL | |||||
| 2nd grade math CST | 396.47a | 77.86 | 208.00 | 600.00 | 211 |
| 2nd grade ELA CST | 363.49a | 58.35 | 234.00 | 525.00 | 211 |
| 3rd grade math CST | 402.27a | 76.11 | 215.00 | 600.00 | 211 |
| 3rd grade ELA CST | 344.03a, b | 57.42 | 191.00 | 468.00 | 211 |
| Hearts and Flowers | 0.74 | 0.19 | 0.20 | 1.00 | 211 |
| Backward digit span | 3.34 | 0.55 | 3.00 | 5.00 | 211 |
| SRL | 3.30 | 0.66 | 1.00 | 4.00 | 211 |
| The study sample with teacher grade data | |||||
| 2nd grade math CST | 395.23a | 72.53 | 208.00 | 600.00 | 153 |
| 2nd grade ELA CST | 366.47a | 60.04 | 234.00 | 525.00 | 153 |
| 3rd grade math CST | 399.15 | 76.10 | 215.00 | 600.00 | 153 |
| 3rd grade ELA CST | 344.22a | 59.55 | 191.00 | 468.00 | 153 |
| Hearts and Flowers | 0.74 | 0.19 | 0.20 | 1.00 | 153 |
| Backward digit span | 3.32 | 0.52 | 3.00 | 5.00 | 153 |
| SRL | 3.28 | 0.67 | 1.00 | 4.00 | 153 |
| Math grade | 7.97 | 2.72 | 1.00 | 11.00 | 153 |
| ELA grade | 7.63 | 2.86 | 0.00 | 11.00 | 153 |
- Note. Count reflects missing data. Mean standardized test scores place students above the state 350-point cutoff for proficiency (CA DOE, 2010); mean math and ELA grades of students in the sample were between a B minus and a B. SRL: self-regulated learning; CST: California Standards Test; ELA: English/language arts.
- a Differs at the p < 0.05 level from the total sample.
- b Differs at the p < 0.05 level from the individually tested sample. There were no differences at the p < 0.05 level between the three individually tested samples on Hearts & Flowers, Backward digit span, or SRL.
5.2 Measures
5.2.1 Executive functions
In light of the factor structure identified with this age group (Lee et al., 2013), EF was assessed with two tasks: one representing both inhibitory control and cognitive flexibility and one representing working memory. We view these tasks as measures of cold EF or basic EF that can support more emotionally loaded executive tasks, such as those represented by SRL (see Dawson & Guare, 2018). The former is typically measured with performance-based tasks (e.g., Go/No-Go; Trommer, Hoeppner, Lorber, & Armstrong, 1988), and the latter is often measured with self- or other-report instruments (e.g., the BRIEF; Gioia et al., 2003).
Inhibitory control and cognitive flexibility
The “Hearts & Flowers” task is a measure that requires inhibitory control and cognitive flexibility in the context of a task with shifting rules (Davidson, Amso, Anderson, & Diamond, 2006). It has been used in previous research as a singular measure or in concert with other measures of EF to predict later academic outcomes in math (Oberle & Schonert-Reichl, 2013) and reading (Van de Sande, Segers, & Verhoeven, 2013), or as an outcome measure in the study of ADHD treatment efficacy (Green et al., 2011).
Within the Hearts & Flowers task, participants are first introduced to the “hearts” rule, where they must push a button on the same side of the screen as the heart appears. Students train for up to three blocks of four trials each, moving on to the task trials once they get all trials in a block correct or exhaust all three blocks. Each trial consists of a fixation cross shown for 500 ms, followed by a blank screen of 500 ms. The next image is the stimulus (a heart or a flower), which remains on screen until the participant makes a response, after which a blank screen is shown for 500 ms before a new trial is begun with a fixation cross. After 12 trials of the hearts rule, participants are introduced to the “flowers” rule, where they must push a button on the opposite side from that on which the flower appears. They complete a training session as with the “hearts” rule, along with 12 task trials. The final 30 trials of the task are presented as the “hearts and flowers game,” where participants must switch between rules. The trials are randomly determined; on average, 52% of all trials were switch trials. Accuracy is calculated as the average percent correct on trials that switched from one stimulus to another (α = 0.82); this accuracy serves as the dependent variable. Within this sample, performance on Hearts & Flowers is comparable with that in Davidson et al. (2006).
Working memory
Backward digit span was used as a measure of working memory and is commonly used with this age group (e.g., Bull, Espy, & Wiebe, 2008; Gathercole & Pickering, 2000a; Gathercole & Pickering, 2000b; Passolunghi & Siegel, 2004). Within backward digit span, participants are presented with a series of numbers and asked to recall these numbers in the reverse order from that which they were presented. The versions used within this study presented each digit for 1,000 ms, centered on the computer screen before presenting a blank screen for 250 ms and then the next digit. When all digits in a trial were shown, participants were prompted to enter the number string, in reverse order, using the computer's keypad and to strike the <ENTER> key when they were finished. A training period ensured each participant could successfully enter strings of three digits before the task trials began. Two trials at each digit string length were presented. After two successful trials with the same string length, the string length of the next two trials was increased by one. The task ended when participants failed both trials of a given length.
A student's span score for each task was the highest number of digits reached before the cutoff. It should be noted that participants completed forward digit span before the backward digit span measure used within this study. Given that we were interested in updating of working memory and not just simple serial recall, we focused on backward digit span only (see Gathercole & Pickering, 2000a). Among this age group, the test–retest reliability of backward digit span after a year interval was 0.33. Although this is lower than reported test–retest reliability of similar measures, such as r = 0.41 attained with the backward digit span measure used in Ramani, Jaeggi, Daubert, and Buschkuehl (2017), the current testing interval is considerably longer than the approximately 2 months from the Ramani et al. study.
5.2.2 Self-regulated learning behaviors
SRL was assessed in the beginning of third grade with teacher responses to four 4-point Likert scale questions on student behavior (Table 2; Farkas, 1996). These questions were selected as those that tap into behaviors associated with SRL: effort, planning and organization, and attentiveness (Zimmerman, 1990; Zimmerman & Martinez-Pons, 1988) and were averaged to create a single scale for SRL for each student (α = 0.85). Similar teacher ratings of SRL have been previously used in school-based research (e.g., Cleary & Kitsantas, 2017; Dias & Seabra, 2017).
| Question text | Area of SRL | |
|---|---|---|
| (1) | This student is very well organized in class | Planning & organization |
| (2) | When I explain new material, this student listens carefully | Attentiveness |
| (3) | In my class, this student works as hard as he/she can | Effort |
| (4) | This student almost always turns in homework on time | Planning & organization; effort |
- Note. Responses are given on a 4-point Likert scale, where 1 = Not at all true and 4 = Very true. SRL: self-regulated learning.
5.2.3 California Standards Tests (CSTs)
The CSTs were administered to all students in grades two through 11 attending California public schools in the spring of each year from 1998 through 2013; second through fourth grade CSTs focused on math and ELA (California Department of Education, 2014). Scale scores calculated to allow comparison across grades and years ranged from 150 to 600, with a score of 350 marking the state's determined level of proficiency (CA Board of Education, 2010). The mathematics CST examined five strands within the California State Standards for mathematics: Number Sense I; Number Sense II; Algebra and Functions; Measurement and Geometry; and Statistics, Data Analysis, and Probability. The ELA CST focused on reading and writing. If a student was classified as English Language Learner, they took the ELA CST in English; however, additional testing may also have also been required—only the test in English was available for the current study.
5.2.4 Classroom grades
Teachers were asked to report end-of-year classroom grades in math and ELA for the students for whom they rated SRL behaviors. Teachers were provided with a scale from F to A at half-point increments. The same teacher reported on each student's math and ELA grades. The letter grades were coded as numbers on a scale from zero (F) to 11 (A). Spring response rate was not as high for teacher-reported grades as it was in the fall for SRL; therefore, the sample for the grade analysis is limited to only 153 students—this sample does not differ from the main study sample in demographic characteristics (see Table 1).
5.2.5 Control variables
Demographic information on student gender, ELL status, and eligibility for the federal free/reduced lunch program were provided by the school districts along with CST scores. Demographic information as reported in second grade is included in the analyses as covariates; individual math and ELA CST scores from second grade were used as pretest measures.
6 ANALYSIS
As an initial step, zero-order correlations were calculated, and the intraclass correlations (ICCs) between outcomes and class group were examined. Two-level hierarchical linear models were estimated to account for nesting of students within classes. Third grade teacher was chosen as the nesting variable because of the focus on both teacher reports of SRL and achievement in this year. Separate models were run for each of the two representations of EF, control/flexibility and working memory. For each outcome, paths were estimated from EF to SRL (Path A), from EF to the outcome without the SRL mediator (Path C), and from EF to the outcome including SRL (Paths C’ and B; Baron & Kenny, 1986). Each model included the relevant pretest (math or ELA) along with demographic covariates (gender, ELL status, eligibility for the free/reduced lunch program). To test the statistical significance of the indirect pathway, bootstrapped tests of mediation (5,000 iterations) were conducted with the ml_mediation algorithm within Stata 14 (StataCorp, 2015; Krull & MacKinnon, 2012).
7 RESULTS
Zero-order correlations for all included variables are provided in Table 3. ICCs differed between measures and samples. For SRL, most variance was within classes, with an ICC of 0.08 for the full sample and 0.02 for the grade analysis sample. ICCs for math CST were 0.37 for the full sample and 0.43 for the reduced sample; for ELA CST, ICCs were 0.45 for both the full and reduced samples. For math grades, the ICC was 0.14; for ELA grades, the ICC was 0.16. This variation suggests that there were more class-to-class differences in standardized test scores than there were for grades and teacher ratings of SRL. The addition of EF and covariates resulted in reduction of variance at each level—variance explained for each model is reported at the bottom of Tables 4 and 5.
| (1) | (2) | (3) | (4) | (5) | (6) | (7) | (8) | ||
|---|---|---|---|---|---|---|---|---|---|
| (1) | 2nd Gr ELA CST | 1 | |||||||
| (2) | 2nd Gr math CST | 0.74 | 1 | ||||||
| (3) | Hearts & Flowers | 0.44 | 0.35 | 1 | |||||
| (4) | Back digit span | 0.24 | 0.18* | 0.21* | 1 | ||||
| (5) | SRL | 0.39 | 0.38 | 0.32 | 0.05*** | 1 | |||
| (6) | 3rd Gr ELA CST | 0.77 | 0.58 | 0.38 | 0.24 | 0.36 | 1 | ||
| (7) | 3rd Gr math CST | 0.68 | 0.67 | 0.43 | 0.17** | 0.42 | 0.71 | 1 | |
| (8) | 3rd Gr ELA grade | 0.71 | 0.60 | 0.38 | 0.11*** | 0.51 | 0.71 | 0.68 | 1 |
| (9) | 3rd Gr math grade | 0.61 | 0.60 | 0.39 | 0.05*** | 0.54 | 0.62 | 0.70 | 0.83 |
- Note. N = 211 except for correlations with ELA and Math grades, where N = 153. All correlations are statistically significant at the p < 0.001 level except. SRL: self-regulated learning; CST: California Standards Test; ELA: English/language arts; EF: executive functions.
- * p < 0.01.
- ** p < 0.05.
- *** p > 0.05.
| N = 211 | (1) | (2) | (3) | (4) | (5) | (6) |
|---|---|---|---|---|---|---|
| Outcome: | SRL | Math CST | Math CST | SRL | ELA CST | ELA CST |
| H&F | 0.74** (0.22) | 73.78*** (20.08) | 56.81** (19.68) | 0.65** (0.23) | 10.49 (13.53) | 4.31 (13.49) |
| SRL | 23.33*** (5.76) | 9.50* (3.84) | ||||
| Pretest | 0.003*** (0.001) | 0.54*** (0.06) | 0.48*** (0.06) | 0.004*** (0.001) | 0.62*** (0.05) | 0.58*** (0.06) |
| Intercept | 1.92*** (0.27) | 144.78*** (25.56) | 96.09*** (27.34) | 1.64*** (0.34) | 127.90*** (21.17) | 112.39*** (21.84) |
| Random parameters | ||||||
| Between | 7.46e-23 (4.67e-22) | 640.54 (293.05) | 720.53 (293.23) | 0.001 (0.01) | 273.62 (107.22) | 306.63 (112.14) |
| Residual | 0.31 (0.03) | 2164.28 (245.65) | 1957.62 (220.78) | 0.32 (0.03) | 920.94 (100.64) | 878.57 (96.01) |
| Explained variance | ||||||
| L2 | >0.999 | 0.70 | 0.67 | 0.98 | 0.81 | 0.78 |
| L1 | 0.22 | 0.41 | 0.47 | 0.20 | 0.48 | 0.50 |
| N = 153 | (7) | (8) | (9) | (10) | (11) | (12) |
|---|---|---|---|---|---|---|
| Outcome: | SRL | Math Grade | Math Grade | SRL | ELA Grade | ELA Grade |
| H&F | 0.76** (0.25) | 3.25*** (0.92) | 1.92* (0.85) | 0.61* (0.26) | 2.17* (0.90) | 1.34 (0.83) |
| SRL | 1.59*** (0.25) | 1.47*** (0.25) | ||||
| Pretest | 0.003*** (0.001) | 0.02*** (0.002) | 0.01*** (0.003) | 0.004*** (0.001) | 0.03*** (0.004) | 0.02*** (0.003) |
| Intercept | 2.05*** (0.35) | −0.68 (1.28) | −4.00** (1.26) | 1.62*** (0.42) | −3.46* (1.45) | −5.70*** (1.36) |
| Random parameters | ||||||
| Between | 1.52e-24 (1.31e-23) | 0.28 (0.36) | 0.48 (0.36) | 1.96e-25 (1.41e-24) | 0.15 (0.22) | 0.062 (0.17) |
| Residual | 0.32 (0.04) | 3.92 (0.53) | 2.96 (0.40) | 0.32 (0.04) | 3.57 (0.45) | 2.97 (0.38) |
| Explained variance | ||||||
| L2 | >0.999 | 0.73 | 0.55 | >0.999 | 0.89 | 0.95 |
| L1 | 0.26 | 0.38 | 0.53 | 0.27 | 0.48 | 0.57 |
- Note. Unstandardized regression coefficients. Standard errors in parentheses. Covariates included in model but omitted from table are dummy variables for whether male, English language learner, or eligible for free/reduced lunch. Pretest is same-subject pretest CST score from the end of second grade.
- * p < 0.05.
- ** p < 0.01.
- *** p < 0.001.
| N = 211 | (1) | (2) | (3) | (4) | (5) | (6) |
|---|---|---|---|---|---|---|
| Outcome: | SRL | Math CST | Math CST | SRL | ELA CST | ELA CST |
| BDS | −0.01 (0.07) | 0.10 (6.53) | 1.20 (6.20) | −0.05 (0.08) | 2.53 (4.20) | 3.13 (4.13) |
| SRL | 26.56*** (5.73) | 9.90** (3.80) | ||||
| Pretest | 0.003*** (0.001) | 0.60*** (0.05) | 0.52*** (0.05) | 0.001*** (0.001) | 0.63*** (0.05) | 0.58*** (0.05) |
| Intercept | 2.27*** (0.34) | 172.26*** (32.24) | 106.75** (33.82) | 1.83*** (0.40) | 122.78*** (23.77) | 104.16*** (24.55) |
| Random parameters | ||||||
| Between | 1.56e-14 (9.18e-14) | 846.59 (342.12) | 840.46 (319.39) | 0.01 (0.02) | 274.75 (107.58) | 301.10 (110.61) |
| Residual | 0.33 (0.03) | 2231.11 (251.98) | 1998.80 (224.43) | 0.33 (0.04) | 921.61 (100.73) | 878.95 (95.95) |
| Explained variance | ||||||
| L2 | >0.999 | 0.61 | 0.61 | 0.81 | 0.81 | 0.79 |
| L1 | 0.18 | 0.39 | 0.45 | 0.19 | 0.48 | 0.50 |
| N = 153 | (7) | (8) | (9) | (10) | (11) | (12) |
|---|---|---|---|---|---|---|
| Outcome: | SRL | Math Grade | Math Grade | SRL | ELA Grade | ELA Grade |
| BDS | −0.07 (0.09) | −0.25 (0.33) | −0.08 (0.29) | −0.13 (0.09) | −0.26 (0.31) | −0.04 (0.28) |
| SRL | 1.70*** (0.25) | 1.52*** (0.25) | ||||
| Pretest | 0.003*** (0.001) | 0.02*** (0.003) | 0.02*** (0.003) | 0.005*** (0.001) | 0.03*** (0.003) | 0.02*** (0.003) |
| Intercept | 2.57*** (0.44) | 1.22 (1.62) | −3.28* (1.57) | 1.99*** (0.47) | −2.66 (1.66) | −5.58*** (1.56) |
| Random parameters | ||||||
| Between | 1.82e-20 (1.79e-16) | 0.67 (0.45) | 0.64 (0.39) | 1.54e-23 (1.03e-22) | 0.26 (0.25) | 0.11 (0.18) |
| Residual | 0.34 (0.04) | 3.95 (0.52) | 2.97 (0.40) | 0.32 (0.04) | 3.60 (0.45) | 2.98 (0.37) |
| Explained variance | ||||||
| L2 | >0.999 | 0.37 | 0.40 | >0.999 | 0.80 | 0.91 |
| L1 | 0.22 | 0.37 | 0.53 | 0.26 | 0.47 | 0.56 |
- Note. Unstandardized regression coefficients. Standard errors in parentheses. Covariates included in model but omitted from table are dummy variables for whether male, English language learner, or eligible for free/reduced lunch. Pretest is same-subject pretest CST score from the end of second grade.
- * p < 0.05.
- ** p < 0.01.
- *** p < 0.001.
Regression results as unstandardized coefficients are also reported in Tables 4 and 5. We turn first to the models including EF measured with Hearts & Flowers as a predictor (Table 4). Standardized effect sizes for these associations are reported in Figure 1 and were calculated using the relevant level-specific standard deviation for each variable using the formula: (B*SDX)/SDY. In text, these are referred to as β. Hearts & Flowers was a statistically significant predictor of SRL across all models—a one standard deviation (SD) increase was associated with approximately a fifth of a SD increase in SRL, depending on model. Hearts & Flowers was also a statistically significant predictor of math CST scores and both math and ELA grades, with total effects ranging from 0.14 SDs to 0.23 SDs. Hearts & Flowers was not a statistically significant predictor of ELA CST scores (p = 0.45). Effect sizes for Hearts & Flowers were reduced with the addition of SRL to the models; association between Hearts & Flowers and math test scores was reduced by 23%, association with math grades was reduced by 39%, and association with ELA grades was reduced by 40%. Indirect effects of Hearts & Flowers through SRL were small, less than 0.10 SD, and only statistically significant for math CST (p = 0.02) and math grades (p = 0.02). Direct effects for Hearts & Flowers also only remained statistically significant for the two math outcomes (β = 0.14, p = 0.02 for CST scores; β = 0.13, p = 0.01 for math grades). SRL had its own statistically significant direct path to achievement for all four outcomes, ranging from 0.11 SDs for ELA test scores to 0.39 SDs for math grades.
Despite some statistically significant zero-order correlations between backward digit span and achievement outcomes, once covariates were accounted for, these associations did not attain statistical significance (all effect sizes less than +/−0.06, ps > 0.05) nor did the association between digit span and SRL (all effect sizes less than +/−0.10, ps > 0.05; see Table 5). -12 Therefore, mediation was not further explored.
8 DISCUSSION
Although correlational, our results are consistent with the idea that the inhibition and shifting elements of EF operate on academic achievement partially through SRL. As students tackle academic assignments, they must martial their executive resources to pay attention, plan, and organize, often manifested in classroom behaviors such as turning homework in on time or listening attentively. These processes are cognitively demanding and require the kinds of basic skills implicated by executive functions. Within our sample, students who scored one SD higher on Hearts & Flowers (measuring inhibition and shifting aspects of EF) scored a fifth of a SD higher on teacher reports of SRL behaviors. This link between EF and SRL was in line with our predictions and has been suggested in previous SRL research (e.g., Avery & Smillie, 2012; Winne & Nesbit, 2010). Results from our study extend to SRL the finding of links between EF and self-control or learning-related behaviors (e.g., Alloway, Gathercole, Kirkwood, & Elliott, 2009; Blair & Razza, 2007; Neuenschwander et al., 2012). Although working memory is the most frequently mentioned element of EF in descriptions of SRL (e.g., Pintrich & Zusho, 2002; Winne, 1995), it is of note that we only found these associations with our inhibition/flexibility measure and not our working memory measure.
We also replicated the previously found link between direct measures of EF and academic achievement (e.g., Blair & Razza, 2007; Brock et al., 2009; Duncan et al., 2007; St Clair-Thompson & Gathercole, 2006), although, as noted, associations between achievement and working memory, measured in our data with backwards digit span, disappeared once a pretest covariate was entered in the models. This may be reflective of the skills needed by our specific measures of achievement. This is supported by recent work by Purpura, Schmitt, and Ganley (2017) who found that specific components of EF were associated with specific academic skills. For example, working memory was related to only one specific math skill, whereas inhibition was more broadly related to most math subcomponents. The strength of our achievement measure in its educational relevance is offset by its general nature, which may have prevented us from finding these specific associations. Replication with different measures of both working memory and achievement are needed before conclusions can be drawn regarding which element of EF associates more with different measures of achievement.
Examining Hearts & Flowers, the total associations between this measure of EF and three of our achievement measures were, on average, a fifth of an SD. These effect sizes are of practical significance to education research—the 0.18 effect size for the association between EF and math test scores is 35% of the average annual growth in third grade mathematics (see Hill, Bloom, Black, & Lipsey, 2008). Although we found associations between Hearts & Flowers and both math measures, of our ELA measures, only classroom ELA grades were predicted by this measure of inhibition/shifting. Zero-order correlations between Hearts & Flowers and third grade ELA CST were similar to other achievement measures; however, this association was no longer statistically significant once second grade ELA test scores were controlled. This may be due to greater stability in the ELA test than the math test—correlations between second and third grade ELA tests were 0.77 as compared with math tests at 0.67 (see Table 3). Alternatively, it may be that skills measured by the ELA CST are those in which participating students have gained sufficient expertise as to make EF less important. As students gain greater facility and reading becomes automatic, they may need to rely less on executive processes (see Pintrich & Zusho, 2002). The finding that EF is more strongly associated with math than ELA is also supported in prior work (e.g., Brock et al., 2009; Fuhs, Nesbitt, Farran, & Dong, 2014). It should be noted that, whereas EF, as assessed with either Hearts & Flowers or backwards digit span, was not a statistically significant predictor of ELA CST scores in our model, SRL behaviors did contribute to ELA CST scores, albeit to a lesser extent than in the models of other achievement outcomes.
Turning to the mediation results, the associations between EF, SRL, and achievement outcomes studied herein present an indication as to how EF may be operating to increase achievement. Teacher's perceptions of regulatory behaviors were represented in our measure of SRL and accounted for 23–40% of the association between EF (inhibition/shifting) and measures of academic achievement with statistically significant indirect effects found for both our math measures. Our finding that SRL mediates the association between EF and achievement on a standardized test is in contrast to Brock et al. (2009) and Neuenschwander et al. (2012), although differences in the nature of the tests may explain this result. As a state-created standardized test, the CSTs are designed to be highly related to the content taught within the classroom (or the classroom is designed to be related to the content of the test)—it seems logical that engaging in self-regulation while learning this content would result in higher performance on such a test and might not result in higher performance on researcher-selected and administered tests as in Brock et al. (2009) and Neuenschwander et al. (2012). Our failure to find an indirect effect linking our association between Hearts & Flowers and ELA achievement is in line with the results of Garner-Neblett and colleagues (2014). We cannot, however, offer the same justification as they do. Their sample was ninth graders, and they noted that middle and high school math likely both implicated more executive processes in problem solving than did language and also required greater self-reliance due to the decrease in motivation for math typically seen in that age group. Some of this may already be at play with our third graders, but we do not have evidence regarding differences in the complexity of the tasks between math and ELA nor of any decreased student math motivation.
8.1 Limitations
As noted, specific associations between EF and achievement may be closely tied to the operationalization of each. Our use of single measures to represent inhibition/shifting and working memory may have limited our ability to identify specific associations. In particular, our backward digit span measure had limited range and lower-than-expected test–retest reliability. Additionally, although CSTs and grades present authentic policy-relevant measures of student math and ELA achievement, they do not allow finer-grained analysis regarding which specific skills may be related to EF and SRL. Our teacher report measure of SRL behaviors is also limited—it is possible that either student self-reports or measures such as trace data (see Azevedo et al., 2013) would associate differently with EF and achievement. It is also worth noting that the multilingual nature of the majority of our participants may impact their executive functioning, as there is some evidence, albeit not uncontested, for a bilingual advantage in EF (Bialystok & Viswanathan, 2009); however, in our data, there was no positive correlation between bilingual status and EF—the correlation between ELL and both measures was negative, although only attaining statistical significance with Hearts & Flowers, r(209) = −0.15, p = 0.03. With respect to our largely Hispanic and lower socioeconomic status sample, we view this less as a limitation and more as an important strength—this work may be generalizable to populations not typically covered in the overwhelmingly North American, White, and middle-class—“WEIRD” populations typically studied in such research (see Henrich, Heine, & Norenzayan, 2010). Lastly, the age of our students may have impacted the results. Although our sample of third graders may have more opportunity to engage in SRL than the kindergarteners in Brock et al. (2009), it is conceivable that the amount of mediation found presents a lower-bound limit for the formal schooling years. The young students in our study may not have as many opportunities to self-direct their learning as do older students in middle/high school and college—it is possible that deficits in EF that work through SRL have stronger effects on concurrent achievement among older students.
8.2 Conclusion and implications
Our results suggest that EF is related to SRL and that this relation explains part of the association between EF and math achievement. Although we did not find a similar association between EF, SRL, and ELA achievement, this may be due to limitations in our measurement. Indirect effects of EF on math achievement were small, less than one tenth of a standard deviation, but it is possible that these effects will compound as the children age. Improved focus and attention from higher EF may lead to greater opportunities for learning—within our study teachers reported that higher EF students were more attentive and organized in class—the resulting opportunities to engage with class material and activities may produce subsequently stronger effects on achievement as children with greater focus and learning regulation progress, whereas their peers with poor regulation skills fall further behind.
Although this study is strictly correlational, the timing of our measures lends support to our hypothesized causal ordering, suggesting that successful SRL is based on better EF functioning. Future studies investigating changes in EF and SRL are required to further support our proposed model. Previous research has indicated that EF is malleable (e.g., Jaeggi, Buschkuehl, Jonides, & Shah, 2011; Neville et al., 2013; Ramani et al., 2017); however, a theory of the underlying mechanisms for any generalizing effects is a matter of extant debate. Our results suggest that improvements in EF may boost achievement via enhancement of SRL. Likewise, interventions to improve SRL (see Dignath & Büttner, 2008) may not reach their full potential if EF limits SRL, possibly as a bottleneck. Our results present some support for the hypothesis that EF influences SRL and suggest that further exploring their joint effect on achievement may be a promising area for future research.
ACKNOWLEDGEMENTS
This research was supported in part by grants from the Institute of Education Sciences to the University of California, Irvine (Grant R305A090527).
In addition, we would like to thank the participating districts, schools, teachers, and students, as well as our project team of personnel at the UCI School of Education, and our lab graduate and undergraduate students.
MB is employed at the MIND Research Institute, whose interest is related to this work, and SMJ has an indirect financial interest in the MIND Research Institute.
