ERIC Number: ED657083
Record Type: Non-Journal
Publication Date: 2021-Sep-28
Pages: N/A
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Noncognitive Skills, Policymaking, and Student Long-Run Success: Comparing the Predictive Validity of Multiple Noncognitive Skills
Jing Liu; Megan Kuhfeld; Monica Lee; Danett Song
Society for Research on Educational Effectiveness
Background: Noncognitive skills are a critical component of human capital and are highly consequential for student life outcomes (Heckman et al., 2006; Cunha et al., 2010). While there is a great deal of debates around what noncognitive skills are, their measurement, and their interpretation (Duckworth & Yeager, 2015; Humphries & Kosse, 2017), decades of research in education, psychology, and economics provide an ever-growing evidence that a lot of these skills are highly malleable (Durlak et al., 2011) and can be purposefully nurtured in school. Due to these reasons, education policymakers, practitioners, and parents are seeing a surge of using noncognitive skills as important student outcomes in education systems. The recent passage of the Every Student Succeeds Act (ESSA), which provides states flexibilities to adopt a fifth indicator for their accountability systems to evaluate school performance beyond academic achievement, further promotes the use of non-cognitive outcomes in facilitating education policymaking. While most education policymakers and scholars agree that cultivating noncognitive skills are important for student success, they often find it difficult to decide which specific skills to prioritize and which measures to collect given limited resources. This is understandable, as noncognitive skills often contain many components and there is still no consensus on their scope and definition. One potentially useful criterion for such decisions is the relative importance of different types of noncognitive skills in contributing to student education attainment. For instance, if academic self-efficacy demonstrates a stronger relationship above and beyond other noncognitive skills in predicting college enrollment, then policymakers might consider allocating more resources in collecting data on student self-efficacy and designing interventions and programs to specifically build this skill. Research Question: In this study, we evaluate two sets of widely used measures of student non-cognitive skills simultaneously--observable student behaviors and student self-reported social emotional learning (SEL) skills--by testing their predictive validity of student educational attainment. Our research questions are as follows: (1) How do observed student behaviors correlate with their self-reported SEL skills? (2) How do observed behaviors and self-reported SEL skills predict student education attainment? (3) Do observed student behaviors and self-reported SEL skills predict education attainment differently for racial minority, low-income, and low-performing students? We hypothesize that: 1) school absenteeism and suspensions have positive correlations with self-management skills and social awareness, but not with self-efficacy or growth mindset; and 2) observed behaviors and self-report socio-emotional skills both have predictive power to education attainment, but some skills have stronger predictive power than others. Setting and Sample Characteristics: Our paper uses a rich administrative dataset from a large, urban school district. The sample we examine in our paper consists of two cohorts of students, those who were enrolled in the district as 9th graders in 2015, and those who were enrolled in 2016. Our sample consists of 5782 students, 2878 of whom were enrolled in the district as 9th graders in 2015, and 2904 of whom were enrolled as 9th graders in 2016. Of the full sample, 25% are Latinx, 6% are Black, and 10% of students live in a neighborhood with a poverty rate higher than 25%. Variables, Research Design, and Analysis: There are three types of non-cognitive student skills we evaluate to predict student educational attainment: 1) detailed, course-level attendance data; 2) discipline data; and 3) student responses to the annual SEL survey. The attendance dataset used in this paper allows us to examine absences at a granular level. We are able to observe whether a student misses a single class on a given day or all of his/her assigned classes on a given day, a departure from most prior attendance measures used in research which define an absence as missing a full day of school. Accordingly, we calculate partial and full-day absences for each student and account for both measures in our analyses. We also code excused and unexcused absences separately in addition to calculating partial and full-day absences, to encapsulate how much of the absences a student accrues is within a student's locus of control. We also link our data to two measures of discipline. The first are suspensions, which records the reason for and duration of the suspension. The second is referral data, which consist of student-by-date level records of all occasions when a student is sent to the principal or assistant principal's office. Lastly, we include four constructs of student SEL as a part of our array of student non-cognitive skills: self-efficacy (the belief that a student is capable of achieving a given academic outcome), self-management (the student's belief that one can regulate emotions, thoughts, and behaviors, especially in challenging circumstances), growth mindset (the belief that academic ability is not fixed, but rather grows with effort), and social awareness (the ability to understand norms, empathize with others, and respect others' perspectives). Our outcome variables consist of high school graduation status and various measures to gauge postsecondary attainment, derived from data links to National Student Clearinghouse. In our analysis, we mainly use linear regression and a logit model to predict the impact of noncognitive skills on educational attainment. We control for relevant observable characteristics, including student demographics, lagged test scores, and neighborhood and school characteristics. In our models, we first enter behavioral measures and self-report SEL measures separately. Then we incorporate all of them in the same model to directly compare their relative strengths in predicting various education attainment measures. We will also run a similar strand of models for student subgroups. Findings and Implications: Our preliminary findings suggest that behavioral measures, especially full- and partial-day absences in 9th grade, are the strongest negative predictors for college enrollment. In contrast, once controlling student behaviors, student self-report SEL measures show little if any predictive power to education attainment. Building an information system that allows for tracking detailed student attendance behavior and designing interventions to cultivate school engagement might be a more productive approach to enhance student success, especially for schools and districts that do not have the resources to collect survey based SEL measures.
Descriptors: Human Capital, Predictive Validity, Skill Development, Educational Policy, Cognitive Measurement, Social Emotional Learning, Educational Attainment, Student Behavior, Predictor Variables, Racial Differences, Low Income Students, Urban Schools
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/
Publication Type: Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Grant or Contract Numbers: N/A