NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: ED663009
Record Type: Non-Journal
Publication Date: 2024-Sep-20
Pages: N/A
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
The Development of a Social-Emotional Competence and Well-Being Measure for Educators
Jingtong Pan; Kimberly Kendziora; Christina LiCalsi; Karthik Ramesh; George Stifel
Society for Research on Educational Effectiveness
Background/Context: Research consistently emphasizes the importance of social-emotional learning (SEL) in education settings (Cipriano et al., 2023; Wigelsworth et al., 2023). In addition, it has become increasingly evident that educators' social-emotional competence and well-being plays a crucial role in fostering SEL among students (Braun et al., 2020; Gimbert et al., 2023). However, existing social-emotional assessment tools primarily focus on evaluating students' social-emotional skills, neglecting the essential dimension of educators' social-emotional competence and well-being. The few existing measures (Friedman & Kass, 2002; Kirk et al., 2008; Yoder, 2014) either are lengthy and time-consuming or provide a narrow perspective of educators' well-being (for a summary, see Hemi & Kasperski, 2023). Furthermore, there has been little research on the psychometric properties of existing measures of educators' social-emotional competence and well-being (Hemi & Kasperski, 2023). A lack of evidence on the measures, such as the reliability, validity, and applicability across diverse backgrounds of educators, can impact researchers' overall findings on educators' social-emotional competence. Building on an educator well-being framework, this study seeks to develop a multidimensional self-report instrument that measures educators' socialemotional competencies. Purpose: This study investigates the psychometric properties of a newly developed measure administered to a diverse group of staff members across 93 schools in Illinois. The study contributes to the research on SEL by developing a social-emotional competence survey for educators and examining the psychometric properties of the measure, thus providing the research community with a reliable and valid tool to enhance understanding of social-emotional competence and well-being across a diverse education workforce. Setting: The research--which is part of a larger impact study funded by the Institute of Education Sciences (IES) and conducted by the American Institutes for Research® (AIR®) in partnership with the Illinois State Board of Education (ISBE)--evaluates Illinois's Learning Renewal--Social-Emotional Learning (LR-SEL) programs. LR-SEL is a statewide pandemic recovery initiative to counter the critical effects of the pandemic by supporting SEL and mental well-being across the state's educational system. Information collected from this study will help ISBE, other state education agencies, and the broader education research, policy, and practice communities better understand the effects, implementation, and costs of the LR-SEL programs, with the goal of improving the ongoing social-emotional supports for districts and schools. Research Design and Participants: As part of the larger study, the research team developed an educator social-emotional competence and well-being measure comprising six scales: school working conditions (four items), learning renewal-SEL practices (four items), emotional intelligence (six items), capacity to support students (four items), general well-being (eight items), and professional well-being (four items). The conceptual framework for the survey was adapted from recent work for the Council of Chief State School Officers on educators' well-being (Yoder et al., 2023). Most of the items were adopted from prior SEL studies in Illinois (e.g., Li et al., 2023). The research team collected survey responses from 2,927 staff members in 93 public schools in Illinois between March and June 2023. The staff survey sample has a demographic makeup that closely mirrors the gender and racial diversity of educators across the state. In addition, the study team conducted follow-up cognitive interviews with nine educators to further understand their perspectives of the survey items. Data Analysis: The research team conducted Rasch analyses and multilevel multinomial logistic regression models to examine the internal reliability and construct validity of the data. The purpose of the Rasch analyses was to evaluate the reliability of person and item measurements and identify any items that exhibit misfit. Furthermore, the study aimed to identify potential biases in the measure that may affect the accuracy of the results by investigating the presence of differential item functioning (DIF) and clustering effects across gender and race. We conducted the Rasch analyses using Winsteps (version 3.91). We treated missing data using the default approach by retaining all nonmissing values for the analysis. We calculated fit statistics--including item and person reliability as well as infit and outfit mean square (MNSQ)--to evaluate item and person fit and assessed unidimensionality using principal component analysis (PCA). In addition to examining the individual level data, the research team acknowledged the clustering nature of the data with staff nested in schools. We conducted an intraclass correlation coefficient (ICC) analysis to examine the clustering effect and used multilevel multinomial logistic regression models to control for the participants' overall ability on the latent scale while predicting item response probabilities based on gender and race. Findings: The overall scale fit of the measurement instrument was good, with most items exhibiting good to acceptable model fit to the data. Analyses of differential item functioning (DIF) indicated that while the DIF for gender was negligible, a few items showed notable DIF across racial groups, including three items in the School Working Conditions domain, one item in the Emotional Intelligence domain, and two items in the Capacity to Support Students domain. The study also identified a clustering effect, indicating a tendency for participant responses to cluster with schools, which may coincide with participants' demographic characteristics. After accounting for the clustering effect with a multilevel model, we identified four items as potentially biased. Cognitive interviews about these four items revealed that staff of different racial groups may have different interpretations of some item wordings. Conclusion: The study's findings highlight the importance of conducting rigorous psychometric analyses on a novel measure for educators to ensure accurate representation of different groups and avoid interpretation biases. The findings can inform further refinement of the measure and ensure that it is culturally responsive and valid for use across different groups. Moreover, the study's findings can inform policies that promote equitable and inclusive research and practice in education settings. By identifying potential biases in measures, researchers and policy makers can work together to ensure that educators' social-emotional competence and well-being are accurately captured and supported in education settings.
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/
Publication Type: Reports - Research
Education Level: Elementary Education; Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Identifiers - Location: Illinois
Grant or Contract Numbers: N/A