Psychological safety in elite sport settings: a psychometric study of the Sport Psychological Safety Inventory

Design

A cross-sectional online survey design was implemented. Participants provided data on a range of standardised mental health screening tools, in addition to items comprising a bespoke scale measuring psychological safety in elite sports.

Participants

Eligible participants were National Sporting Organisation (NSO) contracted elite athletes aged ≥16 years and coaches/high-performance support staff (HPSS, including sports medicine staff and other allied health professionals) identified on the Australian Institute of Sport (AIS) contact list. Other than age and ability to complete the survey in English, no exclusion criteria were applied.

Measures

Procedure

Data was collected between 16 March 2020 and 31 May 2020. The broader survey was designed in consultation with AIS Mental Health and Well-being staff. Recruitment procedures mirrored those previously reported for the 2018 AIS Mental Health Audit.3 Potential participants were contacted via SMS or email. The online survey was anonymous and was completed by participants at the place and time of their choosing.

Data analysis

Statistical procedures are outlined in full in online supplemental information and summarised below. The athlete sample was randomly partitioned into an athlete calibration sub-sample (n=169) and an athlete validation subsample (n=168). Exploratory factor analysis (EFA; principal axis factoring) with direct quartermin (delta=0) rotation was undertaken with data from the athlete calibration sample to identify the factor structure of the SPSI with data provided by the athletes. The item pool (see online supplemental table 1) for the SPSI includes several negatively valenced items, which were reverse scored. Parallel analysis was conducted to identify the number of underlying factors. Confirmatory factor analysis (CFA) was undertaken on data from the athlete validation sample and the coaches/HPSS according to the factor model established using the calibration sample. Reported indices were RMSEA <0.05 (indicating close fit via the p-close value), TLI and CFI ≥0.90 for acceptable fit and ≥0.95 for excellent fit, and SRMR ≤0.08. Bifactor analysis was subsequently undertaken to determine if the SPSI total score should be considered. Internal consistency for subscales of the SPSI was evaluated using Cronbach alpha (α) and McDonald omega (Ω) maximum likelihood coefficients, with coefficients ≥0.70 indicating acceptable reliability. SPSI percentile distributions were examined separately for athletes and coaches/HPSS, with those scoring ≤25% quartile (eg, low perceived psychological safety) examined on distress and well-being outcomes. Multivariate analysis of variance with follow-up univariate tests examined mean differences between athletes and coaches on SPSI domains. Spearman correlations were examined between outcome variables. Generalised linear regression examining binary logistic models (α=0.01, 99% ORs) explored the three SPSI domains as predictors of K10 (≥16) and APSQ (≥15) moderate distress cut-off thresholds. SPSS V.26.0 and Mplus V.8.2 were used for analyses. Reporting is consistent with the STROBE Statement for observational (cross-sectional) studies.

Patient and public involvement

A steering committee (including athlete representatives) was established. This group provided feedback on the structure and overall methodology of the larger survey.

Exploratory factor analysis

The KMO value (0.871) and Bartlett’s test of sphericity (p<0.001) indicated excellent factorability of the data. Candidate items exhibited mild negative skew (see online supplemental table 2) but did not exceed criteria requiring transformation (eg, ≥2.0).34 35 Parallel analysis was undertaken with the full pool of 25 items, indicating a six-factor solution, which was subsequently imposed on the data. This initial analysis yielded factors with eigenvalues ranging from 7.86 to 1.05, accounting for a cumulative total of 56.65% of scale variance, converging in 17 iterations. Fourteen items (items: 15, 16, 14, 13, 4, 19, 20, 21, 22, 24, 23, 25, 18, 17) failed to either load at the threshold of ≥0.56 or loaded as a single or dual item factor. These items were subsequently omitted. Parallel analysis was reiterated on the remaining items, indicating a three-factor solution (KMO=0.842, Bartlett’s p<0.001). This 11-item, 3-factor solution was parsimonious, accounting for a cumulative total of 64.36% of scale variance, converging in six iterations. There were no cross-loading or underloading items or any two-item factors. Factor 1 (eigenvalue=4.24) was named Mentally Healthy Environment and accounted for 40.22% of variance after rotation, Factor 2 (eigenvalue=1.81) was named Mental Health Literacy and accounted for 16.42% of variance after rotation and factor 3 (eigenvalue=0.85) was named Low Self-Stigma and accounted for 7.72% of variance after rotation. As shown in table 2, loadings for each factor were strong, with clear measurement differentiation between the factors (ie, very low loadings on corresponding non-identified factors).

Confirmatory factor analysis

Using data from the calibration sample of athletes, the EFA three-factor solution was subject to CFA. For athletes, the unidimensional model (all items loading on a single factor) was a poor fit for the data, with all indices failing to meet recommended cut-offs (see table 3). In contrast, the second-order hypothesised model indicated a good fit (see table 3), although the SRMR (0.081) was marginally above the recommended cut-off of 0.80. Finally, the bifactor model indicated an excellent model fit on all indices. However, while the bifactor auxiliary indices indicated that the percentage of uncontaminated variance (PUC) index was relatively high (PUC=0.73), the estimated common variance (ECV) index for the total score fell below the recommended cut-off of 0.70 (ECV=0.33), as did the average relative parameter bias (recommend <0.15; ARPB=0.41). Standardised loadings are reported in figure 1.

For coaches/HPSS, the unidimensional and second-order models were poor-mediocre fits to the data, whereas the bifactor model indicated excellent model fit. As per the athlete analysis, the PUC index was relatively high (PUC=0.73), and ARPB was within the accepted range (.12). However, the ECV index for the total score was relatively low (ECV=0.50). The bifactor model reported an excellent fit for the combined sample, with ARPB within the accepted range (0.070) and the ECV=0.46.

SPSI properties

Online supplemental information and online supplemental table 2–5 provide evidence of SPSI internal consistency, percentile distributions, between-group differences (eg, higher scores observed for coaches/HPSS) and SPSI convergent and divergent validity.

SPSI domains and predicting general and athlete-specific distress

Generalised linear modelling (binary logistic) was used to predict categorical outcomes of moderate distress on the K10 (for both athletes and coaches/HPSS) and APSQ (for athletes only) using the three SPSI subscales in addition to past 12 month mental health treatment (yes/no). For the K10, the overall model was significant χ²(4)=54.38, p<0.001, with the Mentally Healthy Environment subscale (OR 0.829, 99% CI .744 to 0.914, p<0.001) significantly predicting general psychological distress caseness. A one-unit increase in Mentally Healthy Environment scores was protective against (eg, decreased the likelihood of) being in the K10 Moderate range by 17.1%. For the APSQ, the overall model was significant χ²(4)=49.50, p<0.001, with the Mentally Healthy Environment subscale (OR 0.872, 99% CI 0.812 to 0.944, p<0.001) and the Low Self-Stigma subscale (OR 0.836, 99% CI 0.756 to 0.925, p<0.001) significantly predicting athlete-specific psychological distress caseness. A one-unit increase in the Mentally Healthy Environment subscale was protective against (eg, decreased the likelihood of) being in the APSQ Moderate range by 12.5%, and a one-unit increase in the Low Self-Stigma subscale was protective against (eg, decreased the likelihood) of being in the APSQ Moderate range by 16.4%. For coaches/HPSS, the overall model for psychological distress (K10) was significant χ²(4)=36.06, p<0.001, with Low Self-Stigma, significantly predicting psychological distress caseness (OR 0.841, 99% CI 0.722 to 0.978, p<0.001). A one-unit increase in Low Self-Stigma was protective against (eg, decreased the likelihood of) being in the K10 Moderate range by 15.9%.

SPSI factor structure

Factor analyses supported the three-factor structure of the SPSI, assessing domains of Mental Healthy Environment, Mental Health Literacy and Low Self-Stigma, each of which was positively inter-correlated for both athletes and coaches/HPSS. The Mentally Healthy Environment domain focuses on perceived organisational support for mental health problems among those in the sporting system. The Mental Health Literacy domain can be considered an integral component of mental health-related psychological safety,24 as this knowledge base is fundamental to recognise symptoms of mental ill health and the need for support. The Low Self-Stigma domain assesses an individual’s perspective related to mental health self-stigma within the sporting context. For athletes, each of the SPSI domains was negatively associated with general and athlete-specific distress (eg, lower psychological safety was associated with greater distress) and positively associated with well-being. For coaches/HPSS, significant negative associations were observed between each SPSI subdomain and general psychological distress (K10).

Utility of the SPSI and implications

Logistic regression analyses predicting moderate distress caseness indicated that high scores on the Mentally Healthy Environment and Low Self-Stigma subscales were associated with lower likelihood of experiencing psychological distress (or athlete-specific distress) of at least moderate severity. It was particularly noteworthy that scores on the Mental Health Literacy subscale were not predictive of distress, suggesting that awareness of mental health problems and symptoms is insufficient in protecting stakeholders in the elite sports environment from distress. Instead, more important in reducing the likelihood of psychological distress is the broader culture related to perceived organisational support for disclosure of mental health problems and proactive support for their management. Consistent with Vella et al’s concept analysis of psychological safety in sport being a group-level construct,22 this ‘culture’ could be perceived by the individual as reflecting their teammates, their team and/or their sport as a whole. This finding serves as a ‘call to arms’ for sporting organisations to actively engage in the process of creating and implementing the systems and environments that support safe disclosures and management of mental health symptoms. Equally, it also arguably requires sporting organisations to demonstrate a genuine commitment to the mental health and well-being of key stakeholders in the sporting system.6 For example, the AIS has implemented Well-being Health Checks. A key aspect of these checks is that they assess how well athletes, coaches and staff are supported by their National Sports Organisation when it comes to a culture of feeling safe to disclose mental health difficulties. The organisation proactively supports them to manage these difficulties.

Evidence from organisational literature supports a range of positive outcomes associated with psychologically safe teams, including facilitated learning task and team performance, engagement and creativity.20 These benefits are thought to occur because psychological safety allows teams to provide open feedback, discuss errors, collaborate and experiment with new ideas.17 Psychological safety may also act as an environmental protective factor against mental ill health,36 37 particularly due to promoting openness and vulnerability between team members. Appropriate and supportive responses to mental ill health disclosures also contribute to psychologically safe climates, as these responses demonstrate that it is safe to express vulnerabilities. Exposure (through observation or direct experience) to adverse reactions following disclosure is informative about cultural norms or boundaries of acceptable behaviour, preventing subsequent help-seeking.38 39 Indeed, athletes report that supportive responses from sport-related networks, particularly coaches, facilitate help-seeking.10

Limitations and future directions

While this study offers a new measurement tool developed with robust methods, several important limitations and future directions should be considered. The response rate from the participating NSO’s suggests that the present sample may not be representative. The period of data collection co-occurred with the early stages of the COVID-19 pandemic and associated uncertainty regarding the occurrence of the Tokyo 2020 Summer Olympics and Paralympics, which were subsequently rescheduled to 2021. This may have increased psychological stressors experienced by participants. Future studies should extend the analysis of convergent and divergent (and predictive) validity, including existing measures of psychological safety (eg, the Team Psychological Safety Scale)16 and other outcomes that would be expected to be associated with the SPSI (eg, established measures of mental health literacy, self-stigma, group level and organisational level drivers). Future work should look to validate cut-off scores for the SPSI. Further, results of the bifactor analysis suggest some caution may be warranted in using the SPSI total score (as it accounted for between 33% and 46% of common variance), below the conservative criteria of 70%.40 Accordingly, we call for replication studies of the psychometric properties of the SPSI in larger and more diverse samples (eg, cross-cultural settings).