Evaluation of CogSport for acute concussion diagnosis in cricket
•,,,,.
...
Abstract
Objective The diagnosis of sport-related concussion is a challenge for practitioners given the variable presentation and lack of a universal clinical indicator. The aim of this study was to describe the CogSport findings associated with concussion in elite Australian cricket players, and to evaluate the diagnostic ability of CogSport for this cohort.
Methods A retrospective study design was used to evaluate CogSport performance of 45 concussed (male n=27, mean age 24.5±4.5 years; female n=18, 23.5±3.5 years) compared with 45 matched non-concussed (male n=27, mean age 27.3±4.5 years; female n=18, 24.1±4.5 years) elite Australian cricket players who sustained a head impact during cricket specific activity between July 2015 and December 2019.
Results Median number of reported symptoms on the day of injury for concussed players was 7 out of 24, with a median symptom severity of 10 out of 120. CogSport performance deteriorated significantly in concussed cricket players’ Detection speed (p<0.001), Identification speed (p<0.001), One Back speed (p=0.001) and One Back accuracy (p=0.022) components. These components, when considered independently and together, had good diagnostic utility.
Conclusion This study demonstrated good clinical utility of CogSport for identifying concussed cricket players, particularly symptoms and Detection, Identification and One Back components. Therefore, CogSport may be considered a useful tool to assist concussion diagnosis in this cohort, and the clinician may place greater weight on the components associated with concussion diagnosis.
What is already known
The diagnostic ability of CogSport to differentiate between a concussed and non-concussed player was good.
CogSport performance for four out of eight tests, Detection speed, Identification speed, One Back speed and One Back accuracy, are typically worse than baseline for players with a concussion diagnosis.
What are the new findings
Neurocognitive testing, such as CogSport, is a useful clinical adjunct tool to assist the diagnosis of concussion following head impact in elite cricket players.
An abbreviated test, using only two or three components of the CogSport test, could be valuable in time-pressured environments, without reducing the diagnostic ability of the test.
Introduction
Sport-related concussion has become a common and significant diagnosis in the athletic population.1 The diagnosis of sport-related concussion is a challenge for practitioners given the variable presentation and lack of a universal clinical indicator.2 3 Sport-related concussion has been described as an evolving physiological process and neurocognitive symptoms may take minutes to hours to become apparent, adding to the complexity of diagnosis for the medical practitioner at the time of injury if an athlete does not report any symptoms.2 The Sport Concussion Assessment Tool (SCAT-5) is currently the gold standard for concussion diagnosis, but it is thought to lack reliability due to the subjective nature of some components and lack of definitive diagnostic ability.
Computerised neurocognitive tests have become an increasingly popular tool over the past 15 years due to the ease and standardisation of test administration, including baseline testing and ability to quantify return of cognitive performance after a concussion diagnosis.3 4 Neurocognitive tests have been shown to improve diagnostic sensitivity of concussion by 19% compared with diagnosis based on symptoms alone.5 These tests have been able to help diagnose subtle impairments following a head injury, revealing decreased scores in concussed athletes who did not report any symptoms up to 4 days after sustaining a head injury.6–8 Reproducible tests such as these are also useful in monitoring an athlete’s neurocognitive recovery, which can occur on a very individualised timeline.9
There are a number of different computerised neurocognitive tests used to assess sport-related concussion. A study in 2012 found the Computerised Cognitive Assessment Tool for Sport (CogSport; Cogstate) was the second most widely used computerised neurocognitive test.10 CogSport combines both reported symptoms and a neurocognitive test comprising of four components which are completed over 10–15 min in a predetermined order: Detection (psychomotor function), Identification (attention/decision making), One Back (working memory) and One Card Learning (new learning).7 11 Measures of speed and accuracy on these components have been shown to be highly reliable in healthy young people.12
The ‘baseline method’ has been recommended within the literature for neurocognitive tests to compare postinjury data scores with preinjury baseline results.11 13 Louey et al11 found the CogSport test to have both a high sensitivity and specificity for detecting cognitive impairment associated with sport-related concussion when using the baseline method. Uncontrolled influences, such as social and cultural backgrounds, learning, language and past head injuries can then be accounted for with each individual’s baseline test, thus improving the test’s sensitivity and reliability.5 11 14
Injury surveillance in Australian cricket has shown the match incidence rate of concussions per 1000 player days was 2.3 and 2.0 for the elite men and women, respectively.15 This was equivalent to one concussive injury sustained every 9000 balls. Only 30% of concussions were diagnosed immediately, at time of the head impact, with the remainder diagnosed as ‘delayed concussions’.15 It is important that these athletes can be assessed immediately to determine whether they can return to the field of play in a safe manner, in the interest of player welfare.
Cricket Australia has employed CogSport as part of a standardised diagnostic protocol to assess elite cricket players who sustain a head impact during a match or training for the previous four seasons, presenting a valuable opportunity for retrospective analysis. The aim of this study was to describe which CogSport findings are associated with concussion in elite Australian cricket players, and to evaluate the diagnostic ability of CogSport for this cohort.
Methods
Study design and setting
Participants
Forty-five elite Australian cricket players (male n=27, mean age 24.5±4.5 years; and female n=18, mean age 23.5±3.5 years) who were diagnosed with concussion following a head-impact during an organised cricket activity between July 2015 and December 2019.
Players who sustained a subjectively determined, potentially concussive head impact and completed the assessment but were later confirmed to not have sustained a concussion were included for comparison as ‘non-concussed players. The sample of non-concussed players were selected as the most recent age-matched 27 male (27.3±4.5 years) and 18 female (24.1±4.5 years) (same number as concussed) assessments.
Concussion diagnosis
Concussion diagnosis was confirmed by experienced medical staff, based on a full clinical assessment including witnessing or witness description of the incident, clinical assessment, SCAT-5,16 and CogSport, both at the time of the incident and over subsequent days. All information pertaining to the concussive episode was obtained directly from Cricket Australia’s online Athlete’s Management System (Fair Play AMS) by the research team.
CogSport
CogSport assesses a player’s speed and accuracy in four components: psychomotor function, attention/decision making, working memory and new learning. It also assesses the presence and severity of any symptoms. Raw data for speed, accuracy and symptoms are continuous variables. The programme also considers speed and accuracy data relative to an individual’s baseline to determine whether a meaningful decline in performance constitutes a ‘fail’.
Cricket players routinely complete a baseline CogSport assessment during the pre-season. A practice test allows players to familiarise themselves with the test, and minimum performance standards, equivalent to age-matched peers, are inbuilt to ensure players complete a test which is considered ‘valid’. All players who sustain a head impact complete a CogSport test as soon as practical on the day of impact. Both baseline and posthead impact tests are completed in a quiet room, under the direction of medical staff (doctor or physiotherapist) familiar with the testing procedure.
Data analysis
CogSport raw data on the day of head impact was compared with each individual’s most recent baseline raw data to calculate intraindividual change. Data were not normally distributed, hence central tendency is reported as median and IQR. Related-samples Wilcoxon signed rank tests were used to compare postinjury and baseline data.
The diagnostic value of the four test components and overall test result were evaluated using 2×2 contingency tables to calculate sensitivity, specificity, positive predictive values (PPV) and negative predictive values (NPV) with Fisher’s exact p values. Performances for specific components which did not meet the inbuilt test integrity criteria were omitted (eg, identification accuracy <80% suggests the identification component was not completed correctly and hence the performance for this component only was omitted).
Receiver operating characteristic (ROC) curves were used to compare the diagnostic ability of different combinations of test components to identify if a certain combination of test components was superior for concussion diagnosis.
Analysis was completed using Excel (Microsoft, 2016 MSO) and SPSS (V.25, IBM).
Patient and public involvement
Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Results
CogSport performance and symptoms on the day of injury for concussed players
Five (out of 355; 1.4%) performances for specific components did not meet the integrity check for the test, hence these specific performances were omitted from analysis.
CogSport performance and symptoms relative to intra-individual baselines are reported in table 1. The median total number of symptoms reported on CogSport on the day of head injury was 7 out of 24 (IQR 5–11) and symptom severity was 10 out of a possible 120 (IQR 7–19).
Table 1
•
CogSport performance and symptoms for concussed elite male and female cricket players, expressed as Raw score and intraindividual change from most recent baseline
The most commonly reported symptoms for concussed players were: headache (91%, median severity 2 out of 5 (IQR 2–4)), ‘don’t feel right’ (78%, median severity 2 (1–3)), pressure in head (71%, median severity 2 (1–3)), fatigue or low energy (58%, median severity 1 (1–3)), feeling slowed down (51%, median severity 1 (1–3)), balance problems or dizzy (51%, median severity 1 (1–2)) and feeling ‘dinged’ or ‘dazed’ (44%, median severity 2 (1–3)).
Number of symptoms and symptom severity were strongly associated with concussion diagnosis, along with the speed to complete three of the four test components: Detection, Identification, One Back (all p≤0.001).
Diagnostic ability of CogSport
The diagnostic ability of the CogSport test taken as a whole (performance on four components plus symptoms) was: sensitivity 67% (95% CI 54% to 78%), specificity 96% (95% CI 79% to 100%), PPV 98% (95% CI 87% to 100%), and NPV 51% (95% CI 42% to 60%) (p<0.001). The diagnostic ability of separate and combined components, without the addition of patient-reported symptoms, is detailed in tables 2 and 3, relative to normative values and individual baselines respectively. ROC curve analysis (table 4) demonstrated that combinations of test components, without symptoms, were superior to chance, yet no particular combination of components or comparison between individual baseline and normative values was better than others (all p>0.2).
Table 2
•
Diagnostic ability of CogSport for concussion diagnosis compared to CogSport normative values
Table 3
•
Diagnostic ability of CogSport for concussion diagnosis compared to individual baseline performance
Table 4
•
Area under the receiver operating characteristic curve
Discussion
Sport-related concussion is a challenge for clinicians to diagnose. Objective assessments such as the CogSport neurocognitive test may assist clinicians with their diagnosis. The purpose of this retrospective study was to describe the findings of CogSport test on the day of injury for elite Australian cricket players diagnosed with a concussion, and to evaluate the diagnostic ability of CogSport for this cohort.
Concussion was associated with impaired performance on three of the four CogSport components, in addition to increased number of symptoms and symptom severity. The diagnostic ability of CogSport was enhanced when combining multiple test components, particularly three-component and four-component combinations. Scoring against individual preinjury baseline performance also helped improve the diagnostic ability.
Symptoms
Symptoms most commonly reported by concussed cricket players included headache, ‘don’t feel right’, pressure in head and fatigue, which are consistent with findings in other athlete cohorts.17–20 Concussed cricket players in this study reported a median total number of seven symptoms, yet they were typically of relatively low self-reported severity. While self-reported symptoms are a common and telling aspect of concussion diagnosis,20 21 clinicians must be mindful that symptoms may develop up to 24–72 hours postinjury.22 23 There is also a potential risk for athletes to intentionally under-report any symptoms in order to avoid a diagnosis of concussion.24 Therefore, an objective measure such as CogSport may complement symptom reporting.
CogSport performance
Concussed cricket players typically performed slower for detection, identification and One Back components compared with individual baselines. There was also a decline in the One Back accuracy. A previous study in a similar population has demonstrated that such changes can not be dismissed as normal decline as a result of the physical and mental exertion of participating in a cricket match.25
Slower performance on CogSport following head injury are indicative of cognition changes.12 The simple psychomotor reaction time measure (Detection) has previously been shown to provide the most sensitive index of cognition changes after a concussive head injury.4 Psychomotor (Detection) and attentional (Identification) components have also been observed to be impaired in elite Australian Rules Footballers and Rugby Union players after a concussion when compared with preseason baseline.11 26
Diagnostic ability of CogSport
The present results suggest the CogSport test, in various forms, were all meaningful in its proficiency to differentiate between concussed and non-concussed players. The use of the baseline method is superior to the normative method, which is consistent with the literature,11 however, both are acceptable. The baseline method was more sensitive, possibly negating the influence of individual factors such as age, gender, education and language.17 22 27 Our results support those of Louey et al11 who found the CogSport normative method to have moderate sensitivity and high specificity for concussion diagnosis in male elite or professional athletes.
Symptoms add further value to the neurocognitive components. This is consistent with previous findings suggesting symptoms and neurocognitive tests can complement one another to assist diagnosis.5 24 In settings where athletes may be prone to under-reporting symptoms, the addition of an objective neurocognitive test may be particularly useful.2 20 24 Czerniak et al1 recently compared a number of computerised neurocognitive tests, including CogSport, and found that while there was an overall low sensitivity and specificity to these type of tests, they should not be abandoned, but used in conjunction with clinical examination and as part of a multidimensional approach. The findings of this study support this approach.
The CogSport test takes approximately 10–15 min for completion of all four components. The findings of this study support the possibility that the test may be abbreviated to three (Detection, Identification and One Back) or even the first two components without considerably reducing the diagnostic ability of the test. An abbreviated test would be valuable in time-pressured environments such as determining whether or not a player is able to return to the field of play during a match which is in-play. This is with the caveat that the clinician may choose to complete the entire four component test if they feel it is required, and only use the CogSport results to inform their clinical judgement along with other assessment(s). The complete four component CogSport test would still remain beneficial in a recovery assessment or when time taken for testing is not critical.
Limitations
A key limitation of this study is that the CogSport results are not independent of concussion diagnosis, as they were used at the time to assist clinical diagnosis. However, the clinical diagnosis is informed by multiple assessments over several days and was completed without knowledge of this study. A further limitation is that the severity of injury and individual factors such as past history of concussions, cultural backgrounds and language or other confounding medical conditions such as headache disorders/migraines or mental health issues which have been shown to affect concussion diagnosis were not accounted for.17 22 27 However, a focus on intraindividual change from baselines reduces the potential impact of individual factors on findings. A learning effect with repeated CogSport assessment may have improved CogSport performance,28 29 yet the findings of concussed players observed in this study suggest this source of error may be minimal.
While this concussive sample size is a larger cohort compared with any current literature documenting concussion in cricket, with only 27 male and 18 female cricketers, the small number meant we were unable to adequately compare results between the male and female cricketers to find any gender variance. Males and females may present differently when concussed,19 30 31 hence combining data in this study may have affected findings. Future research is needed to investigate potential gender differences in concussed players.
Conclusion
This study used a retrospective design in a large cohort of elite Australian cricket players to investigate the changes in the CogSport test, including reported symptoms, that are associated with a concussive head injury and to evaluate the diagnostic ability of CogSport in these athletes. The results of this study suggest the total symptoms and symptom severity score recorded in CogSport are elevated in concussed cricket players. CogSport performance for Detection speed, Identification speed, One Back speed and One Back accuracy are typically worse than baseline for concussed players. Therefore, clinicians may put more weight on these test components when interpreting results to inform their clinical judgement when diagnosing concussion. The diagnostic ability of CogSport to differentiate between a concussed and non-concussed player was noteworthy, however was further improved when combining the CogSport test with a player’s reported symptoms. The assessment of a concussion continues to be multimodal and neurocognitive testing, such as CogSport, is a useful clinical adjunct tool to assist the diagnosis of concussion following head impact in elite cricket players.
Twitter: @FNQSports_Med
Contributors: All authors contributed to the design of the study, interpretation of results and drafting of the manuscript. KJ collated data and AES conducted analyses. All authors have read and approved of the final version of the manuscript and agree with the order of presentation of the authors.
Funding: The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests: None declared.
Provenance and peer review: Not commissioned; externally peer reviewed.
Data availability statement
All data relevant to the study are included in the article.
Ethics statements
Patient consent for publication:
Not required.
Ethics approval:
Ethics approval was obtained from La Trobe University Human Research Ethics Committee (S17-195).
Acknowledgements
Cricket Australia supported this research in-kind. We thank the medical practitioners who judiciously assessed players and recorded data over the previous seasons to enable this research. Cricket Australia purchased the CogSport licence for clinical use and did not receive any funding from CogSport for this study.
Czerniak LL, Liebel SW, Garcia G-GP, et al. Sensitivity and specificity of computer-based neurocognitive tests in sport-related concussion: findings from the NCAA-DoD care consortium. Sports Med2021; 51.
McCrory P, Meeuwisse W, Dvořák J, et al. Consensus statement on concussion in sport-the 5th international conference on concussion in sport held in Berlin, October 2016. Br J Sports Med2017; 51:838–47.
Nelson LD, LaRoche AA, Pfaller AY, et al. Prospective, head-to-head study of three computerized neurocognitive assessment tools (CNTs): reliability and validity for the assessment of sport-related concussion. J Int Neuropsychol Soc2016; 22:24–37.
Stuss DT, Stethem LL, Hugenholtz H, et al. Reaction time after head injury: fatigue, divided and focused attention, and consistency of performance. J Neurol Neurosurg Psychiatry1989; 52:742–8.
Lau BC, Collins MW, Lovell MR, et al. Sensitivity and specificity of subacute computerized neurocognitive testing and symptom evaluation in predicting outcomes after sports-related concussion. Am J Sports Med2011; 39:1209–16.
Fazio VC, Lovell MR, Pardini JE, et al. The relation between post concussion symptoms and neurocognitive performance in concussed athletes. NeuroRehabilitation2007; 22:207–16.
Karr JE, Areshenkoff CN, Garcia-Barrera MA, et al. The neuropsychological outcomes of concussion: a systematic review of meta-analyses on the cognitive sequelae of mild traumatic brain injury. Neuropsychology2014; 28:321–36.
Whitehouse DP, Kelleher‐Unger IR, Newcombe VFJ, et al. Head injury and concussion in cricket: incidence, current guidance, and implications of sports concussion literature. Transl Sports Med2020; 54.
Meehan WP, d'Hemecourt P, Collins CL, et al. Computerized neurocognitive testing for the management of sport-related concussions. Pediatrics2012; 129:38–44.
Louey AG, Cromer JA, Schembri AJ, et al. Detecting cognitive impairment after concussion: sensitivity of change from baseline and normative data methods using the CogSport/Axon cognitive test battery. Arch Clin Neuropsychol2014; 29:432–41.
Collie A, Maruff P, Makdissi M, et al. CogSport: reliability and correlation with conventional cognitive tests used in postconcussion medical evaluations. Clin J Sport Med2003; 13:28–32.
McCrory P, Meeuwisse W, Aubry M, et al. Consensus statement on Concussion in Sport--the 4th International Conference on Concussion in Sport held in Zurich, November 2012. J Sci Med Sport2013; 16:178–89.
Hill T, Orchard J, Kountouris A, et al. Incidence of concussion and head impacts in Australian Elite-Level male and female cricketers after head impact protocol modifications. Sports Health2019; 11:180–5.
Putukian M, Riegler K, Amalfe S, et al. Preinjury and Postinjury factors that predict sports-related concussion and clinical recovery time. Clinical Journal of Sport Medicine2018; Publish Ahead of Print.
Makdissi M, Darby D, Maruff P, et al. Natural history of concussion in sport: markers of severity and implications for management. Am J Sports Med2010; 38:464–71.
Sicard V, Moore RD, Ellemberg D, et al. Long-Term cognitive outcomes in male and female athletes following sport-related concussions. Int J Psychophysiol2018; 132:3–8.
Knollman-Porter K, Thaxton S, Hughes MR, et al. Management and recovery patterns following sport-related mild traumatic brain injury in male and female college athletes. Top Lang Disord2019; 39:257–75.
Dunn J, Feng D, Girouard TJ, et al. Sex specific Post-concussion symptom reporting in adolescents: a systematic review and meta-analysis. Adolesc Res Rev2020; 5:67–76.
McCrory P, Feddermann-Demont N, Dvořák J, et al. What is the definition of sports-related concussion: a systematic review. Br J Sports Med2017; 51:877–87.
Broglio SP, Katz BP, Zhao S, et al. Test-Retest reliability and interpretation of common concussion assessment tools: findings from the NCAA-DoD care consortium. Sports Med2018; 48:1255–68.
Goh SC, Saw AE, Kountouris A, et al. Neurocognitive changes associated with concussion in elite cricket players are distinct from changes due to post-match with no head impact. J Sci Med Sport2021; 24:420-424.
Van Kampen DA, Lovell MR, Pardini JE, et al. The "value added" of neurocognitive testing after sports-related concussion. Am J Sports Med2006; 34:1630–5.
Weber ML, Dean J-HL, Hoffman NL, et al. Influences of mental illness, current psychological state, and concussion history on baseline concussion assessment performance. Am J Sports Med2018; 46:1742–51.
Collie A, Maruff P, Makdissi M, et al. Statistical procedures for determining the extent of cognitive change following concussion. Br J Sports Med2004; 38:273–8.
Falleti MG, Maruff P, Collie A, et al. Practice effects associated with the repeated assessment of cognitive function using the CogState battery at 10-minute, one week and one month test-retest intervals. J Clin Exp Neuropsychol2006; 28:1095–112.
Fehr SD, Nelson LD, Scharer KR, et al. Risk factors for prolonged symptoms of mild traumatic brain injury: a pediatric sports concussion clinic cohort. Clin J Sport Med2019; 29:11–17.
Covassin T, Elbin RJ, Harris W, et al. The role of age and sex in symptoms, neurocognitive performance, and postural stability in athletes after concussion. Am J Sports Med2012; 40:1303–12.
