Article Text
PDF
Abstract
Objectives Infectious mononucleosis (IM) is a clinical syndrome that is characterised by lymphadenopathy, fever and sore throat. Although generally not considered a serious illness, IM can lead to significant loss of time from school or work due to profound fatigue, or the development of chronic illness. This study aimed to derive and externally validate clinical prediction rules (CPRs) for IM caused by Epstein-Barr virus (EBV).
Design Prospective cohort study.
Setting and participants 328 participants were recruited prospectively for the derivation cohort, from seven university-affiliated student health centres in Ireland. Participants were young adults (17–39 years old, mean age 20.6 years) with sore throat and one other additional symptom suggestive of IM. The validation cohort was a retrospective cohort of 1498 participants from a student health centre at the University of Georgia, USA.
Main outcome measures Regression analyses were used to develop four CPR models, internally validated in the derivation cohort. External validation was carried out in the geographically separate validation cohort.
Results In the derivation cohort, there were 328 participants, of whom 42 (12.8%) had a positive EBV serology test result. Of 1498 participants in the validation cohort, 243 (16.2%) had positive heterophile antibody tests for IM. Four alternative CPR models were developed and compared. There was moderate discrimination and good calibration for all models. The sparsest CPR included presence of enlarged/tender posterior cervical lymph nodes and presence of exudate on the pharynx. This model had moderate discrimination (area under the receiver operating characteristic curve (AUC): 0.70; 95% CI: 0.62–0.79) and good calibration. On external validation, this model demonstrated reasonable discrimination (AUC: 0.69; 95% CI: 0.67–0.72) and good calibration.
Conclusions The alternative CPRs proposed can provide quantitative probability estimates of IM. Used in conjunction with serological testing for atypical lymphocytosis and immunoglobulin testing for viral capsid antigen, CPRs can enhance diagnostic decision-making for IM in community settings.
- primary care
- infectious diseases
- public health
Data availability statement
All data relevant to the study are included in the article or uploaded as supplemental information.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
STRENGTHS AND LIMITATIONS OF THIS STUDY
A prospective cohort study with external validation in a geographically separate population.
Proposes alternative clinical prediction rules for a relatively common condition in primary care, infectious mononucleosis.
Only examined infectious mononucleosis where the underlying cause was Epstein-Barr virus; 5%–10% of cases may be due to other organisms such as cytomegalovirus, acute HIV infection and toxoplasmosis.
Derivation and validation cohorts differ in terms of the reference standard diagnostic test used; lower specificity of the heterophile antibody test in the validation cohort may have resulted in more false-negative results.
Introduction
The diagnosis of infectious mononucleosis (IM) is made on clinical grounds, based on characteristic symptoms such as fever, sore throat, lymphadenopathy and malaise. Less common features include a maculopapular rash, jaundice, splenomegaly and more rarely hepatomegaly.1 Accurate diagnosis of IM in family practice is important. Correct diagnosis enables family physicians to provide patients with advice on avoidance of contact sports activity (to prevent the possibility of splenic rupture) and will help patients avoid unnecessary antibiotic treatment.2 It also supports the provision of prognostic information about IM, with some evidence suggesting that approximately 12% of patients affected will experience persistent fatigue.3
Clinical prediction rules (CPRs) are tools that combine and weight the contribution of symptoms, clinical signs and available diagnostic tests, and in doing so stratify patients according to the probability of having a target disorder.4 There are three main stages in the development of a CPR: derivation, validation (internal and external) and impact analysis.5 6 External validation is needed for two primary reasons: first, associations between explanatory variables and outcomes may occur because of chance, and second, the explanatory variables for the CPR may be particular to the derivation population setting and may not be widely generalisable to a new setting.5 6
There are no widely used CPRs for IM. A 2021 systematic review of diagnostic accuracy of clinical signs, symptoms and haematological parameters for IM concluded that the presence of posterior cervical lymphadenopathy, inguinal or axillary lymphadenopathy, palatine petechiae, splenomegaly and atypical lymphocytosis with or without high lymphocyte count increased the likelihood of a diagnosis of IM.7 The systematic review concluded that well-designed prospective studies are required to further examine the importance of clinical and laboratory-based findings and study the natural history of IM.
The aim of this study was to develop a CPR to aid in the clinical diagnosis of IM and externally validate in an independent population.
Methods
Cohorts and procedures
The derivation cohort was recruited from seven university student health centres in Ireland. Recruitment took place from September 2017 until May 2019. The external validation cohort was recruited from a student health centre at the University of Georgia, USA, between September 2015 and January 2019.
For the derivation cohort, inclusion criteria for participants were those aged over 18 years (who were able to give informed consent) presenting with sore throat and at least one other of the following symptoms: malaise, fatigue, lymphadenopathy, fever, headache, symptom duration 7 days or less. Patients were excluded if they suffered from a medical condition or treatment associated with significant impaired immunity as determined by the recruiting physician; if they had health literacy or language difficulties. For the validation cohort, participants were included if their treating physician had a clinical suspicion of IM and ordered a diagnostic test.
The outcome measure is the diagnosis of acute IM caused by Epstein-Barr virus (EBV). In the derivation cohort, the reference standard test was positive EBV serology determined by a positive anti-VCA (viral capsid antigen) IgM result. In the validation cohort, positive EBV status was determined by a positive heterophile antibody test.1 In general, EBV antibody testing is more sensitive than heterophile antibody testing, and ideally, the diagnostic tests in the derivation and validation populations would be the same; however, this was not possible in this study for pragmatic reasons.
Explanatory variables considered for inclusion in the derivation CPR model were based on the four most likely clinical variables (presence of enlarged posterior cervical lymph nodes, presence of enlarged inguinal or axillary lymph nodes, palatine petechiae, splenomegaly) based on a systematic review of signs and symptoms for the diagnosis of IM.1 2 Further variables including tonsillar exudate and presence of fever, based on consensus discussion between clinical colleagues, were also included. The validation population had similar characteristics to the derivation cohort, but many of the variables collected in the derivation cohort were not present in the validation cohort. Hence, for the purposes of validation, only one of the four alternative derived CPR models proposed was subject to external validation.
Sample size for the derivation study was calculated using a ratio of 1:4 of cases to non-cases. The guidance for estimating the sample size required to derive a CPR advises a minimum of 10 participants with the outcome and 10 participants without the outcome for each explanatory variable used.8 Using the six most likely variables, a sample size of 300 participants (with sore throat at presentation) was calculated. For the validation study, there is no agreed consensus on determining an adequate sample size in external validation studies; however, a minimum of 100 events and 100 non-events is recognised as an acceptable sample size when externally validating a CPR.9
From the literature, EBV IM accounts for approximately 8% of cases of sore throat. However, the validation population differed, in that it consisted of patients in whom IM was suspected and the treating clinician requested a diagnostic test. As previously mentioned, approximately one-third of tests performed for EBV IM have a positive result. Using this 1:2 ratio, for the purposes of calculating sample size, it was estimated that a sample size of approximately 300 would be required to yield 100 events of EBV IM.
Missing values for each factor were tabulated and multiple imputation considered. However, due to the relatively small number of missing values, the analysis included only participants with complete data.
Descriptive statistics, univariable and multivariable associations
Descriptive statistics (mean, standard deviation (SD), frequency, percentage) were generated for patients testing positive and negative for EBV. An assessment of the diagnostic effect of each explanatory variable was considered separately, with the results expressed as odds ratio (OR) and associated 95% confidence intervals (CIs). A threshold p value of ≤0.15 in the univariate analysis, or variables which were considered important following literature review and/or consensus, were then included in a multivariable logistic regression model.
Four multivariable models were developed as CPRs and transformed to point-based rules. For two of the models, points/scores were assigned by rounding the regression coefficients to the nearest integer. For the other two models, the regression coefficients were first multiplied by 10 and then rounded to the nearest integer. In model 1, all explanatory variables of prior clinical importance or associated with a threshold p value of ≤0.15 in the univariable analysis were included in the model. In model 2, stepwise backward regression was used to optimise the prediction model through simplification. The same variables were included as in model 1, but a backward selection was used with a p value threshold of 0.05. This simplification improves the practicality of use for the physician, by reducing the number of components of the CPR that need inputting. Lastly, the impact of global clinical assessment by the examining physician has been shown to be important in relation to the diagnosis of pneumonia.10 For this reason, the impact of the explanatory variable ‘clinical impression’ on overall CPR performance was assessed in two further models (models 3 and 4) by excluding the ‘clinician impression’ variable from model 1 and model 2, respectively.
Discrimination, calibration and external validation
There are two main ways to assess how well a clinical prediction rule model performs: discrimination and calibration. Discrimination is a model’s ability to differentiate between individuals at higher risk and lower risk of having an outcome. Discrimination of the CPRs was quantified using the area under the receiver operating characteristic curve statistic (AUC) and 95% CI. An AUC of 0.5 represents chance, 0.7–0.9 represents moderate discrimination and 1.0 represents perfect discrimination.11
Calibration is the agreement between the predicted absolute risk and the observed risk for the outcome of interest. Calibration is measured by assessing the predicted and observed risk at different strata of risk. In this study, initial internal calibration was carried out by means of split sampling in the derivation cohort and assessed visually and using the Hosmer-Lemeshow test for goodness of fit, which examines whether the difference between the predicted and observed outcome can be explained by chance.11 Subsequent, external calibration was then carried out using the US population of participants on CPR model 4 again using the Hosmer-Lemeshow test for goodness of fit.11
Likelihood ratios, post-test probabilities and further diagnostic testing
Likelihood ratios (LRs) and subsequent post-test probabilities for EBV were calculated for all the models, to assess the extent to which using the different CPR models changed the probability of a positive diagnosis of EBV using a Bayesian approach.12 12
Based on a previous systematic review, the impact of further diagnostic testing after application of CPRs was assessed incorporating the variable ‘atypical lymphocytosis >10%’. An LR for atypical lymphocytosis was calculated from the univariable analysis. This LR was applied in series, following the CPR, adopting a Bayesian approach.12 13 The previously calculated post-test probabilities for all four models now became pretest probabilities, and the LR for atypical lymphocytosis was applied to these to arrive at new post-test probabilities.12 13
Data were analysed using Stata software V.16 (College Station, Texas, USA) for statistical analysis.14
Patient and public involvement
None.
Results
Descriptive statistics, univariable and multivariable associations
In the derivation cohort, there were 328 participants, of whom 42 (12.8%) had a positive EBV serology test result. Participants’ ages ranged from 17 to 39 years old, with 225 (68.6%) female. From unadjusted logistic regression analyses (table 1), the following clinical variables were identified (p≤0.15) for inclusion in the multivariable analysis: presence of enlarged/tender posterior cervical lymph nodes, presence of exudate on tonsils, presence of splenomegaly, absence of shortness of breath, absence of feeling generally unwell, presence of exudate on pharynx, atypical lymphocytosis and ‘clinician impression’. In the validation population of 2342 participants, 379 (16.2%) had positive heterophile antibody tests.
Summary of variables and univariable associations between explanatory variables and EBV positivity in derivation cohort
We fitted four alternative CPR models (models 1–4) by means of multivariable analysis with results expressed as estimated regression coefficients and 95% CIs (table 2). These regression coefficients were then converted to integer-based CPR scores/points (table 2).
Multivariable analysis of alternative clinical prediction rule models 1–4
Discrimination, calibration and external validation
The discrimination and calibration were investigated for the four CPR models (table 1, figures 1 and 2). In terms of discrimination, the AUC values ranged from 0.70 to 0.80 for all models, indicating moderate discrimination between those with and those without EBV (figure 1). The number of expected EBV cases (expected) was compared with the actual number of EBV cases (observed) and shown in figure 2. The Hosmer-Lemeshow goodness-of-fit test showed no evidence of a significant difference between the expected and the observed risk of EBV for all models, indicating good calibration (table 1).
Receiver operating characteristic (ROC) curve for models 1–4; a graph of the sensitivity (y‐axis) and the specificity (x-axis).
Internal validation showing observed and expected risk of EBV for models 1–4. EBV, Epstein-Barr virus.
A summary of symptoms and signs in the derivation and validation cohorts is presented in table 3.
Summary of clinical variables and EBV positivity in derivation and validation cohorts
It is evident from table 3 that there were clear differences between the participants with EBV and the overall population for some of the variables, in particular for the variables ‘enlarged/tender posterior cervical lymph nodes’, ‘exudate on pharynx’ and ‘atypical lymphocytosis’. The EBV CPR score using model 4 was calculated for each individual in the validation cohort. Figure 3 shows the AUC which was 0.69 (95% CI: 0.67–0.72) and the Hosmer-Lemeshow goodness-of-fit test showed no evidence of a significant difference between the expected and the observed risk of EBV (χ2=1.94; p=0.378), indicating good calibration (figure 4).
External validation of model 4. ROC, receiver operating characteristic.
External validation of model 4—observed and expected risk of Epstein-Barr virus (EBV).
LRs, post-test probabilities and further diagnostic testing
Online supplemental tables 1 and 2 show the positive LRs and post-test probabilities in the derivation cohort for the four alternative CPR models (1–4), based on a pretest probability estimate of 8%. Online supplemental table 3 shows the LRs and post-test probabilities for positive EBV in validation cohort, based on a range of pretest (prior) probabilities. Online supplemental tables 4 and 5 show that inclusion of the positive diagnostic test result of ‘atypical lymphocytosis >10%’ with a positive LR of 60.03 has a substantial impact on all CPR models with post-test probabilities all greater than 80% for patients with the lowest level of pretest probability (again, based on the derivation cohort). Finally, online supplemental table 6 presents total scores/points and the number and percentage of EBV-positive and negative participants for each model.
Supplemental material
Discussion
Summary
We present four alternative multivariable CPRs that can be used to aid clinicians when assessing young adults with prolonged symptoms of fever, sore throat, lymphadenopathy or malaise where the possibility of IM arises as part of a differential diagnosis workup. The AUC values for the four models ranged from 0.70 to 0.80 indicating moderate discrimination between those with and without EBV. The more complex CPR models (1 and 3) provide slightly higher discrimination metrics; these CPRs can be integrated into computer-based clinical decision support systems (CDSS) and used as diagnostic aids via electronic health record software though do require collection of more clinical data than might routinely be considered in clinical practice.15 The briefer CPR models are also presented and provide diagnostic decision-making evidence by applying a minimal number of clinical variables converted as integer-based CPR scores.
Consistent with previous systemic reviews, incorporating atypical lymphocytosis (applied as a diagnostic test in series after initial CPR assessment) produces high post-test probabilities for EBV ranging from 94.8% to 99.0%, ‘ruling in’ EBV IM. Even applying atypical lymphocytosis to a low or zero CPR score produced a post-test probability of >80% (with relatively narrow CIs) for all four CPR models, confirming the high diagnostic value of this test.2
The simple, two-item (model 4) CPR had reasonable discrimination (AUC 0.69, 95% CI: 0.67–0.72) and good calibration when externally validated in a separate US population. In terms of clinical value, post-test probabilities were calculated from a range of plausible pretest probabilities (8%–16%). The results demonstrated that the post-test probability of EBV would be approximately 40% if a patient has a pretest probability of 16% and both CPR variables are present (online supplemental table 3). In this context, further diagnostic testing with a more specific blood test (positive anti-VCA IgM or IgG) would be required.12 If one or both of the variables from the model 4 CPR are present, the patient may be categorised as intermediate risk of EBV IM diagnosis, and the clinician may be more inclined to perform a definitive test, whereas if neither variable is present, the patient may be categorised as low risk, allowing the clinician to adopt a ‘watch-and-wait’ approach. Ultimately, the test/treatment and test/reassurance thresholds will depend on several factors—availability and cost of diagnostic testing and patient’s and clinician’s willingness to accept diagnostic uncertainty without testing all patients who present with possible IM.
Strengths and limitations
Several limitations in relation to this study need to be considered. We only examined IM where the underlying cause was EBV and did not consider other known pathogens such as cytomegalovirus or herpes simplex virus, which may account for between 5% and 10% of IM cases.1 2 The derivation and validation cohorts differ in terms of their inclusion criteria and in terms of the reference standard diagnostic test used though both cohorts targeted similar populations presenting with symptoms associated with IM. Broader inclusion criteria in the validation cohort meant that a higher proportion of participants tested positive for EBV. However, lower specificity of the heterophile antibody test in the validation cohort may have resulted in more false-negative results. Lastly, while we included 28 variables, we did not assess all known clinical variables that could be possibly associated with IM. This was on pragmatic grounds as participating physicians recruited, examined and reported on symptoms and signs during real-time clinical practice without the help of a research assistant. Despite these limitations, there was broad similarity in terms of clinical features between the two cohorts (table 3) enabling external validation of the derived CPR.
Comparison with existing literature
The findings of this study add further evidence to previous systematic reviews about the signs and symptoms of IM caused by EBV in community settings.2 7 Though there are currently no widely used CPRs for IM, many of the clinical variables in our four proposed CPR models have been identified as being of diagnostic value in previous literature.2 7
In addition, in the present study, we have been able to estimate the independent effects of symptoms and signs by means of multivariable logistic regression. Previous systematic reviews, though useful, have only reported the diagnostic value of individual symptoms and signs based on univariable analysis.2 7 Deriving and externally validating a CPR for IM provide clinicians with four alternative CPRs that can be integrated as part of a computer-based CDSS (models 1 and 3) or as a simpler point-scoring CPR (models 2 and 4).
The data collected may also be useful in the context of CPRs for sore throat caused by group A beta haemolytic Streptococcus, as diagnostic testing was performed for this on all participants also.
Implications for research and practice
The CPR would benefit from further validation studies in more diverse populations, particularly in terms of recruiting sufficient number of higher-risk patients who have most of the symptoms and signs of IM. Lastly, the validated CPRs should be subject to impact analysis by means of randomised controlled trials that assess the use of IM on patient outcome.
The CPRs developed provide additional diagnostic evidence about young adults who present to their family physician with symptoms suggestive of IM. Completing either the longer (models 1 and 3) or shorter (models 2 and 4) CPRs will provide clinicians with a means of estimating EBV infection and deciding on further diagnostic testing.16 It seems reasonable that initial diagnostic testing should be performed to establish the presence of lymphocytosis and/or atypical lymphocytosis, as this is a readily available and low-cost test. More specific diagnostic testing based on positive anti-VCA immunoglobulin response (IgM or IgM) can be reserved for situations of remaining diagnostic uncertainty. Where the diagnostic threshold for confirming IM lies will depend on individual preferences and availability and cost of diagnostic tests. Although the CPR for diagnosis of IM caused by EBV demonstrates moderate rather than high discrimination, it may help clinicians stratify patients into low, intermediate and high-risk groups. Additional diagnostic testing in series by examining the presence of atypical lymphocytosis will aid in diagnostic risk stratification. In terms of clinical decision-making, application of a CPR is likely to be most useful in performing an important ‘rule out’ function. There are no predefined test/treat thresholds for IM—unlike some other conditions, for example, influenza, acute coronary syndrome and community-acquired pneumonia.13 17 However, if neither CPR variable is present in a patient, especially if the patient’s age is under 15 or over 25 years (at which point the pretest probability drops to approximately 2% or less), the clinician may be reasonably satisfied that the post-test probability is low enough for them to manage the patient expectantly, or consider an alternative diagnosis.18 The LRs from having both variables in the CPR are too low (post-test probability at 23.2% for the age range 16–20 years, and lower at other ages) for a clinician to diagnose IM caused by EBV without ordering a triage (atypical lymphocytosis) or more definitive VCA serology test. Use of IM CPRs will provide greater explicit quantitative information about these implicit probability estimates.12
Conclusion
We have developed four simple CPRs to help family physicians identify which of their symptomatic patients have EBV-associated IM. External validation of a simple two-item CPR (model 4) shows good calibration. These CPRs should be subject to further external validation and impact analysis, but provide an evidence-based structure for assessment and further diagnostic testing of younger patients with suspected IM in community settings.
Data availability statement
All data relevant to the study are included in the article or uploaded as supplemental information.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants and ethical approval was initially granted for this study from the Royal College of Surgeons in Ireland (RCSI) research ethics committee in 2017. An extension of ethical approval, to allow adequate time for follow-up data collection to occur (reference number: REC201908006, see online supplemental appendix 5: Ethical approval and amendments for cohort study), was granted in 2019. Collection of de-identified data from electronic health records at the University of Georgia Health Center was approved by the university’s human subjects committee in September 2019, with the approval valid until September 2024. Participants gave informed consent to participate in the study before taking part.
Acknowledgments
The authors thank all the study participants and student health centres who gave so generously of their time in taking part in this study.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors MdP, FB, SS and TF conceptualised and designed the study. EMD and MdP prepared the derivation study data. XC and MHE prepared the validation study data. FB and MdP performed the statistical analysis. All authors contributed to interpretation of the findings. TF and MdP wrote the original draft. All authors contributed to reviewing and editing the manuscript and approved the final manuscript.
MDP is responsible for the overall content as the guarantor.
Funding MdP was in receipt of a clinical research fellowship awarded by the Health Service Executive (HSE) and Irish College of General Practitioners (ICGP; grant number N/A).
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.