Prognostic factors and prediction models for hospitalisation and all-cause mortality in adults presenting to primary care with a lower respiratory tract infection: a systematic review
=======================================================================================================================================================================================

* Merijn H Rijk
* Tamara N Platteel
* Teun M C van den Berg
* Geert-Jan Geersing
* Paul Little
* Frans H Rutten
* Maarten van Smeden
* Roderick P Venekamp

## Abstract

**Objective** To identify and synthesise relevant existing prognostic factors (PF) and prediction models (PM) for hospitalisation and all-cause mortality within 90 days in primary care patients with acute lower respiratory tract infections (LRTI).

**Design** Systematic review.

**Methods** Systematic searches of MEDLINE, Embase and the Cochrane Library were performed. All PF and PM studies on the risk of hospitalisation or all-cause mortality within 90 days in adult primary care LRTI patients were included. The risk of bias was assessed using the Quality in Prognostic Studies tool and Prediction Model Risk Of Bias Assessment Tool tools for PF and PM studies, respectively. The results of included PF and PM studies were descriptively summarised.

**Results** Of 2799 unique records identified, 16 were included: 9 PF studies, 6 PM studies and 1 combination of both. The risk of bias was judged high for all studies, mainly due to limitations in the analysis domain. Based on reported multivariable associations in PF studies, increasing age, sex, current smoking, diabetes, a history of stroke, cancer or heart failure, previous hospitalisation, influenza vaccination (negative association), current use of systemic corticosteroids, recent antibiotic use, respiratory rate ≥25/min and diagnosis of pneumonia were identified as most promising candidate predictors. One newly developed PM was externally validated (c statistic 0.74, 95% CI 0.71 to 0.78) whereas the previously hospital-derived CRB-65 was externally validated in primary care in five studies (c statistic ranging from 0.72 (95% CI 0.63 to 0.81) to 0.79 (95% CI 0.65 to 0.92)). None of the PM studies reported measures of model calibration.

**Conclusions** Implementation of existing models for individualised risk prediction of 90-day hospitalisation or mortality in primary care LRTI patients in everyday practice is hampered by incomplete assessment of model performance. The identified candidate predictors provide useful information for clinicians and warrant consideration when developing or updating PMs using state-of-the-art development and validation techniques.

**PROSPERO registration number** CRD42022341233.

*   Respiratory infections
*   Primary Care
*   Prognosis
*   Systematic Review

### Strengths and limitations of this study

*   Both prognostic factors and prediction models were covered in this comprehensive systematic review.

*   The systematic search syntax included a validated prognostic search string to identify all relevant prognostic literature.

*   Prevailing guidelines were followed for data extraction (Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of prognostic factors studies (CHARMS) and CHARMS), quality assessment (Quality in Prognostic Studies tool, Prediction Model Risk Of Bias Assessment Tool and Grading of Recommendations, Assessment, Development and Evaluations) and reporting (Preferred Reporting Items for Systematic Reviews and Meta-Analyses).

*   Substantial between-study heterogeneity hampered the pooling of the results in a meta-analysis.

## Introduction

Acute lower respiratory tract infections (LRTI) are common in primary care.1 2 Uncomplicated LRTI episodes generally have a favourable natural course in otherwise healthy adults, and antibiotic treatment with amoxicillin confers only little benefit in terms of earlier symptom resolution, both overall and in higher-risk subgroups of patients.3–5 The clinical spectrum of LRTI patients, however, is heterogeneous in terms of patient and disease-specific characteristics, and thus the risk of hospitalisation and death varies substantially. For example, observational evidence suggests that adverse outcomes such as hospitalisation and death occur in fewer than 1% of patients with uncomplicated LRTI,5 whereas this risk is far more pronounced among those with community-acquired pneumonia (CAP).6 7 However, diagnosing CAP in a primary care setting is notoriously challenging,8 9 leaving general practitioners (GP) with uncertainties about how to identify higher-risk groups among those presenting with LRTI symptoms.

Nevertheless, identification of LRTI patients at the highest risk of complications is pivotal as this might help GPs identify those in whom close follow-up or early (antibiotic) treatment is warranted. Currently, several scores exist to assist physicians in estimating the risk of adverse outcomes in LRTI patients, such as the Pneumonia Severity Index (PSI) and CURB-65 (a prediction rule including confusion, blood urea nitrogen, respiratory rate, blood pressure and age ≥65 years).10 11 However, these prediction rules have been developed in secondary care and the inclusion of laboratory and/or radiographic features hampers applicability to outpatients. A modified version of the CURB-65 (CRB-65, not including blood urea nitrogen) has been proposed as primary care alternative, but its development in hospitalised patients raises uncertainty about its performance in outpatients.10 An overview of existing primary care-specific prognostic factors (PF) and prediction models (PM) for adverse outcomes in LRTI patients could provide insight on (1) existing PFs or PMs suitable for use in clinical practice and (2) relevant PFs to include when developing a new or updating an existing model. We, therefore, aimed to synthesise existing knowledge on PFs and PMs for hospitalisation and mortality within 90 days in adult patients presenting to primary care with LRTI.

## Methods

The protocol for this review was registered in PROSPERO and the review was reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses reporting guideline for systematic reviews.12

### Searches and study selection

The electronic databases of MEDLINE, Embase and the Cochrane Library were searched on 27 June 2022 for eligible studies. Keywords for ‘respiratory tract infection’, ‘hospitalisation’, ‘death’, ‘adults’ and ‘primary care’ were combined with a validated search string to identify relevant prognostic studies (online supplemental table S1).13 Eligibility criteria were based on PICOTS (patients, index, comparison, outcome, timing, and setting) guidance; we included all studies on adults with LRTI (patients) in which PF or PM (index) for 90-day hospitalisation or all-cause mortality (outcome) were assessed on the day of diagnosis (timing) in primary care (setting).14 15 Non-English articles or those without full-text availability were excluded.

### Supplementary data

[[bmjopen-2023-075475supp001.pdf]](pending:yes)

After de-duplication, two review authors (MHR and TMCvdB) independently reviewed titles/abstracts of unique records retrieved from the electronic databases against the eligibility criteria. Next, the same authors independently reviewed the full text of potentially relevant articles for eligibility. Any disagreements were resolved by discussion and, where necessary, by consulting a third reviewer (TNP). To increase the yield of potentially eligible articles, reference lists of the included articles were reviewed for eligibility. Studies assessing the prognostic value of PFs, developing a new PM, or externally validating a previously developed PM were eligible for inclusion.

### Data collection

Data from eligible PF and PM studies were extracted by MHR using standardised forms based on the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of PF studies (CHARMS-PF) and PM studies (CHARMS), respectively.14 16 The main outcomes of interest were hospitalisation or death within 90 days after the initial LRTI consultation. For PF studies we extracted ORs, HRs or risk ratios with accompanying 95% CIs from both univariable and multivariable analyses. For those studies that did not report such estimates, these associations were calculated from crude data using Rothman’s Episheet V.15 November 2021 wherever possible. For PM studies, model performance estimates in terms of discrimination (c statistic with accompanying 95% CI) and calibration (intercept and slope) were extracted. If calibration measures were absent, the total observed over expected (O/E) ratio with accompanying 95% CI of patients with the outcome was calculated wherever possible as a proxy of overall model calibration.15 To this end, we used data on the distribution of subjects and outcomes across different PM risk strata and the corresponding predicted risks of the strata as reported in the original model development study. These calculations were performed in R V.4.2.2 using the ‘metamisc’ package.

Other data extraction items included study population, candidate predictors, sample size, (handling of) missing data, analysis (modelling method and/or model development and validation method) and interpretation of results. Where needed, corresponding authors of individual studies were contacted to request additional data.

### Assessment of risk of bias and applicability

Risk of bias assessment was performed by two authors (MHR and TNP) independently using the Quality in Prognostic Studies tool (QUIPS, assessing study participation, study attrition, PF measurement, outcome measurement and statistical analysis) and the Prediction Model Risk Of Bias Assessment Tool (PROBAST, covering the domains of participants, predictors, outcome and analysis) for PF and PM studies, respectively.17 18 We excluded the ‘study confounding’ item from the QUIPS, since confounding is not applicable to prognostic research. The applicability of the included articles was assessed based on the PICOTS criteria.

### Synthesis methods

Due to substantial heterogeneity in domain, predictor and outcome definition, combining PF effect estimates or PM performance measures in meta-analyses was considered inappropriate and results were therefore descriptively summarised, and stratified according to diagnosis (ie, LRTI vs pneumonia). Forest plots were created to visually summarise (1) univariable and multivariable effect estimates of PFs, and (2) discrimination and calibration performance measures of PMs. PFs were categorised into either demographics, patient history, healthcare use, medication use, signs and symptoms, laboratory tests or diagnosis. Promising candidate predictors were identified based on reported results of multivariable analysis in PF studies. For each promising PF and PM, we applied the Grading of Recommendations, Assessment, Development and Evaluations framework using prognostic research-specific guidance to rate the quality of evidence.19 20

### Patient and public involvement

There was no patient or public involvement in this study.

## Results

### Study selection

The search strategy yielded a total of 3191 records. Removing duplicates left 2799 unique records (figure 1). After screening the title and abstract, 34 studies remained for full-text review. Of these, 19 were excluded (figure 1). One additional publication was retrieved from screening reference lists,21 leaving 16 publications for inclusion: 9 PF studies,21–29 6 PM studies30–35 and 1 combination of PM and PF.36

![Figure 1](http://bmjopen.bmj.com/https://bmjopen.bmj.com/content/bmjopen/14/3/e075475/F1.medium.gif)

[Figure 1](http://bmjopen.bmj.com/content/14/3/e075475/F1)

Figure 1 
Flow of articles through study. *Systematic reviews have been screened for eligible primary studies.

### Study characteristics

The PF studies were conducted between 1997 and 2021 of which most (7/10) used a retrospective cohort design (table 1). The study population in terms of age (minimum age ranging from 1 to 80 years) and LRTI definition (ranging from suspected LRTI to diagnosis of CAP), as well as the hospitalisation rate (from 0.7% in LRTI patients to 76.5% patients with CAP) varied substantially across studies. Mortality was used as an outcome in four studies, hospitalisation in three studies and a composite of both in one study. Two studies also included deterioration of existing illness and late-onset pneumonia in addition to hospitalisation and mortality in their outcome,23 36 while the timing of outcome assessment was unknown in one study.29

View this table:
[Table 1](http://bmjopen.bmj.com/content/14/3/e075475/T1)

Table 1 
Study characteristics of included prognostic factor studies

The PM studies were conducted between 2000 and 2013 and the majority (4/7) used a prospective cohort design. In total, nine different PMs could be included in the synthesis: CRB (CRB-65, not including age ≥65 years), CRB-65, CURB (CURB-65, not including age ≥65 years), CURB-65, PSI, a model developed by Bont *et al*, RISSC85 (a priori risk of poor outcome by country, inference in daily activities, number of years stopped smoking, severe sputum, presence of crackles, and diastolic blood pressure <85mmHg), CORB-75 (confusion, oxygen saturation ≤90%, respiratory rate ≥30/minute, systolic blood pressure ≤90mmHg or diastolic blood pressure ≤60mmHg, and age ≥75 years) and a model developed by Moore *et al* (table 2). Four studies developed a new PM30 32 35 36 and five externally validated previously developed models,31–35 of which the CRB-65 was validated in all five studies. Clinical heterogeneity was mostly attributed to age (minimum age ranging from 16 to 65 years) and LRTI definition (ranging from suspected LRTI to radiographically confirmed CAP). Hospitalisation rates varied between 0.5% in LRTI patients and 86.9% in patients with CAP. All PM studies used binary outcomes. Mortality was used as an outcome in three studies, hospitalisation in one study and a composite of both in one study. One study also included reconsultation with new or worsened symptoms in addition to hospitalisation in their outcome,35 while another study included late-onset pneumonia in addition to hospitalisation and mortality.36

View this table:
[Table 2](http://bmjopen.bmj.com/content/14/3/e075475/T2)

Table 2 
Study characteristics of included prediction model studies

### Risk of bias assessment

Overall risk of bias was judged high for all studies (online supplemental figures S2 and S3). For PF studies, this was mostly due to issues in the statistical analysis domains. For instance, five studies selected predictors for multivariable analysis based on univariable associations,21–25 and two included multiple episodes per patient but did not perform recurrent event analysis.25 27 Furthermore, in some studies, an adequate definition of LRTI or PFs was lacking, and methods to account for missing data were mostly judged inadequate or not described.

Similarly, the risk of bias due to the issues in the analysis domain was judged high for all PM studies. In all model development studies, continuous predictors were dichotomised, and handling of missing data was judged inadequate or not reported. None of the PM studies reported any evaluation of model calibration.

### Prognostic factors

Three PF studies did not report estimates of univariable associations of PFs, and these could also not be calculated from the reported data.24 27 28 The remaining PF studies reported univariable analysis of 69 PFs, of which sex, age and history of pulmonary disease were analysed most frequently (online supplemental figure S4). If reported in the primary study, absolute risks according to the presence and absence of individual PFs per study are presented in online supplemental table S5.

Estimates of multivariable associations were reported for 74 PFs (online supplemental figure S6), of which 21 were analysed in at least two studies. The number of covariates per multivariable analysis ranged from 3 to 29, and a common set of covariates across studies was lacking. Age was analysed in five studies, and all found increasing age to be associated with poor outcome.23 24 27 28 36 Apart from increasing age, PFs that were identified as promising based on multiple multivariable analyses were sex (significant association in two/five studies), current smoking (three/four studies), diabetes (two/three studies), a history of stroke (two/three studies), cancer (three/three studies) or heart failure (two/two studies), hospitalisation in the previous year (two/two studies), current use of systemic corticosteroids (two/three studies), antibiotic use in the previous month (two/three studies), a respiratory rate ≥25/min (two/two studies) and diagnosis of pneumonia (two/two studies) (figure 2). Having received an influenza vaccination in the previous year was the only promising candidate predictor estimated to have a negative association, based on a significant multivariable association in two/three studies. The quality of evidence was judged low for age, hospitalisation in the previous year and diagnosis of pneumonia and very low for the remaining promising PFs (online supplemental table S7).

![Figure 2](http://bmjopen.bmj.com/https://bmjopen.bmj.com/content/bmjopen/14/3/e075475/F2.medium.gif)

[Figure 2](http://bmjopen.bmj.com/content/14/3/e075475/F2)

Figure 2 
Overview of most promising candidate predictors for hospitalisation or mortality within 90 days based on multivariable analysis. Blue coloured points: studies on LRTI patients. Orange coloured points: studies on patients with pneumonia. Quality of evidence: as judged based on the Grading of Recommendations, Assessment, Development and Evaluations framework. *Composite outcome also includes dysregulation of diabetes, stroke, heart failure, myocardial infarction. **Composite outcome also includes late-onset pneumonia. CAP, community-acquired pneumonia; H, hospitalisation; LRTI, lower respiratory tract infection; M, mortality; min, minute; RR, risk ratio.

### Prediction models

Several PMs included predictors that were also identified as promising candidate predictors based on PF studies (online supplemental table S8). Sex, current smoking and influenza vaccination were the only promising candidate predictors that were not incorporated in any of the PMs. Predictors that were not identified as promising PFs but were incorporated in PMs include the history of lung, renal or liver disease, the extent to which issues interfere with daily activities, sputum or coryza, chest pain, confusion, presence of crackles, heart rate, blood pressure, oxygen saturation, body temperature, blood urea nitrogen and an overall GP severity assessment score.

Four studies developed a new PM (three for use in LRTI and one for patients with CAP), in which the number of events per predictor ranged from 3.3 to 13.7.30 34–36 C statistics ranged from 0.63 (95% CI 0.61 to 0.67) to 0.82 (95% CI 0.75 to 0.88) after (internal or external) model validation (figure 3). One study performed external validation,30 whereas other studies only conducted internal(-external) validation.34–36 None reported measures of model calibration, while a total O/E ratio could not be calculated for any model.

![Figure 3](http://bmjopen.bmj.com/https://bmjopen.bmj.com/content/bmjopen/14/3/e075475/F3.medium.gif)

[Figure 3](http://bmjopen.bmj.com/content/14/3/e075475/F3)

Figure 3 
Overview of discrimination and calibration performance measures of the included prediction models. (A) Performance of prediction models evaluated in LRTI patients. (B) Performance of prediction models evaluated in community-acquired pneumonia patients. CAP, community-acquired pneumonia; CORB-75, confusion, oxygen saturation ≤90%, respiratory rate ≥30/minute, and systolic blood pressure ≤90mmHg or diastolic blood pressure ≤60mmHg, and age ≥75 years; CRB(-65), confusion, respiratory rate ≥30/minute, systiolic blood pressure ≤90mmHg or diastolic blood pressure ≤60mmHg, and age ≥65 years (and age ≥65 years); CURB(-65), confusion, blood urea nitrogen >7 millimole/liter, respiratory rate ≥30/minute, systiolic blood pressure ≤90mmHg or diastolic blood pressure ≤60mmHg (and age ≥65 years); E, external; H, hospitalisation; I, internal; IE, internal external; LoP, late-onset pneumonia; LRTI, lower respiratory tract infection; M, mortality; O/E, observed over expected; PSI, Pneumonia Severity Index; rC, reconsultation; RISSC85, a priori risk of poor outcome by country, inference in daily activities, number of years stopped smoking, severe sputum, presence of crackles, and diastolic blood pressure <85mmHg.

All five PM validation studies externally validated the CRB-65.31–35 C statistics for 30-day mortality in patients with CAP (range outcome events: 11–80) ranged from 0.72 (95% CI 0.66 to 0.78) to 0.79 (95% CI 0.65 to 0.92)31 32 34 (figure 3). The c statistic for 28-day reconsultation or hospitalisation in LRTI patients was substantially lower: 0.53 (95% CI 0.51 to 0.56).35 None reported measures of model calibration. A total O/E ratio could be calculated for the CRB-65 in four studies, ranging from 0.67 (95% CI 0.26 to 1.08) to 1.42 (95% CI 1.13 to 1.71).

Other validated PMs were PSI, CURB-65, CURB and CRB. C statistics for 28-day reconsultation or hospitalisation in LRTI patients ranged from 0.51 (95% CI 0.50 to 0.54) to 0.53 (95% CI 0.50 to 0.55).35 Calibration measures were not reported and total O/E ratios could not be calculated. C statistics of PSI and CURB-65 for 30-day mortality in patients with CAP were 0.73 (95% CI 0.67 to 0.79) and 0.67 (95% CI 0.61 to 0.74), respectively.32 Model calibration measures were not reported, while the total O/E ratios were estimated at 1.46 (95% CI 1.16 to 1.76) and 1.92 (95% CI 1.53 to 2.32), respectively. The quality of evidence was judged low for CRB-65 and very low for all other PMs included in this systematic review (online supplemental table S9).

## Discussion

### Summary of main findings

This systematic review summarised evidence on PFs and PMs for hospitalisation and mortality within 90 days in patients presenting to primary care with LRTI. The risk of bias in included studies was judged high, mainly due to issues in the analysis domains. Of all PFs analysed, 13 were identified as most promising; increasing age, sex, current smoking, diabetes, a history of stroke, cancer or heart failure, hospitalisation in the previous year, current use of systemic corticosteroids, antibiotic use in the previous month, a respiratory rate ≥25/min and diagnosis of pneumonia were associated with poor outcome whereas seasonal influenza vaccination was estimated to have a negative association. Most of these promising candidate predictors have also been incorporated into the identified PMs. The secondary care-derived CRB-65, predicting mortality in patients with CAP and a newly developed PM by Bont *et al*30 predicting hospitalisation and mortality in elderly LRTI patients both showed promising results after external validation. However, none assessed model calibration, leaving the precision of predicted risks unknown.

### Comparison with existing literature

During the coronavirus disease 2019 (COVID-19) pandemic, many studies have focused on PFs associated with poor outcomes in patients with COVID-19, and some were similar to those identified in our review of all-cause LRTI patients. A recent, field-wide systematic review and meta-analysis of PFs for adverse outcomes in patients with COVID-19 found cancer, insulin use and smoking—among other factors—to be associated with mortality and age with both hospitalisation and mortality.37 Another community-based systematic review of patients with European COVID-19 found male sex, heart failure and diabetes to be associated with hospitalisation, whereas stroke, heart failure and neurological disease were associated with increased mortality risk.38 Although we did not identify studies that assessed the prognostic value of specific viral respiratory pathogens such as SARS-CoV-2, respiratory syncytial virus and influenza, it is likely that the specific pathogen is a relevant PF and point-of-care tests for respiratory pathogens are increasingly available in the post-pandemic era. The potential usefulness of such point-of-care tests is supported by evidence of patients with a viral LRTI admitted to the emergency room in whom specific viral pathogens were associated with an increased risk of hospitalisation.39

Some of the identified PMs have been incorporated in LRTI guidelines—mainly those focusing on CAP—as additional tools to aid patient management decisions. The CAP guideline issued by the American Thoracic Society (ATS) and the Infectious Diseases Society of America (IDSA) supports the use of PSI or CURB-65—in addition to clinical judgement—to determine the appropriate level of care, that is, inpatient versus outpatient management.40 41 Although this guideline applies to all levels of healthcare, the value of the suggested models for GPs is limited since the models’ laboratory and radiographic features are not routinely available in primary care. The UK National Institute for Health and Care Excellence (NICE) guideline on pneumonia advocates the use of CRB-65; it recommends that GPs consider hospital admission in patients with a score of ≥1.42 Guidelines vary with regard to the incorporation of individual factors that are associated with poor outcome. The ATS/IDSA guideline and the Dutch primary care guideline both list tachypnoea and confusion as indicators of disease severity, while the Dutch guideline also emphasised the importance of PFs such as age, history of heart failure and diabetes.40 43 The NICE guideline does not explicitly highlight individual factors associated with poor outcome in adult patients.42

Similar to our findings, a review of the performance of CRB-65 for predicting 30-day mortality among patients with CAP in community and hospital settings concluded that this model has been insufficiently validated for use in primary care.44 Although the population of this review slightly differed from ours by also including patients who visited the emergency department, a similarly low number of outcome events was found. Contrary to the slight overprediction reported in this review, our synthesis of more recent studies suggested underprediction of risks based on total O/E ratios. However, in the absence of a proper assessment of model calibration, the true precision of predicted risks remains to be elucidated.

### Strengths and limitations

By covering both PFs and PMs in a single review, we have provided a comprehensive overview of available prognostic literature on primary care LRTI patients. We prospectively registered our review protocol, conducted a broad search with the incorporation of a validated prognostic search string and thoroughly reviewed citations of eligible studies to assure all relevant literature was included in the synthesis. By including various prognostic study designs, our synthesis provides a comprehensive overview of all prognostic evidence on primary care LRTI patients. Studies assessing the prognostic value of individual factors, and developing a new or validating an existing model all provide insight on important PFs that could be used in future prognostic research, whereas studies validating a new or previously developed model such as CRB-65 provide insight on the suitability for its use in clinical practice.

Some limitations should, however, be acknowledged. First, our findings should be interpreted with caution since the risk of bias was judged high for all included studies, and the quality of the evidence was judged low to very low for the promising PFs and PMs. Second, due to substantial heterogeneity in domain, predictor and outcome definitions between studies, we refrained from conducting meta-analyses of PF effect estimates or PM performance measures, which makes direct comparison of study results difficult. It is plausible that PF effect estimates for poor outcome vary according to age, diagnosis and disease severity (a priori risk of event). For example, half of the participants in Hak *et al*23 had either pneumonia or an exacerbation of chronic obstructive pulmonary disease, whereas Moore *et al*36 had fewer than 2% of such patients. Furthermore, one study included patients aged >1 year (median age of included participants: 54 years (IQR: 37)) but did not specifically report separate results for age strata.24 Given the median age, we considered including this study on PF in our review justified. Consistency in effect estimates would have further facilitated direct comparison of PFs, but converting the various reported estimates to ORs appeared not possible. Although our synthesis of PFs provides some useful information for clinicians, it merely informs future research aimed at developing or updating PMs. Third, we excluded most studies on patients with a specific respiratory pathogen (eg, SARS-CoV-2 or influenza) since these included patients with both upper and lower respiratory tract infections and did not report stratified results for LRTI. As such, we were unable to present any respiratory pathogen-specific results regarding PF for poor outcome in patients presenting to primary care with LRTI. However, since GPs are typically unaware of the specific pathogen in LRTI patients, such results would have limited applicability to current clinical practice. Lastly, PM building techniques were generally suboptimal which increased risk of bias, and none of the identified PM studies assessed model calibration. In contrast to discrimination, calibration generally receives little attention when assessing PM performance, whereas well-discriminating but poorly calibrated models can lead to imprecise, misleading predictions.45 Total O/E ratios calculations provided some insight into mean model calibration—the most basic level in the hierarchy of calibration assessment—but this is insufficient to provide guidance on the safe use of a PM in clinical practice.46

### Implications for research and clinical practice

Based on our synthesis, implementation of existing PMs for individualised risk prediction of 90-day hospitalisation or all-cause mortality in primary care LRTI patients in everyday practice is hampered by incomplete assessment of model performance. The identified candidate predictors provide useful information for clinicians about factors associated with poor outcome in primary care LRTI patients and warrant consideration when developing or updating PMs using state-of-the-art development and appropriate validation techniques.

### Supplementary data

[[bmjopen-2023-075475supp002.pdf]](pending:yes)

## Data availability statement

Data are available upon reasonable request. Study materials (eg, template data collection forms, data extracted from included studies) are available from the corresponding author upon reasonable request.

## Ethics statements

### Patient consent for publication

Not applicable.

### Ethics approval

Not applicable.

## Footnotes

*   Twitter @TamaraPlatteel1, @gjgeersing

*   MvS and RPV contributed equally.

*   Contributors MHR, TNP, MvS and RV were involved in the conceptualisation and design of the study. MHR, TMCvdB and TNP conducted the study selection procedure. Data of eligible studies was extracted by MHR. MHR, TNP and MvS performed the risk of bias assessment. MHR wrote the first draft of the manuscript, which was initially reviewed by TNP, MvS and RV. MHR, TNP, TMCvdB, G-JG, PL, FHR, MvS and RV reviewed and approved the final manuscript. As guarantor, MHR declares that this manuscript is an honest, accurate, and transparant account of the study, and accepts full responsibility for its conduct and reporting.

*   Funding This study was funded by ZonMw (grant number 08391052110003). The funder played no role in study design, data collection, analysis and interpretation of data, or the writing of this manuscript.

*   Competing interests None declared.

*   Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

*   Provenance and peer review Not commissioned; externally peer reviewed.

*   Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

[https://creativecommons.org/licenses/by/4.0/](https://creativecommons.org/licenses/by/4.0/)
This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: [https://creativecommons.org/licenses/by/4.0/](https://creativecommons.org/licenses/by/4.0/).

## References

1.  Bergmann M, Haasenritter J, Beidatsch D, et al. Prevalence, aetiologies and prognosis of the symptom cough in primary care: a systematic review and meta-analysis. BMC Fam Pract 2021;22:151. [doi:10.1186/s12875-021-01501-0](http://dx.doi.org/10.1186/s12875-021-01501-0)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1186/s12875-021-01501-0&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=34253179&link_type=MED&atom=%2Fbmjopen%2F14%2F3%2Fe075475.atom) 

2.  Hak E, Rovers MM, Kuyvenhoven MM, et al. Incidence of GP-diagnosed respiratory tract infections according to age, gender and high-risk co-morbidity: the second Dutch national survey of general practice. Fam Pract 2006;23:291–4. [doi:10.1093/fampra/cmi121](http://dx.doi.org/10.1093/fampra/cmi121)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1093/fampra/cmi121&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=16464869&link_type=MED&atom=%2Fbmjopen%2F14%2F3%2Fe075475.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000237931600006&link_type=ISI) 

3.  Little P, Stuart B, Moore M, et al. Amoxicillin for acute lower-respiratory-tract infection in primary care when pneumonia is not suspected: a 12-country, randomised, placebo-controlled trial. Lancet Infect Dis 2013;13:123–9. [doi:10.1016/S1473-3099(12)70300-6](http://dx.doi.org/10.1016/S1473-3099(12)70300-6)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/S1473-3099(12)70300-6&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=23265995&link_type=MED&atom=%2Fbmjopen%2F14%2F3%2Fe075475.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000313921900029&link_type=ISI) 

4.  Moore M, Stuart B, Coenen S, et al. Amoxicillin for acute lower respiratory tract infection in primary care: subgroup analysis of potential high-risk groups. Br J Gen Pract 2014;64:e75–80. [doi:10.3399/bjgp14X677121](http://dx.doi.org/10.3399/bjgp14X677121)
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYmpncCI7czo1OiJyZXNpZCI7czoxMDoiNjQvNjE5L2U3NSI7czo0OiJhdG9tIjtzOjI2OiIvYm1qb3Blbi8xNC8zL2UwNzU0NzUuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

5.  Little P, Stuart B, Smith S, et al. Antibiotic prescription strategies and adverse outcome for uncomplicated lower respiratory tract infections: prospective cough complication cohort (3C) study. BMJ 2017;357. [doi:10.1136/bmj.j2148](http://dx.doi.org/10.1136/bmj.j2148)
    
    

6.  Welte T, Torres A, Nathwani D. Clinical and economic burden of community-acquired pneumonia among adults in Europe. Thorax 2012;67:71–9. [doi:10.1136/thx.2009.129502](http://dx.doi.org/10.1136/thx.2009.129502)
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToidGhvcmF4am5sIjtzOjU6InJlc2lkIjtzOjc6IjY3LzEvNzEiO3M6NDoiYXRvbSI7czoyNjoiL2Jtam9wZW4vMTQvMy9lMDc1NDc1LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

7.  McLaughlin JM, Khan FL, Thoburn EA, et al. Rates of hospitalization for community-acquired pneumonia among US adults: a systematic review. Vaccine 2020;38:741–51. [doi:10.1016/j.vaccine.2019.10.101](http://dx.doi.org/10.1016/j.vaccine.2019.10.101)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.vaccine.2019.10.101&link_type=DOI) 

8.  Teepe J, Broekhuizen BDL, Loens K, et al. Predicting the presence of bacterial pathogens in the Airways of primary care patients with acute cough. CMAJ 2017;189:E50–5. [doi:10.1503/cmaj.151364](http://dx.doi.org/10.1503/cmaj.151364)
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czo5OiIxODkvMi9FNTAiO3M6NDoiYXRvbSI7czoyNjoiL2Jtam9wZW4vMTQvMy9lMDc1NDc1LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

9.  Minnaard MC, de Groot JAH, Hopstaken RM, et al. The added value of C-reactive protein measurement in diagnosing pneumonia in primary care: a meta-analysis of individual patient data. CMAJ 2017;189:E56–63. [doi:10.1503/cmaj.151163](http://dx.doi.org/10.1503/cmaj.151163)
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czo5OiIxODkvMi9FNTYiO3M6NDoiYXRvbSI7czoyNjoiL2Jtam9wZW4vMTQvMy9lMDc1NDc1LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

10. Lim WS, van der Eerden MM, Laing R, et al. Defining community acquired pneumonia severity on presentation to hospital: an international derivation and validation study. Thorax 2003;58:377–82. [doi:10.1136/thorax.58.5.377](http://dx.doi.org/10.1136/thorax.58.5.377)
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToidGhvcmF4am5sIjtzOjU6InJlc2lkIjtzOjg6IjU4LzUvMzc3IjtzOjQ6ImF0b20iO3M6MjY6Ii9ibWpvcGVuLzE0LzMvZTA3NTQ3NS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

11. Fine MJ, Auble TE, Yealy DM, et al. A prediction rule to identify low-risk patients with community-acquired pneumonia. N Engl J Med 1997;336:243–50. [doi:10.1056/NEJM199701233360402](http://dx.doi.org/10.1056/NEJM199701233360402)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1056/NEJM199701233360402&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=8995086&link_type=MED&atom=%2Fbmjopen%2F14%2F3%2Fe075475.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=A1997WE63200002&link_type=ISI) 

12. Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71. [doi:10.1136/bmj.n71](http://dx.doi.org/10.1136/bmj.n71)
    
    

13. Geersing G-J, Bouwmeester W, Zuithoff P, et al. Search filters for finding prognostic and diagnostic prediction studies in MEDLINE to enhance systematic reviews. PLoS One 2012;7:e32844. [doi:10.1371/journal.pone.0032844](http://dx.doi.org/10.1371/journal.pone.0032844)
    
    

14. Moynihan R, Henry D, Moons KGM. Using evidence to combat overdiagnosis and overtreatment: evaluating treatments, tests, and disease definitions in the time of too much. PLoS Med 2014;11:e1001655. [doi:10.1371/journal.pmed.1001655](http://dx.doi.org/10.1371/journal.pmed.1001655)
    
    

15. Debray TPA, Damen JAAG, Snell KIE, et al. A guide to systematic review and meta-analysis of prediction model performance. BMJ 2017;356:i6460. [doi:10.1136/bmj.i6460](http://dx.doi.org/10.1136/bmj.i6460)
    
    

16. Riley RD, Moons KGM, Snell KIE, et al. A guide to systematic review and meta-analysis of Prognostic factor studies. BMJ 2019;364. [doi:10.1136/bmj.k4597](http://dx.doi.org/10.1136/bmj.k4597)
    
    

17. Hayden JA, van der Windt DA, Cartwright JL, et al. Assessing bias in studies of prognostic factors. Ann Intern Med 2013;158:280–6. [doi:10.7326/0003-4819-158-4-201302190-00009](http://dx.doi.org/10.7326/0003-4819-158-4-201302190-00009)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.7326/0003-4819-158-4-201302190-00009&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=23420236&link_type=MED&atom=%2Fbmjopen%2F14%2F3%2Fe075475.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000315580300019&link_type=ISI) 

18. Wolff RF, Moons KGM, Riley RD, et al. PROBAST: a tool to assess the risk of bias and applicability of prediction model studies. Ann Intern Med 2019;170:51–8. [doi:10.7326/M18-1376](http://dx.doi.org/10.7326/M18-1376)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.7326/M18-1376&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=30596875&link_type=MED&atom=%2Fbmjopen%2F14%2F3%2Fe075475.atom) 

19. Foroutan F, Guyatt G, Zuk V, et al. GRADE guidelines 28: use of GRADE for the assessment of evidence about prognostic factors: rating certainty in identification of groups of patients with different absolute risks. J Clin Epidemiol 2020;121:62–70. [doi:10.1016/j.jclinepi.2019.12.023](http://dx.doi.org/10.1016/j.jclinepi.2019.12.023)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.jclinepi.2019.12.023&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjopen%2F14%2F3%2Fe075475.atom) 

20. Brozek JL, Canelo-Aybar C, Akl EA, et al. GRADE guidelines 30: the GRADE approach to assessing the certainty of modeled evidence—an overview in the context of health decision-making. J Clin Epidemiol 2021;129:138–50. [doi:10.1016/j.jclinepi.2020.09.018](http://dx.doi.org/10.1016/j.jclinepi.2020.09.018)
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjopen%2F14%2F3%2Fe075475.atom) 

21. Houston MS, Silverstein MD, Suman VJ. Risk factors for 30-day mortality in elderly patients with lower respiratory tract infection: community-based study. Arch Intern Med 1997;157:2190–5.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1001/archinte.1997.00440400040005&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=9342995&link_type=MED&atom=%2Fbmjopen%2F14%2F3%2Fe075475.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=A1997YC00900004&link_type=ISI) 

22. Seppä Y, Bloigu A, Honkanen PO, et al. Severity assessment of lower respiratory tract infection in elderly patients in primary care. Arch Intern Med 2001;161:2709–13. [doi:10.1001/archinte.161.22.2709](http://dx.doi.org/10.1001/archinte.161.22.2709)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1001/archinte.161.22.2709&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=11732936&link_type=MED&atom=%2Fbmjopen%2F14%2F3%2Fe075475.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000172449000009&link_type=ISI) 

23. Hak E, Bont J, Hoes AW, et al. Prognostic factors for serious morbidity and mortality from community-acquired lower respiratory tract infections among the elderly in primary care. Fam Pract 2005;22:375–80. [doi:10.1093/fampra/cmi020](http://dx.doi.org/10.1093/fampra/cmi020)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1093/fampra/cmi020&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=15805127&link_type=MED&atom=%2Fbmjopen%2F14%2F3%2Fe075475.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000230513600005&link_type=ISI) 

24. Winchester CC, Macfarlane TV, Thomas M, et al. Antibiotic prescribing and outcomes of lower respiratory tract infection in UK primary care. Chest 2009;135:1163–72. [doi:10.1378/chest.07-2940](http://dx.doi.org/10.1378/chest.07-2940)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1378/chest.07-2940&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=19420194&link_type=MED&atom=%2Fbmjopen%2F14%2F3%2Fe075475.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000265876100010&link_type=ISI) 

25. van de Nadort C, Smeets HM, Bont J, et al. Prognosis of primary care patients aged 80 years and older with lower respiratory tract infection. Br J Gen Pract 2009;59:e110–5. [doi:10.3399/bjgp09X420239](http://dx.doi.org/10.3399/bjgp09X420239)
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYmpncCI7czo1OiJyZXNpZCI7czoxMToiNTkvNTYxL2UxMTAiO3M6NDoiYXRvbSI7czoyNjoiL2Jtam9wZW4vMTQvMy9lMDc1NDc1LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

26. Myles PR, Hubbard RB, Gibson JE, et al. The impact of statins, ace inhibitors and gastric acid suppressants on pneumonia mortality in a UK general practice population cohort. Pharmacoepidemiol Drug Saf 2009;18:697–703. [doi:10.1002/pds.1769](http://dx.doi.org/10.1002/pds.1769)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1002/pds.1769&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=19455553&link_type=MED&atom=%2Fbmjopen%2F14%2F3%2Fe075475.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000269009600007&link_type=ISI) 

27. Millett ERC, De Stavola BL, Quint JK, et al. Risk factors for hospital admission in the 28 days following a community-acquired pneumonia diagnosis in older adults, and their contribution to increasing hospitalisation rates over time: a cohort study. BMJ Open 2015;5:e008737. [doi:10.1136/bmjopen-2015-008737](http://dx.doi.org/10.1136/bmjopen-2015-008737)
    
    

28. Hamilton F, Arnold D, Payne R. Association of prior lymphopenia with mortality in pneumonia: a cohort study in UK primary care. Br J Gen Pract 2021;71:e148–56. [doi:10.3399/bjgp20X713981](http://dx.doi.org/10.3399/bjgp20X713981)
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYmpncCI7czo1OiJyZXNpZCI7czoxMToiNzEvNzAzL2UxNDgiO3M6NDoiYXRvbSI7czoyNjoiL2Jtam9wZW4vMTQvMy9lMDc1NDc1LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

29. Martínez-Redondo J, Comas C, Pujol Salud J, et al. The risk of hospitalization in COVID-19 patients can be predicted by lung ultrasound in primary care. Int J Environ Res Public Health 2021;18. [doi:10.3390/ijerph18116083](http://dx.doi.org/10.3390/ijerph18116083)
    
    

30. Bont J, Hak E, Hoes AW, et al. A prediction rule for elderly primary-care patients with lower respiratory tract infections. Eur Respir J 2007;29:969–75. [doi:10.1183/09031936.00129706](http://dx.doi.org/10.1183/09031936.00129706)
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiZXJqIjtzOjU6InJlc2lkIjtzOjg6IjI5LzUvOTY5IjtzOjQ6ImF0b20iO3M6MjY6Ii9ibWpvcGVuLzE0LzMvZTA3NTQ3NS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

31. Bont J, Hak E, Hoes AW, et al. Predicting death in elderly patients with community-acquired pneumonia: a prospective validation study reevaluating the CRB-65 severity assessment tool. Arch Intern Med 2008;168:1465–8. [doi:10.1001/archinte.168.13.1465](http://dx.doi.org/10.1001/archinte.168.13.1465)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1001/archinte.168.13.1465&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=18625928&link_type=MED&atom=%2Fbmjopen%2F14%2F3%2Fe075475.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000257595100014&link_type=ISI) 

32. Ochoa-Gondar O, Vila-Corcoles A, Rodriguez-Blanco T, et al. Comparison of three predictive rules for assessing severity in elderly patients with CAP. Int J Clin Pract 2011;65:1165–72. [doi:10.1111/j.1742-1241.2011.02742.x](http://dx.doi.org/10.1111/j.1742-1241.2011.02742.x)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1111/j.1742-1241.2011.02742.x&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=21951687&link_type=MED&atom=%2Fbmjopen%2F14%2F3%2Fe075475.atom) 

33. Francis NA, Cals JW, Butler CC, et al. Severity assessment for lower respiratory tract infections: potential use and validity of the CRB-65 in primary care. Prim Care Respir J 2012;21:65–70. [doi:10.4104/pcrj.2011.00083](http://dx.doi.org/10.4104/pcrj.2011.00083)
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=21938349&link_type=MED&atom=%2Fbmjopen%2F14%2F3%2Fe075475.atom) 

34. Ochoa-Gondar O, Vila-Corcoles A, Rodriguez-Blanco T, et al. Validation of the Corb75 (confusion, oxygen saturation, respiratory rate, blood pressure, and age ≥75 years) as a simpler pneumonia severity rule. Infection 2014;42:371–8. [doi:10.1007/s15010-013-0565-1](http://dx.doi.org/10.1007/s15010-013-0565-1)
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjopen%2F14%2F3%2Fe075475.atom) 

35. Bruyndonckx R, Hens N, Verheij TJ, et al. Development of a prediction tool for patients presenting with acute cough in primary care: a prognostic study spanning six European countries. Br J Gen Pract 2018;68:e342–50. [doi:10.3399/bjgp18X695789](http://dx.doi.org/10.3399/bjgp18X695789)
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYmpncCI7czo1OiJyZXNpZCI7czoxMToiNjgvNjcwL2UzNDIiO3M6NDoiYXRvbSI7czoyNjoiL2Jtam9wZW4vMTQvMy9lMDc1NDc1LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

36. Moore M, Stuart B, Lown M, et al. Predictors of adverse outcomes in uncomplicated lower respiratory tract infections. Ann Fam Med 2019;17:231–8. [doi:10.1370/afm.2386](http://dx.doi.org/10.1370/afm.2386)
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6ODoiMTcvMy8yMzEiO3M6NDoiYXRvbSI7czoyNjoiL2Jtam9wZW4vMTQvMy9lMDc1NDc1LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

37. Bellou V, Tzoulaki I, van Smeden M, et al. Prognostic factors for adverse outcomes in patients with COVID-19: a field-wide systematic review and meta-analysis. Eur Respir J 2022;59. [doi:10.1183/13993003.02964-2020](http://dx.doi.org/10.1183/13993003.02964-2020)
    
    

38. Vardavas CI, Mathioudakis AG, Nikitara K, et al. Prognostic factors for mortality, intensive care unit and hospital admission due to SARS-Cov-2: a systematic review and meta-analysis of cohort studies in Europe. Eur Respir Rev 2022;31. [doi:10.1183/16000617.0098-2022](http://dx.doi.org/10.1183/16000617.0098-2022)
    
    

39. Lemarie B, Boussaid G, Gault E, et al. Predictors of hospitalization and superinfection in viral respiratory tract infections between influenza and Paramyxoviruses: the SUPERFLUOUS study. J Infect Dis 2022;226:1027–35. [doi:10.1093/infdis/jiab525](http://dx.doi.org/10.1093/infdis/jiab525)
    
    

40. Metlay JP, Waterer GW, Long AC, et al. Diagnosis and treatment of adults with community-acquired pneumonia. Am J Respir Crit Care Med 2019;200:E45–67. [doi:10.1164/rccm.201908-1581ST](http://dx.doi.org/10.1164/rccm.201908-1581ST)
    
    

41. Armstrong CAFP. Community-acquired pneumonia: updated recommendations from the ATS and IDSA. Am Fam Physician 2020;102:121–4.
    
    

42. National Institute for Health and Care Excellence (NICE). Pneumonia in adults: diagnosis and management (clinical guideline 191). 2014. Available: [www.nice.org.uk/guidance/cg191](http://www.nice.org.uk/guidance/cg191)
    
    

43. Butler CC, van der Velden AW, Bongard E, et al. Oseltamivir plus usual care versus usual care for influenza-like illness in primary care: an open-label, pragmatic, randomised controlled trial. Lancet 2020;395:42–52. [doi:10.1016/S0140-6736(19)32982-4](http://dx.doi.org/10.1016/S0140-6736(19)32982-4)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/S0140-6736(19)32982-4&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=31839279&link_type=MED&atom=%2Fbmjopen%2F14%2F3%2Fe075475.atom) 

44. McNally M, Curtain J, O’Brien KK, et al. Validity of British Thoracic society guidance (the CRB-65 rule) for predicting the severity of pneumonia in general practice: systematic review and meta-analysis. Br J Gen Pract 2010;60:e423–33. [doi:10.3399/bjgp10X532422](http://dx.doi.org/10.3399/bjgp10X532422)
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYmpncCI7czo1OiJyZXNpZCI7czoxMToiNjAvNTc5L2U0MjMiO3M6NDoiYXRvbSI7czoyNjoiL2Jtam9wZW4vMTQvMy9lMDc1NDc1LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

45. Van Calster B, McLernon DJ, van Smeden M, et al. Calibration: the Achilles heel of predictive analytics. BMC Med 2019;17:230. [doi:10.1186/s12916-019-1466-7](http://dx.doi.org/10.1186/s12916-019-1466-7)
    
    

46. Van Calster B, Nieboer D, Vergouwe Y, et al. A calibration hierarchy for risk models was defined: from Utopia to empirical data. J Clin Epidemiol 2016;74:167–76. [doi:10.1016/j.jclinepi.2015.12.005](http://dx.doi.org/10.1016/j.jclinepi.2015.12.005)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.jclinepi.2015.12.005&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=26772608&link_type=MED&atom=%2Fbmjopen%2F14%2F3%2Fe075475.atom)