Seeing the wood and the trees: the impact of the healthcare system on variation in primary care referrals
=========================================================================================================
* Thomas Round
* Gary Abel
* accreditation
* anaesthesia
* attitudes
It is generally accepted that quality improvement efforts in health service delivery are best guided by measurement of performance and activity. Because of this, there are thousands of metrics and indicators produced worldwide attempting to capture the processes performed by, and outcomes achieved by, healthcare providers. However, for any of these indicators to be most useful, the variability in the indicator should reflect the performance or activity of the organisation being profiled. For example, it has been shown that much of the variation in doctor communication skills and in non-steroidal anti-inflammatory drug (NSAID) prescribing can be attributed to individual physicians rather than the organisations they work for.1 2 Those are examples of variation being attributed to a lower level than that at which it is often measured—that is, performance measures for inappropriate NSAID prescriptions often focus on practices, even though variation may occur predominantly at the individual prescriber level.3
Yet variations in performance may also reflect features a level up from the typical performance measures. This is the issue addressed in this issue of *BMJ Quality & Safety* by Burton and colleagues4 in their analysis of UK data on fast-track referrals for suspected cancer from general practices. In this example, they present an analysis of how much variation in general practitioner (GP) referrals for suspected cancer is attributable to local health services rather than practices or their populations.
Referral pathways for suspected cancer have been available to GPs in England since 2000. These pathways enable rapid access to a specialist opinion or diagnostic test within 2 weeks (ie, 2-week wait (2WW) referrals) for patients with specified symptoms based on referral criteria defined by the National Institute for Health and Care Excellence guidelines.5 Since 2009/2010 Public Health England (PHE) has publicly reported data6 on the use of these pathways. These data include for each general practice in England the standardised referral ratio, cancer detection rate (ie, sensitivity of referral) and conversion rate (positive predictive value or PPV of referral). There is substantial variation in referral use between general practices, which has been associated with mortality of patients with cancer.7
2WW referrals have continued to increase by 10% year on year, and are now running at over 2 million referrals per year in England, with significant cost implications. The total number of cancers detected via urgent referral has increased, with a concurrent decrease in those diagnosed via an emergency presentation to health services8 with worse outcomes.
At a practice level, urgent referral metrics for a single year can be based on relatively small numbers of referrals and cancer cases. The average full-time GP typically makes 50–60 2WW referrals, and having eight to nine patients newly diagnosed with cancer per year. These metrics exhibit year-on-year random variation,9 with previous work focusing on the potential primary care drivers of this including differences in random variation and therefore case-mix,10 referral selection accuracy and thresholds,11 12 practice and GP characteristics.13 14
The study by Burton *et* *al* 4 is the first to analyse these data at the levels of general practice, primary care organisation (206 clinical commissioning groups or CCG) and secondary care provider (126 acute hospital trusts) levels. They found that primary care organisations were associated with substantial variation in both the volume of referrals and in the effectiveness of those referrals in capturing cancer cases. Specifically, 21% of the variation between general practices in the standardised 2WW referral ratio, and 42% of the variation between general practices in the cancer detection rate, was attributed to primary care organisations (ie, to CCGs). Their analysis went further and attempted to disentangle to what degree the primary care organisations responsible for commissioning services was the source of this higher level variation, or rather should it be attributed to the hospitals that patients are referred to. They found that for both the 2WW referral ratio and 2WW detection rate, hospitals accounted for around two-thirds of the variability-attributed CCGs in the simpler analyses.
So what are the implications for the general practice level metric being investigated? First, it is worth noting that the majority of variation is still associated with the general practice. Furthermore, by using 5-year aggregate statistics, and restricting the analysis to practices with at least 50 cancer diagnoses over that time period issues, the play of chance and random variations in case-mix are minimised.9 Thus, the substantial and apparently real variability in referral behaviour by different GPs provides some validity for their use. However, care is needed in the interpretation of these metrics, as a naive approach may lead to misclassification of the source of any under-referrals. One option might be to provide indicator values adjusted for CCG or alternatively to provide benchmarking against local practices. Indeed the latter option is offered by the PHE fingertips website6 which presents the metrics.
While the majority of the observed variation in referral is attributable to general practices and their constituent GP referral decisions, there are clear implications that the whole of the healthcare system also needs to be taken into account. This includes both national and local funding and capacity. General practices and their GPs are not working in isolation, with the majority of 2WW referrals accessing diagnostic investigation in secondary care. Ninety per cent of National Health Service (NHS) contact occurs in primary care, while only an estimated 1 in 20 consultations leads to referral to secondary care.15 However, GPs in the UK have substantially less access to diagnostic testing than many other comparable countries.16 For example, GPs in Australia have much greater access to diagnostic testing such as CT, MRI and endoscopy,17 while achieving substantially better cancer outcomes.18 Variation in secondary care treatment is also important to consider, with a recent study in lung cancer showing substantial geographical variation in treatment significantly associated with worse patient outcomes.19
The interface between primary and secondary care is a key target for the NHS in England going forward. The NHS long-term plan has the aim of reducing outpatient visits by up to 30% of the current 100+ million hospital outpatient visits per year, and the development of larger scale primary care networks.20 With increasingly elderly, multimorbid populations, and stretched health service staff and resources, these aims face huge challenges. There is some evidence of attempts by primary care organisations to manage referrals to secondary care, including potentially for suspected cancer via referral management centres.21 While technologies such as e-advice and virtual clinics22 23 have the potential to reduce some of these outpatient referrals and visits, when cancer or serious pathology is considered there should be the facilities and resources for rapid referral and diagnostic testing. Otherwise, there is a clear risk of worse outcomes.7 With planned future investment in the NHS and other healthcare systems, the core underpinning of effective primary care population coverage needs to be emphasised. This includes continued work breaching the primary and secondary divide, with a whole systems approach to face the challenge of earlier cancer diagnosis.24
So what do these findings say about quality measurement and improvement as a whole? The analysis performed by Burton and colleagues points to various sources of variation in the performance indicator studied. By implicating secondary care as a key source of variation, intervention in a small number of hospitals could have a substantial impact in reducing variation between general practices. Perhaps, we should encourage more studies of this type, looking both to higher and lower levels of the healthcare system to shine a light on sources of variation,25 26 and question whether the level indicators are published for is the right one. That said, we need to be careful not to throw the baby out with the bathwater. Every indicator is different; some will be dominated by higher level organisations and some will not. Some will be strongly influenced by chance and small numbers and others will not.27 Not until all indicators are subject to proper scrutiny can we decide which are really useful and which less so.
## Footnotes
* Twitter @drtomround
* Funding TR is funded by a National Institute for Health Research (NIHR) Doctoral Research Fellowship (DRF; DRF-2016-09-054). GA is a senior faculty of the multi-institutional CanTest Collaborative, which is funded by Cancer Research UK (C8640/A23385).
* Disclaimer The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care.
* Competing interests None declared.
* Patient consent for publication Not required.
* Provenance and peer review Commissioned; internally peer reviewed.
## References
1. Roberts MJ , Campbell JL , Abel GA , et al . Understanding high and low patient experience scores in primary care: analysis of patients' survey data for general practices and individual doctors. BMJ 2014;349.[doi:10.1136/bmj.g6034](http://dx.doi.org/10.1136/bmj.g6034)
2. Guthrie B , Donnan PT , Murphy DJ , et al . Bad apples or spoiled barrels? multilevel modelling analysis of variation in high-risk prescribing in Scotland between general practitioners and between the practices they work in. BMJ Open 2015;5:e008270.[doi:10.1136/bmjopen-2015-008270](http://dx.doi.org/10.1136/bmjopen-2015-008270)
3. Dreischulte T , Donnan P , Grant A , et al . Safer Prescribing--A Trial of Education, Informatics, and Financial Incentives. N Engl J Med 2016;374:1053–64.[doi:10.1056/NEJMsa1508955](http://dx.doi.org/10.1056/NEJMsa1508955)
[CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1056/NEJMsa1508955&link_type=DOI)
[PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=26981935&link_type=MED&atom=%2Fqhc%2F29%2F4%2F274.atom)
4. Burton C , O'Neill L , Oliver P , et al . Contribution of primary care organisation and specialist care provider to variation in GP referrals for suspected cancer: ecological analysis of national data. BMJ Qual Saf 2020;29:296–303.[doi:10.1136/bmjqs-2019-009469](http://dx.doi.org/10.1136/bmjqs-2019-009469)
[Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjg6IjI5LzQvMjk2IjtzOjQ6ImF0b20iO3M6MTg6Ii9xaGMvMjkvNC8yNzQuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9)
5. (NICE) NIfHaCE. Suspected cancer: recognition and referral [NG12], 2015.
6. England PH. Fingertips - Cancer Services, 2019. Available: [https://fingertips.phe.org.uk/profile/cancerservices](https://fingertips.phe.org.uk/profile/cancerservices)
7. Møller H , Gildea C , Meechan D , et al . Use of the English urgent referral pathway for suspected cancer and mortality in patients with cancer: cohort study. BMJ 2015;351.[doi:10.1136/bmj.h5102](http://dx.doi.org/10.1136/bmj.h5102)
8. Herbert A , Abel GA , Winters S , et al . Cancer diagnoses after emergency GP referral or A&E attendance in England: determinants and time trends in Routes to Diagnosis data, 2006-2015. Br J Gen Pract 2019;69:e724–30.[doi:10.3399/bjgp19X705473](http://dx.doi.org/10.3399/bjgp19X705473)
[Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYmpncCI7czo1OiJyZXNpZCI7czoxMToiNjkvNjg3L2U3MjQiO3M6NDoiYXRvbSI7czoxODoiL3FoYy8yOS80LzI3NC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=)
9. Abel G , Saunders CL , Mendonca SC , et al . Variation and statistical reliability of publicly reported primary care diagnostic activity indicators for cancer: a cross-sectional ecological study of routine data. BMJ Qual Saf 2018;27:21–30.[doi:10.1136/bmjqs-2017-006607](http://dx.doi.org/10.1136/bmjqs-2017-006607)
[Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjc6IjI3LzEvMjEiO3M6NDoiYXRvbSI7czoxODoiL3FoYy8yOS80LzI3NC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=)
10. Murchie P , Chowdhury A , Smith S , et al . General practice performance in referral for suspected cancer: influence of number of cases and case-mix on publicly reported data. Br J Cancer 2015;112:1791–8.[doi:10.1038/bjc.2015.110](http://dx.doi.org/10.1038/bjc.2015.110)
[CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1038/bjc.2015.110&link_type=DOI)
[PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=25880009&link_type=MED&atom=%2Fqhc%2F29%2F4%2F274.atom)
11. Burton CD , McLernon DJ , Lee AJ , et al . Distinguishing variation in referral accuracy from referral threshold: analysis of a national dataset of referrals for suspected cancer. BMJ Open 2017;7:e016439.[doi:10.1136/bmjopen-2017-016439](http://dx.doi.org/10.1136/bmjopen-2017-016439)
12. Kostopoulou O , Nurek M , Cantarella S , et al . Referral decision making of general practitioners: a signal detection study. Med Decis Making 2019;39:21–31.[doi:10.1177/0272989X18813357](http://dx.doi.org/10.1177/0272989X18813357)
13. Mendonca SC , Abel GA , Gildea C , et al . Associations between general practice characteristics with use of urgent referrals for suspected cancer and endoscopies: a cross-sectional ecological study. Family practice 2018.
14. Lyratzopoulos G , Mendonca SC , Gildea C , et al . Associations between diagnostic activity and measures of patient experience in primary care: a cross-sectional ecological study of English general practices. Br J Gen Pract 2018;68:e9–17.[doi:10.3399/bjgp17X694097](http://dx.doi.org/10.3399/bjgp17X694097)
[Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYmpncCI7czo1OiJyZXNpZCI7czo5OiI2OC82NjYvZTkiO3M6NDoiYXRvbSI7czoxODoiL3FoYy8yOS80LzI3NC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=)
15. Foot C , Naylor C , Imison C . The quality of GP diagnosis and referral. London: The King’s Fund, 2010.
16. Rose PW , Rubin G , Perera-Salazar R , et al . Explaining variation in cancer survival between 11 jurisdictions in the International cancer benchmarking partnership: a primary care vignette survey. BMJ open 2015;5:e007212.
17. Richards M , Thorlby R , Fisher R . Unfinished business: an assessment of the National approach to improving cancer services in England 1995–2015. The Health Foundation, 2018.
18. Arnold M , Rutherford MJ , Bardot A , et al . Progress in cancer survival, mortality, and incidence in seven high-income countries 1995–2014 (ICBP SURVMARK-2): a population-based study. Lancet Oncol 2019;20:1493–505.[doi:10.1016/S1470-2045(19)30456-5](http://dx.doi.org/10.1016/S1470-2045(19)30456-5)
19. Møller H , Coupland VH , Tataru D , et al . Geographical variations in the use of cancer treatments are associated with survival of lung cancer patients. Thorax 2018;73:530–7.
[Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToidGhvcmF4am5sIjtzOjU6InJlc2lkIjtzOjg6IjczLzYvNTMwIjtzOjQ6ImF0b20iO3M6MTg6Ii9xaGMvMjkvNC8yNzQuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9)
20. Alderwick H , Dixon J . The NHS long term plan. BMJ 2019.[doi:10.1136/bmj.l84](http://dx.doi.org/10.1136/bmj.l84)
21. Round T . Gatekeeping and referral management: is cancer different? British Journal of Family Medicine 2015;3.
22. Healy P , McCrone L , Tully R , et al . Virtual outpatient clinic as an alternative to an actual clinic visit after surgical discharge: a randomised controlled trial. BMJ Qual Saf 2019;28:24–31.[doi:10.1136/bmjqs-2018-008171](http://dx.doi.org/10.1136/bmjqs-2018-008171)
[Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjc6IjI4LzEvMjQiO3M6NDoiYXRvbSI7czoxODoiL3FoYy8yOS80LzI3NC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=)
23. Cornejo-Palma D , Urbach DR . Virtual postoperative clinic: can we push virtual postoperative care further upstream? BMJ Qual Saf 2019;28:7–9.
[FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjY6IjI4LzEvNyI7czo0OiJhdG9tIjtzOjE4OiIvcWhjLzI5LzQvMjc0LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==)
24. Round T . Primary care and cancer: facing the challenge of early diagnosis and survivorship. Eur J Cancer Care 2017;26:e12703.[doi:10.1111/ecc.12703](http://dx.doi.org/10.1111/ecc.12703)
25. Diez-Roux AV . Multilevel analysis in public health research. Annu Rev Public Health 2000;21:171–92.[doi:10.1146/annurev.publhealth.21.1.171](http://dx.doi.org/10.1146/annurev.publhealth.21.1.171)
[CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1146/annurev.publhealth.21.1.171&link_type=DOI)
[PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10884951&link_type=MED&atom=%2Fqhc%2F29%2F4%2F274.atom)
[Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=000087921400010&link_type=ISI)
26. Leyland AH , Groenewegen PP . Multilevel modelling and public health policy. Scand J Public Health 2003;31:267–74.[doi:10.1080/14034940210165028](http://dx.doi.org/10.1080/14034940210165028)
[CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1080/14034940210165028&link_type=DOI)
[PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=15099032&link_type=MED&atom=%2Fqhc%2F29%2F4%2F274.atom)
[Web of Science](http://qualitysafety.bmj.com/lookup/external-ref?access_num=000184752100005&link_type=ISI)
27. Abel G , Elliott MN . Identifying and quantifying variation between healthcare organisations and geographical regions: using mixed-effects models. BMJ Qual Saf 2019;28:1032–8.[doi:10.1136/bmjqs-2018-009165](http://dx.doi.org/10.1136/bmjqs-2018-009165)
[Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjEwOiIyOC8xMi8xMDMyIjtzOjQ6ImF0b20iO3M6MTg6Ii9xaGMvMjkvNC8yNzQuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9)