Testicular cancer: improving outcomes with national quality performance indicators

QPI development

The first step in the process was to form a QPI development group. The membership of this group consisted of clinical representatives from each of the regional cancer networks in Scotland (including oncologists, urologists, radiologists and pathologists) along with patient/carer and government representation. It was called the Testicular Cancer QPI Development Group and was established in November 2013. There was a breadth of experience within the group but the chairperson was required to have had prior experience in the QPI process.

In order to determine what potential QPIs should be included, a systematic search was carried out by Healthcare Improvement Scotland (HIS) using selected websites and two primary medical databases to identify national and international guidelines. The scope for development of testicular cancer QPIs and a search narrative were defined and agreed by the development group.

Twenty-one guidelines were appraised for quality using the Appraisal of Guidelines for Research and Evaluation II instrument.12 This instrument assesses the methodological rigour used when developing a guideline. Fifteen of the guidelines were not recommended for use with the remaining six used. Table 2 shows the final search criteria used in the literature search.

Using the identified guidelines, the development group set out to create evidence-based, measurable indicators with a clear focus on what could make a real difference to quality of care (figure 1). Subgroups were formed (diagnosis, staging and surgical treatment and non-surgical treatment) and identified draft QPIs that fulfilled the following three criteria: were important to quality and outcome, were evidence based and were measurable.

Once QPIs were identified, an engagement period followed whereby the draft QPIs were widely circulated. Clinical and management colleagues from across National Health Service (NHS) Scotland, patients affected by testicular cancer and the wider public were given the opportunity to provide feedback. Feedback received was considered by the group and appropriate adjustments made where required. If differences in opinion arose, they were discussed, taking into account the available evidence, and a group consensus decision reached.

The QPIs were designed to be clear and measurable, based on sound clinical evidence while also taking into account other recognised standards and guidelines. Each QPI was given a short title to be used in reports. A fuller description was also written, which explained exactly what the indicator was measuring, with specific criteria for inclusion and exclusion, followed by an overview of the evidence base and rationale, which explained why the development of this indicator was important.

The next step was detailing measurability specifications, highlighting how the indicator would be measured in practice to allow for comparison across NHS Scotland. Finally, a target was indicated, suggesting the level which each unit should be aiming to achieve against each indicator. Flexibility in these targets was allowed due to inherent confounding factors.

An example of this is demonstrated in QPI 1—radiological staging. CT scanning is an essential part of the staging of all germ cell tumours. We know that timely imaging is important to ensure timely treatment decision-making and that unnecessary delays can have an impact on prognostic groups and hence survival rates. The time point of 3 weeks from orchidectomy was decided based on the fact that due to postoperative wound healing, chemotherapy is unlikely to be commenced within 4 weeks of surgery. Ideally, the target would be 100% but tolerance is built into the target to allow for patient choice, for example, they may not be able to attend their allocated appointment. A more lenient tolerance is facilitated in the surveillance imaging described in QPI 9, again reflecting patient factors, allowing for unavailability, lack of compliance with follow-up and those who relapse prior to their scans.

Table 3 shows the agreed QPIs and their targets. Of note, QPIs 10 and 11, which relate to mortality and clinical trials, are standard QPIs for all tumour types. The generic nature of QPI 10 in particular has a limitation as we would want tumour site-specific targets.

Implementation of QPIs

The Scottish Government’s Better Cancer Care (2008)13 included an obligation to develop indicators of quality. The Scottish Cancer Task Force introduced the National Cancer Quality Steering Group tasked with the development of generic and tumour-specific QPI sets and a national governance framework, which mandated NHS Scotland’s territorial health boards, to comply with this process.14 After the QPIs were finalised, they were circulated to individual health boards. Each NHS health board nominated a QPI clinical lead who was responsible for ensuring all members of the MDT were aware of the QPIs and relevant targets. Pathways for clinical care, from diagnosis to treatment, were reviewed at a local level and suggestions for improvement were discussed and implemented where relevant.

Data analysis

Each new diagnosis of testicular cancer is registered at a local level. The NHS board is then required to track the progress of each individual patient through the cancer pathway to produce a minimum core data set according to QPIs. The NHS boards collate and validate the data on a yearly basis and an annual regional report is produced by the regional networks. This allows areas of best practice and variance to be highlighted.

Improving survival forms an integral part of the national cancer quality improvement programme. To ensure consistent application of survival analysis at specific time points, it was agreed that a single analyst on behalf of all three regional cancer networks would undertake this work. The annual regional report is therefore submitted to the central Information Services Division (ISD) for collation and publication.

Mechanism for quality improvement

Following central analysis of the QPI data, an expert group of clinicians, with HIS assistance, review comparative national results and write to regional centres highlighting areas of good practice and variance. In cases of variance, NHS boards may be requested to submit improvement plans. HIS is available to provide support and expertise on improvement methodologies if required. Progress is monitored and if not acceptable, HIS will visit the service concerned and work at a local level to address the issues. Following this visit, a report will be submitted to the Scottish Cancer Task Force with an improvement proposal to take forward to the Scottish Government Health Department.

Initial pilot and QPI review

As part of the process of the refinement of the QPIs, they were initially piloted for 1 year. Following baseline review and publication of results at the end of this period, some changes were made to measurability in an attempt to ensure that the QPIs appropriately measured what they were intended to. The review process was undertaken by senior clinicians and in consultation with various other clinical specialties.

After this review, it was agreed that formal review of all QPIs should take place following 3 years of national reporting with revisions being implemented from year 4 onwards (currently under way). At each review the overall importance of the indicator along with the most recent high-quality clinical evidence and the measurability will be considered. If required, the QPIs will then be adjusted according to the need.

Overall

We present a single prospectively comprehensive set of national data, with figures collected and collated from the three geographical regions—North of Scotland Cancer Network, South-East Scotland Cancer Network (SCAN) and West of Scotland Cancer Network (WoSCAN) (table 1). As described above, the data are collated in each region and then forwarded to a national centre. The data also enable us to look at individual hospital site’s performance if required. As part of this process, each regional clinical lead has access to the data and is able to comment on areas of poor performance prior to national reports. This encourages active feedback locally and promotes quality improvement development.

It is important to note that when analysing the regional and national data, the cancer audit is a dynamic process with patient data continually being revised and updated as more information becomes available. This means that apparently comparable reports for the same time period and cancer site may produce slightly different figures if extracted at different times. Moreover, due to small numbers in certain centres, there will be background and regional variation often resulting in small increases and decreases in performance. All of these factors are taken into account when the report is produced. The key focus of the QPI system is to identify consistently large variations that cannot be explained by confounding factors.

Patients may be excluded from individual QPIs in certain situations (tables 3 and 4). We present data from the first three yearly reports, obtained between October 2014 and September 2017. Over this time period there were 525 recorded diagnoses of testicular cancer in Scotland: 139 in 2014/2015; 213 in 2015/2016; and 173 in 2016/2017 (table 5). The results for the last 3 years of national data are presented in table 6.

Example of good performance

From table 6 we can see that we have maintained a very low national 30-day mortality rate. We have also demonstrated consistently high national standards in timescales for pathology reporting and multidisciplinary meetings. Of note, the five patients who were not discussed in MDT prior to orchidectomy in 2016/2017 had valid reasons including requirement of urgent treatment of their disease.

Examples of poor performance and subsequent intervention

Urban areas with large catchment populations and underdeveloped referral pathways to multiple hospitals provide particular challenges for delivering services and meeting targets due to demands. This is highlighted by one particular urban area which only achieved 16/34 (47%) of orchidectomies being performed within 2 weeks. This is compared with the target of 95%.

Following feedback of this result to the regional board, a solution was proposed in the form of a clinical nurse specialist (CNS) one-stop clinic, whereby direct referral could occur and patients, in a single visit, could be seen by a CNS±urologist, receive an ultrasound, have tumour markers performed and a CT scan requested (if required), with surgery planned by the end of the visit. The patient would then be listed for the regional MDT at an appropriate time.

This intervention is in the process of being established and it is hoped it will improve performance. A similar scheme has been introduced in Birmingham, UK, and has been shown to produce a median time from clinic to having an orchidectomy of 5 days.15

A further example can be seen in the QPI based on the quality of adjuvant chemotherapy for patients with seminoma. Looking at this QPI in more detail, it was noted that regions that performed better tended to have centralised treatment, supporting the argument for this model of care. On a regional level, poor performance was noted in WoSCAN in 2014/2015 and 2015/2016. When this was reported, root cause analysis identified failings in the referral pathway as a contributing factor. These have since been addressed with a subsequent improvement in performance.

Trials

The QPI definition of an interventional trial is ‘a clinical study in which participants are assigned to receive one or more interventions (or no intervention) so that researchers can evaluate the effects of the interventions on biomedical or health-related outcomes. The assignments are determined by the study protocol. Participants may receive diagnostic, therapeutic, or other types of interventions’. A translational study is defined as ‘a term used to describe the process by which the results of research done in the laboratory are used to develop new ways to diagnose and treat disease.’

The data set demonstrated that our recruitment to clinical trials, both interventional and translational, was poor.