Protocol••

Developing, implementing and disseminating a core outcome set for neonatal medicine

•,,,,,.on behalf of the Core Outcomes in Neonatology (COIN) Project Steering Group

...

Abstract

Background In high resource settings, 1 in 10 newborn babies require admission to a neonatal unit. Research evaluating neonatal care involves recording and reporting many different outcomes and outcome measures. Such variation limits the usefulness of research as studies cannot be compared or combined. To address these limitations, we aim to develop, disseminate and implement a core outcome set for neonatal medicine.

Methods A steering group that includes parents and former patients, healthcare professionals and researchers has been formed to guide the development of the core outcome set. We will review neonatal trials systematically to identify previously reported outcomes. Additionally, we will specifically identify outcomes of importance to parents, former patients and healthcare professionals through a systematic review of qualitative studies. Outcomes identified will be entered into an international, multi-perspective eDelphi survey. All key stakeholders will be invited to participate. The Delphi method will encourage individual and group stakeholder consensus to identify a core outcome set. The core outcome set will be mapped to existing, routinely recorded data where these exist.

Discussion Use of a core set will ensure outcomes of importance to key stakeholders, including former patients and parents, are recorded and reported in a standard fashion in future research. Embedding the core outcome set within future clinical studies will extend the usefulness of research to inform practice, enhance patient care and ultimately improve outcomes. Using routinely recorded electronic data will facilitate implementation with minimal addition burden.

Trial registration number Core Outcome Measures in Effectiveness Trials (COMET) database: 842 (www.comet-initiative.org/studies/details/842).

What is already known on this topic?

Poor outcome selection is an important cause of research waste.
Much neonatal practice lacks high-quality evidence to support it, and there is considerable variation in neonatal practice and in the outcomes of neonatal care.
Core outcome sets (an agreed, standardised group of outcomes to be reported by all trials within a research field) have been developed in other specialties to improve outcome selection for research and to facilitate the generation of high-quality evidence.

What this study hopes to add?

Developing a core outcome set for neonatal medicine will identify which outcomes are most important to all affected by neonatal care.
Standardised reporting of outcomes in clinical trials will facilitate evidence synthesis in neonatal medicine.
A neonatal core outcome set will also be used to improve the national registry, audit and benchmarking processes.

Introduction

One in 10 babies born in high resource settings is admitted to a neonatal unit, and this proportion is increasing.1 Care provided can range from a few hours of observation to prolonged periods of intensive care, requiring interventions such as intubation and ventilation, intravenous medications and surgery. Neonatal conditions are a leading cause of childhood mortality and morbidity; approximately 2.7 million babies die within the first month of postnatal life each year.2 Preterm birth is increasingly implicated in the pathogenesis of adult non-communicable diseases.3 Despite this importance, there is a paucity of evidence to underpin many routine neonatal clinical practices, leading to a wide variation in clinical care and uncertain outcomes.4 5

It is rare for neonatal systematic reviews and meta-analyses of clinical trials to result in conclusive recommendations for clinical practice.6 This is largely due to the different, often incomparable, outcomes measured in individual trials.7 In a clinical trial, an outcome is the recorded effect that an intervention has on a participant; for example, a trial of a new antibiotic could involve collection of a number of different outcomes such as survival (to different points in time), side effects or longer-term neurodevelopment. If studies measure different outcomes, or measure them in differing ways using different outcome measures, then results from these studies cannot be compared or combined. Another concern, shared with many paediatric specialties,8 is that selection of outcomes rarely involves parents or former patients. The limitations stemming from multiple, non-comparable outcomes are also seen across neonatal observational research, benchmarking, clinical audit and quality improvement studies.

One solution to improve outcome selection, recording and reporting is the development of a core outcome set.9 A core outcome set is a collection of important outcomes identified through robust consensus methods by key stakeholders. The purpose of a core outcome set is to ensure that all research in a specialty involves the recording of standard measures, important to key stakeholders and clinically meaningful, which are reported consistently in publications. In recent years, a number of core outcome sets have been developed in both adult and paediatric medicine.10–13 The use of core outcome sets for clinical trials is promoted by journals14 and research funders.15

We aim to develop a core outcome set for neonatal care that incorporates the views of stakeholders including health professionals, researchers, parents and patients.

Methods

Prospective registration

The Core Outcome Measures in Effectiveness Trials (COMET) initiative maintains a registry of core outcome sets. We prospectively registered our core outcome set (registration number 842).16 Systematic reviews were prospectively registered with Prospective Register of Systematic Reviews (PROSPERO). The review of outcomes reported in neonatal trials has registration number identification CRD42016042110.17 The review of qualitative research has registration number identification CRD42016037874.18

Research ethics review

National Health Service (NHS) Research Ethics Committee (REC) approval is not required (see online supplementary additional file 1).

Study funding

This study is funded as part of a peer-reviewed Medical Research Council Clinician Scientist Fellowship (MR/N008405/1).

Steering group

A steering group was formed to guide the development of the core outcome set. The members of the steering group have been selected to represent different disciplines, perspectives and expertise. Steering group members are listed at the end of this manuscript.

Within the steering group, a project management team will be established. The project coordinator (JW) and one other member of the steering group (CG) will be responsible for the day-to-day running of the project. The project management team will manage day-to-day aspects of the study; the wider the steering group will be contacted for key decisions. A majority of steering group members need to respond to form a quorum. A decision will be followed when more than half of the quorum agrees.

Scope of the core outcome set

At the initial steering group meeting, it was decided that the core outcome set will apply to babies receiving medical care on a neonatal unit, with no limitation by gestational age at birth or illness severity. For this project, care delivered exclusively on labour or postnatal wards or in the community is beyond scope. The core outcome set is intended to apply to clinical trials in neonatal care and, where appropriate, to neonatal observational research, benchmarking, audit and quality improvement.

Study overview

The study will be divided in two distinct stages:

Identification of outcomes reported in neonatal clinical trials and qualitative research.
Use of these identified outcomes to determine a set of core outcomes (a neonatal core outcome set) using consensus methods and involving all relevant stakeholders.

Stage 1: identification of potential outcomes

A search of the Cochrane and PROSPERO databases (11 November 2016) indicated that no systematic review of neonatal outcomes has been performed. Two systematic reviews will be performed to identify outcomes reported in neonatal clinical trials and outcomes reported in qualitative research. All outcomes will be considered, including both those within the neonatal period and outcomes occurring in later life. Where appropriate, different methodological approaches will be used for the systematic review of clinical trials and for the systematic review of qualitative literature.

Systematic review to identify outcomes reported in clinical trials

The aim of this systematic review is to identify outcomes that have been recorded in clinical trials for treatments used in neonatal care. We will search the following databases: Cochrane Central Register of Controlled Trials (CENTRAL), Cumulative Index to Nursing and Allied Health Literature (CINAHL), Excerpta Medica Database (EMBASE) and Medical Literature Analysis and Retrieval System Online (MEDLINE); the proposed search strategy is included (see online supplementary additional file 2). Randomised or cluster randomised trials evaluating interventions for neonates (age 0–28 days) of any gestation or infants requiring ongoing care in a neonatal unit will be included. All trials published in the last 5 years will be included. Translations will be sought if a trial is not reported in English. When screening studies for inclusion, studies will be reviewed by two independent reviewers (SA and JW). Where there is disagreement, the study will be reassessed by an additional reviewer (CG).

Using a pilot-tested extraction form, data will be extracted relating to trial identifiers, number of infants recruited, primary and secondary outcomes, whether the study was prospectively registered and whether there is evidence of parent or patient involvement in outcome selection. Two reviewers will independently assess the methodological quality of the included studies using the Jadad scale (score range 1 to 5).19 Methodological quality will be dichotomised as high (score ≥3) and low (score <3). Outcomes will be extracted from included trials; the primary outcome(s) will be defined as the outcome explicitly defined as such or used for a sample size calculation; secondary outcomes will be all other outcomes. Studies will be reviewed by two independent reviewers (SA and JW). Where there is disagreement, the study will be reassessed by an additional reviewer (CG).

Results will be reported in a narrative synthesis and presented in a tabular form. Outcomes will be grouped using a previously defined framework of biological systems.20 The number and frequency of outcomes measured in clinical trials will be reported. Additional analyses will be conducted to examine the frequency with which predefined neonatal comorbidities are reported in the largest clinical trials (see online supplementary additional file 3).

Systematic review to identify outcomes reported by parents, patients and other stakeholders

The aim of this systematic review is to identify which outcomes of neonatal care are described by former patients, parents and healthcare providers as important.

We will search the following databases: Applied Social Sciences Index and Abstracts, CENTRAL, CINAHL, EMBASE, MEDLINE and Psychological Information Database. The proposed search strategy is included (see online supplementary additional file 4). As this review relates to qualitative research, the review question has been formulated in a SPIDER format to define eligibility criteria.21 Qualitative studies (including phenomenology, ethnography, case studies and grounded theory) relating to neonates (age 0–28 days) of any gestation and any infants requiring ongoing care in a neonatal unit beyond this age in a high resource setting will be included. Quantitative research will be included if relevant data have been gathered, for example, surveys developed with parent or caregiver input. All studies published in a peer-reviewed journal or a funder's report in the last 20 years will be included. Non-English trial reports will be translated. The primary outcome will be a comprehensive list of neonatal care outcomes reported by former patients, their parents and the healthcare professionals caring for them. All screening will be done by one individual (JW), but for quality assurance, an independent reviewer (CG) will screen 10% of abstracts and titles. We will examine for agreement using Cohen's Kappa coefficient.

After screening, all papers will be coded using a piloted coding tool and using Eppi-Reviewer 4.22 The coding tool will be pilot tested on 10 papers and refined accordingly. All papers will be coded independently by two researchers (JW and CG/GB), with any disagreement resolved by a third researcher (CG/GB). Data will be extracted from each study relating to details of the stakeholders included, the gestational age and birth weight of neonates involved, the country in which the study was performed, the qualitative methodology used and the outcomes identified by stakeholders.

Outcomes identified by multiple studies will be combined. Outcomes identified will be grouped according to a framework of biological health systems.20 Textual findings from data related to each organ system will be analysed as part of a thematic synthesis to determine key themes. Further thematic analysis will be undertaken to identify novel outcome categories. Additional analyses will examine whether outcomes differ by stakeholder groups and gestational age.

Stage 2: determining core outcomes

A comprehensive inventory of outcomes will be created by combining the results of the two systematic reviews. The steering group will eliminate duplicate and group outcomes based on a framework of biological health systems.20 These will be used as the starting point for the consensus process to determine a core outcome set. Consensus methods will involve multiple stakeholder groups; two methods will be used, a multi-round, online Delphi survey followed by a consensus meeting.

The following groups of stakeholders will be identified:

Former patients admitted to a neonatal unit in infancy and parents of neonatal patients. These will be recruited by placing adverts on neonatal charity websites and contacting bloggers discussing preterm and neonatal experiences.
Clinicians including neonatal nurses, neonatologists, general paediatricians, paediatric specialists and community paediatricians specialising in neurodevelopment. Recruitment will be through adverts placed on the Royal College of Paediatrics and Child Health (RCPCH) website and through professional organisations such as the British Association of Perinatal Medicine and the Neonatal Society.
Allied health professionals including physiotherapists, speech and language therapists, occupational therapists and clinical psychologists. Adverts will be placed in professional journals and on profession-specific specialty interest websites.
Academics and researchers in the neonatal field or those involved in the collection of routine neonatal datasets. Recruitment will be through national meetings, academic publications and through academic organisations.

We will aim for 30 or more participants in each group, which will give a total panel of at least 120 participants, in line with previous core outcome methods.23

The panel will undergo a three-round Delphi survey (using a web-based questionnaire) to establish consensus. The Delphi process allows for consensus to be reached from a selection of disparate expert opinions.24 In each round, panel members will be asked to rank the outcomes. In the first round of the Delphi process, panel members will also have the opportunity to suggest additional outcomes that they feel are important and were not identified in the two systematic reviews. These will be integrated into round 2 by the study management group following principles established by the steering group. The Delphi surveys will enable all stakeholders to participate and will assess the extent of agreement (consensus measurement) and resolve disagreement (consensus development).

Panel members will be asked to score each outcome between 1 and 9 during each round, using a Likert-type scale.25 After each round, results will be collated and any outcomes universally scored to be of limited importance (scored between 1 and 3) will not be carried forward to the next round. In later rounds, panel members will be presented with the median score, by stakeholder group, for each outcome and asked to review their results before re-scoring outcomes. Repeated reflection and scoring has been demonstrated to increase the likelihood of convergence and consensus.26

In the final round, panel members will also be asked whether each outcome should be included in the core outcome set. A standardised definition will then be applied to the results from this round:

Consensus in (classified as a core outcome): Over 70% of panel members in each group score the outcome ‘critical for decision making’ (score 7 to 9) and less than 15% of panel members in each group score the outcome of limited importance for decision making’ (score 1 to 3).
Consensus out (do not classify as a core outcome): Over 70% of panel members in each group score the outcome ‘of limited importance for decision making’ (score 1 to 3) and less than 15% of panel members in each group score the outcome ‘critical for decision making’ (score 7 to 9).
No Consensus (do not classify as a core outcome): Anything else.

The results of this final round will be taken to a consensus meeting to determine a final core outcome set.

A consensus meeting of stakeholders will use the results of the completed Delphi process to identify a final core outcome set. The remit of the consensus meeting will be to refine the final results from the Delphi; no new outcomes will be considered at this stage and the results of the Delphi will be paramount when selecting the core outcome set. To ensure transparency, these outcomes will be published with the survey results to show the degree of consensus for each outcome. If outcomes are excluded at this stage, they will also be published (with survey results) with an explanation of the reasons for exclusion and which stakeholder groups agreed or disagreed with the exclusion. The core outcome set will be reported in line with reporting guidelines.27

Discussion

Once identified, the core outcome set will be disseminated to raise awareness and uptake. We will publish our methodology and results in peer-reviewed journals. Raising the awareness of the core outcome set among neonatal practitioners and researchers is a crucial part of this project. The core outcome set will be made freely available through the COMET initiative website and through the Core Outcomes in Women's and Newborn Health Network Initiative.28 Use of the core outcome set in future research will be promoted by research funders15 and journals.14 In the UK, a standardised set of data items, the Neonatal Data Set (NDS), exists for admissions to NHS neonatal care. These data form the National Neonatal Research Database (NNRD), a national resource used for multiple purposes including research,29 benchmarking and national audit.30 We will work with the Neonatal Data Analysis Unit and other bodies to ensure data relating to the core outcomes are extracted from routinely collected data and held in the NNRD. The NDS is reviewed annually and therefore provides an opportunity for neonatal core outcomes to be captured from routinely recorded data. This will facilitate audit and benchmarking with a standardised group of outcomes that are important to former patients, parents and clinicians.

A core outcome set in neonatal medicine will ensure that researchers select outcomes that are clinically relevant and important to parents and patients. Implementing a core outcome set in neonatal medicine will standardise the selection of outcomes in individual clinical trials. A standardised set of outcomes across studies will facilitate evidence synthesis in meta-analyses and systematic reviews. This will assist the delivery of evidence-based improvements in neonatal care.

Contributors: CG conceived of this project. JW, CG and SA planned the systematic review of outcomes reported in clinical trials. JW, CG and GB planned the systematic review of outcomes identified by patients, parents and other stakeholders. The first draft of the manuscript was written by JW, CG, GB and JMND; NM edited and reviewed the manuscript. It was approved by JW, CG, GB, SA, JMND, NM and the Core Outcomes in Neonatology (COIN) Project Steering Group.
Funding: This study was funded as part of a Medical Research Council (MRC) Clinician Scientist Fellowship awarded to CG (MR/N008405/1). The MRC were not involved in the design of the study, collection, analysis and interpretation of data or in writing the manuscript.
Competing interests: None declared.
Provenance and peer review: Not commissioned; externally peer reviewed.
Collaborators: Steering Group Chair: Marian Knight, Professor of Maternal and Child Population Health, National Perinatal Epidemiology Unit, Oxford. Project Management Team: Chris Gale, Clinical Senior Lecturer in Neonatal Medicine, Imperial College, London. James Webbe, Clinical Research Fellow, Imperial College, London. Steering Group: Elsa Afonso, Neonatal Staff Nurse, Cambridge University Hospitals NHS Foundation Trust, Cambridge. Iyad Al-Muzaffar, Consultant Neonatologist and Parent of preterm baby, Cwm Taf University Health Board, Wales. Ginny Brunton, Midwife and qualitative methodologist, UCL Institute of Education, London. James Duffy, Doctoral Research Fellow in Obstetrics and Gynaecology, Nuffield Department of Primary Health Sciences, Oxford. Anne Greenough, Professor of Neonatology and Clinical Respiratory Physiology, King's College, London. Nigel Hall, Associate Professor of Paediatric Surgery, University of Southampton. Jos Latour, Professor of Clinical Nursing, Plymouth University, Plymouth. Neil Marlow, Professor of Neonatal Medicine, University College, London. Neena Modi, Professor of Neonatal Medicine, Imperial College, London. Laura Noakes, Parent of neonatal patient. Julie Nycyk, Consultant Neonatologist, Sandwell and West Birmingham Hospitals NHS Trust. Mehali Patel, Research Engagement Officer, Bliss, London. Angela Richard-Londt, Parent of neonatal patients. Ben Wills-Eve, Ex-neonatal patient.

Acknowledgements

We wish to thank the members of the COIN Steering Group for their support, and BLISS and the RCPCH for helping promote this project.

Harrison W, Goodman D. Epidemiologic trends in neonatal intensive care, 2007–2012. JAMA Pediatr 2015; 169:855–62.
doi:10.1001/jamapediatrics.2015.1305•Google Scholar
Liu L, Oza S, Hogan D, et al. Global, regional, and national causes of under-5 mortality in 2000–15: an updated systematic analysis with implications for the sustainable development goals. Lancet 2016; 388:3027–35.
doi:10.1016/S0140-6736(16)31593-8•Google Scholar
Gluckman PD, Hanson MA, Cooper C, et al. Effect of in utero and early-life conditions on adult health and disease. N Engl J Med 2008; 359:61–73.
doi:10.1056/NEJMra0708473•Google Scholar
Klingenberg C, Embleton ND, Jacobs SE, et al. Enteral feeding practices in very preterm infants: an international survey. Arch Dis Child Fetal Neonatal Ed 2012; 97:F56–F61.
doi:10.1136/adc.2010.204123•Google Scholar
Shah PS, Lui K, Sjörs G, et al. Neonatal outcomes of very low birth weight and very preterm neonates: an international comparison. J Pediatr 2016; 177:144–52.
doi:10.1016/j.jpeds.2016.04.083•Google Scholar
Sinclair JC, Haughton DE, Bracken MB, et al. Cochrane neonatal systematic reviews: a survey of the evidence for neonatal therapies. Clin Perinatol 2003; 30:285–304.
doi:10.1016/S0095-5108(03)00025-3•Google Scholar
Clarke M. Standardising outcomes in paediatric clinical trials. PLoS Med 2008; 5:e102.
doi:10.1371/journal.pmed.0050102•Google Scholar
Sinha IP, Williamson PR, Smyth RL, et al. Outcomes in clinical trials of inhaled corticosteroids for children with asthma are narrowly focussed on short term disease activity. PLoS One 2009; 4.
doi:10.1371/journal.pone.0006276•Google Scholar
Williamson PR, Altman DG, Blazeby JM, et al. Developing core outcome sets for clinical trials: issues to consider. Trials 2012; 13:132.
doi:10.1186/1745-6215-13-132•Google Scholar
Tugwell P, Boers M, Brooks P, et al. OMERACT: an international initiative to improve outcome measurement in rheumatology. Trials 2007; 8:38.
doi:10.1186/1745-6215-8-38•Google Scholar
Al Wattar BH, Tamilselvan K, Khan R, et al. Development of a core outcome set for epilepsy in pregnancy (E-CORE): a national multi-stakeholder modified Delphi consensus study. BJOG 2017; 124:661–7.
doi:10.1111/1471-0528.14430•Google Scholar
van ʼt Hooft J, Duffy JM, Daly M, et al. A core outcome set for evaluation of interventions to prevent preterm birth. Obstet Gynecol 2016; 127:49–58.
doi:10.1097/AOG.0000000000001195•Google Scholar
Sinha IP, Gallagher R, Williamson PR, et al. Development of a core outcome set for clinical trials in childhood asthma: a survey of clinicians, parents, and young people. Trials 2012; 13:103.
doi:10.1186/1745-6215-13-103•Google Scholar
The CROWN Initiative: journal editors invite researchers to develop core outcomes in women's health. Hum Reprod Update 2014; 20:465–6.
doi:10.1093/humupd/dmu029•Google Scholar
COMET database launched to encourage use of core outcome sets in health research.
Google Scholar
COMET database.
Google Scholar
PROSPERO database.
Google Scholar
PROSPERO database.
Google Scholar
Jadad AR, Moore RA, Carroll D, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials 1996; 17:1–12.
doi:10.1016/0197-2456(95)00134-4•Google Scholar
Gale C, Morris I. The UK National Neonatal Research Database: using neonatal data for research, quality improvement and more. Arch Dis Child Educ Pract Ed 2016; 101:216–8.
doi:10.1136/archdischild-2015-309928•Google Scholar
Cooke A, Smith D, Booth A, et al. Beyond PICO: the SPIDER tool for qualitative evidence synthesis. Qual Health Res 2012; 22:1435–43.
doi:10.1177/1049732312452938•Google Scholar
Thomas J, Brunton J, Graziosi S, et al. EPPI-Reviewer 4: software for research synthesis. EPPI-Centre Software. London, Social Science Research Unit, UCL Institute of Education 2010;
Google Scholar
Sinha IP, Smyth RL, Williamson PR, et al. Using the Delphi technique to determine which outcomes to measure in clinical trials: recommendations for the future based on a systematic review of existing studies. PLoS Med 2011; 8:e1000393.
doi:10.1371/journal.pmed.1000393•Google Scholar
Hsu C, Sandford B. The Delphi technique: making sense of consensus. Pract Assess Res Eval 2007; 12:1–8.
Google Scholar
Likert R. A technique for the measurement of attitudes.. Archives of Psychology 1932; 22:5–55.
Google Scholar
Dalkey N, Helmer O. An experimental application of the DELPHI method to the use of experts. Manage Sci 1963; 9:458–67.
doi:10.1287/mnsc.9.3.458•Google Scholar
Kirkham JJ, Gorst S, Altman DG, et al. Core Outcome Set-STAndards for Reporting: the COS-STAR statement. PLoS Med 2016; 13:e1002148.
doi:10.1371/journal.pmed.1002148•Google Scholar
Khan K. The CROWN Initiative: journal editors invite researchers to develop core outcomes in women's health. BJOG 2014; 121:1181–2.
doi:10.1111/1471-0528.12929•Google Scholar
Gale C, Morris I. The UK National Neonatal Research Database: using neonatal data for research, quality improvement and more. Arch Dis Child Educ Pract Ed 2016; 101:216–8.
doi:10.1136/archdischild-2015-309928•Google Scholar
National Neonatal Data set.
Google Scholar