Article Text
PDF
Abstract
Introduction The Early Detection of Deterioration in Elderly residents (EDDIE+) programme is a theory-informed, multi-component intervention aimed at upskilling and empowering nursing and personal care staff to identify and manage early signs of deterioration in residents of aged care facilities. The intervention aims to reduce unnecessary hospital admissions from residential aged care (RAC) homes. Alongside a stepped wedge randomised controlled trial, an embedded process evaluation will be conducted to assess the fidelity, acceptability, mechanisms of action and contextual barriers and enablers of the EDDIE+ intervention.
Methods and analysis Twelve RAC homes in Queensland, Australia are participating in the study. A comprehensive mixed-methods process evaluation, informed by the integrated Promoting Action on Research Implementation in Health Services (i-PARIHS) framework, will assess intervention fidelity, contextual barriers and enablers, mechanisms of action, and the acceptability of the programme from various stakeholder perspectives. Quantitative data will be collected prospectively from project documentation, including baseline context mapping of participating sites, activity tracking and regular check-in communication sheets. Qualitative data will be collected postintervention via semi-structured interviews with a range of stakeholder groups. The i-PARIHS constructs of innovation, recipients, context and facilitation will be applied to frame the analysis of quantitative and qualitative data.
Ethics and dissemination Ethical approval for this study has been granted by the Bolton Clarke Human Research Ethics Committee (approval number: 170031) with administrative ethical approval granted by the Queensland University of Technology University Human Research Ethics Committee (2000000618). Full ethical approval includes a waiver of consent for access to residents’ demographic, clinical and health services de-identified data. A separate health services data linkage based on RAC home addresses will be sought through a Public Health Act application. Study findings will be disseminated through multiple channels, including journal publications, conference presentations and interactive webinars with a stakeholder network.
Trial registration number Australia New Zealand Clinical Trial Registry (ACTRN12620000507987).
- Change management
- EDUCATION & TRAINING (see Medical Education & Training)
- QUALITATIVE RESEARCH
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
Theory-informed process evaluation, framed by the integrated Promoting Action on Research Implementation in Health Services framework and an intervention logic model.
Process data from a range of sources to assess implementation processes and outcomes.
Outcomes could help inform planning for future development and implementation of hospital avoidance strategies in residential aged care (RAC) facilities.
High staff turnover and workload within the RAC sector may impact staff availability to participate in surveys and interviews.
Data relating to residents’ experiences will be collected from family members and nominated advocates, rather than directly from residents.
Introduction
When older adults living in residential aged care (RAC) are admitted to hospital, they face increased risk of hospital associated complications and invasive interventions.1 Hospital presentations and admissions among this population group are relatively high and there is a evidence to suggest some hospital encounters are avoidable.2 A report published by the Australian Medical Association estimated 27 000 potentially preventable admissions from RAC homes in Australia in 2021, equating to 160 000 bed days with a cost of $A312 million.3 RAC residents, family members and staff express a preference for care to be provided in their home where possible.4 Previous research indicates that this is possible and will reduce hospital presentations and admissions from RAC, from implementing models of care that provide access to resources and improve the clinical skills and confidence of nursing staff.5
The ‘Early Detection of Deterioration In Elderly residents’ or ‘EDDIE’ programme was developed in Queensland, Australia as a hospital avoidance intervention targeted at nursing and other care staff working in RAC. The aim was to empower and enable staff to identify and appropriately respond to early clinical signs of a deteriorating resident.5 6 An initial pilot of EDDIE demonstrated that the intervention was feasible and acceptable to RAC staff, reduced hospital transfer rates and resulted in a 41% reduction in total hospital bed days.7 EDDIE+ builds on the learning from the EDDIE pilot5 6 8 and aims to develop and test a scalable hospital avoidance intervention in RAC. The evaluation study involves a type 1 stepped-wedge randomised controlled effectiveness-implementation trial9 with embedded economic and mixed-methods process evaluation. Details of the trial, which involves 12 participating RAC homes in metropolitan and regional Queensland, have been described in a previously published trial protocol paper.10 This paper presents the protocol for the process evaluation component of the study. Process evaluations are increasingly recognised as an important part of developing and testing complex interventions such as EDDIE+, which comprises multiple components and is implemented across multiple sites.11 12 Process evaluations often include assessing an intervention’s fidelity, namely, if the intervention was implemented as intended, the acceptability of an intervention from various stakeholder perspectives, the mechanism of impact, or what initiates a change, and an assessment of barriers and enablers to implementation.
EDDIE+ intervention
EDDIE+ focuses on upskilling nursing and personal care staff working within RAC, by giving them the knowledge, skills and support needed to manage subacute episodes such as urinary tract infections, chest pain, falls and dyspnoea within the home setting. It comprises four components: advanced clinical skills education and training (provided initially by a project-funded nurse educator), decision support tools, provision of diagnostic equipment (eg, bladder scanners and vital signs monitors) and implementation facilitation and support (via a locally appointed clinical facilitator supported by a project implementation facilitator).6 The development of EDDIE+ was underpinned by a widely used implementation framework, the integrated Promoting Action on Research Implementation in Health Services (i-PARIHS) framework.13 i-PARIHS proposes that the successful implementation of evidence-informed innovations results from the active facilitation of an innovation with the intended recipients of implementation within their local, organisational and system context. As such, attention to facilitation, engagement with RAC stakeholders, involvement of staff and responsiveness to context are key features of EDDIE+.
By embedding implementation facilitation within the bundle of components that comprise EDDIE+, implementation is integral to the intervention. Consistent with facilitation as a primary implementation strategy, clinical facilitators can tailor the implementation of EDDIE+ according to their own home’s needs. This will be achieved through the identification of core and adaptable features of each EDDIE+ component (table 1).
Core and adaptable components of EDDIE+ intervention
Figure 1 presents a logic model summarising how EDDIE+ is expected to work and produce intended changes to processes and outcomes of care.
EDDIE+ intervention logic model. EDDIE, Early Detection of Deterioration in Elderly residents; RAC, residential aged care.
Methods and analysis
Process evaluation
While the trial component of the study focuses on intervention effectiveness, the process evaluation aims to understand how and why the intervention works in real-world contexts. This involves examining whether the intervention has been implemented as planned and resulted in expected outcomes. Understanding whether and how an intervention is affecting change can provide insights into the processes of implementation and the extent to which these account for positive or negative study outcomes. This is particularly helpful if the actual study outcomes differ from expected outcomes, enabling the study team to understand whether there has been implementation failure, such as poor delivery of the intervention, or intervention failure, such as poor or inappropriate design.14 This might inform planning of future interventions and implementation strategies.
To evaluate how and how well EDDIE+ was implemented, the process evaluation of EDDIE+ will follow published guidance on conducting and reporting studies with a process evaluation component.12 Consistent with the application of i-PARIHS to inform the development of EDDIE+, the process evaluation will be framed by i-PARIHS and the intervention logic model that was developed at the study design stage (figure 1). Implementation outcomes of interest in the process evaluation include fidelity and acceptability of EDDIE+ to multiple stakeholders, the mechanisms through which EDDIE+ achieves an effect (or not), and contextual barriers and enablers of implementation.
Aims
The aim of the process evaluation is to track the implementation of EDDIE+in the 12 participating RAC homes to:
Assess EDDIE+ intervention fidelity.
Assess the acceptability of EDDIE+ from the perspective of staff, residents’ family members, EDDIE+facilitators and wider stakeholders.
Identify the mechanisms of impact.
Identify contextual barriers and enablers of implementation.
Study design and data collection
An embedded and formative mixed methods process evaluation will be undertaken. This will be guided by a series of templates based on i-PARIHS to assess fidelity and acceptability of EDDIE+, mechanisms of impact, and contextual barriers and enablers within and across the 12 regional and metropolitan homes. Data from all four intervention phases of the stepped wedge trial will be collected and analysed. These are the preparation, baseline exposure, intervention introduction and intervention exposure phases.
We first summarise how the theoretical propositions of the i-PARIHS framework inform the questions of interest within the process evaluation, before describing the methods of data collection and analysis (tables 2 and 3).
Overview of process evaluation data collection and analysis
Description of process evaluation data sources
i-PARIHS theoretical framing
Innovation
According to the theoretical proposition of i-PARIHS, implementation effectiveness is enhanced if there is support for the innovation to be implemented. The innovation in this case is EDDIE+, an intervention to improve the identification and management of clinical deterioration in residents within the home setting and in turn, reduce unnecessary hospital transfers. Support is more likely if key stakeholders including RAC staff, managers, residents, family members and external care providers, agree with the idea of keeping residents at home where possible and perceive implementation to be workable in practice. In relation to EDDIE+, this includes support for the education and training offered and the introduction and use of new diagnostic equipment. Therefore, it will be important to collect stakeholder views on the acceptability, relevance and importance of EDDIE+ within the context of the RAC home setting.
Recipients
i-PARIHS proposes that recipients of an innovation (eg, staff, residents and family members) need both ‘want to’ and ‘can do’ factors to achieve successful implementation.15 RAC staff in particular have to be motivated to address the issue of clinical deterioration in residents and have the capacity and capability to implement EDDIE+. These areas will be explored as part of the data collection.
Context
Contextual factors at multiple levels are identified as important barriers or enablers of implementation in i-PARIHS and will be examined as part of the process evaluation. The inner context spans the local and organisational settings. At a local level, inner context refers to the immediate place of implementation—the RAC home—and encompasses factors such as the workplace culture, management and leadership support, workload, receptiveness, and attitudes to change. The local context is embedded within the organisational context—the aged care provider organisation—where factors relating to culture, leadership, support and resources are also important. Outer context relates to the wider aged care system, including policy drivers, regulatory standards and frameworks, other initiatives that influence the care of deteriorating residents, and more general health, social and economic issues that affect aged care. Initial mapping of contextual factors will be undertaken pre-implementation and tracked throughout the intervention phase of the study.
Facilitation
Facilitation in the i-PARIHS framework is positioned as the active ingredient of implementation, comprising facilitator roles and the use of enabling facilitation strategies. It is the facilitator’s role to assess innovation, recipient and contextual factors that present barriers to or enablers of implementation and plan appropriate facilitation strategies to address these. The main facilitator role in EDDIE+ is the clinical facilitator appointed from within the RAC home to support implementation, with funding provided for backfill support. The clinical facilitator receives additional support from the EDDIE+ project team including the nurse educator and the project implementation facilitator. This is based on a model of internal-external facilitation.16 The nurse educator is responsible for developing and delivering the training on clinical deterioration and the diagnostic equipment to RAC staff, while the implementation facilitator will undertake the baseline context assessment and support the clinical facilitators to develop facilitation skills. As part of the process evaluation, it will be important to collect data about the different facilitator roles, the strategies used to facilitate implementation and how well these worked.
Process evaluation elements
Fidelity
Fidelity will be evaluated in relation to the delivery of EDDIE+ as intended, namely: attendance at mandatory EDDIE+ training by nurses and personal care workers (expressed as a percentage of total staff employed who attended training), number of EDDIE+ sessions delivered/attended, use of the new equipment and recruitment and retention of clinical facilitators. These data will be extracted from EDDIE+ check in forms completed by the nominated clinical facilitator at each site and the communication and tracking data collected from the project team, including education attendance records. Additional data sources will be used to determine any critical time junctures such as COVID-19 lockdowns, infection outbreaks and other events that may have impacted the implementation of EDDIE+.
Acceptability
Data will be collected on the acceptability of EDDIE+ from the perspective of four stakeholder groups: RAC staff including Registered Nurses, Enrolled Nurses and Personal Care Workers, family members or nominated advocates of residents, clinical facilitators, and local and external stakeholders (see tables 2 and 3). Semi-structured interviews will be conducted with these different groups to ascertain their views about EDDIE+. Family members and nominated advocates will be asked about their awareness and experiences of EDDIE+ and how it impacted the resident’s care. RAC staff and other stakeholders will be interviewed about EDDIE+ and how it was implemented to determine what they found most and least helpful about EDDIE+ and whether they thought the intervention was transferable to other RAC homes (see online supplemental files S1 and S2 for interview guides). Additionally, a three-question traffic light survey will be distributed to family members and nominated advocates to determine if their experience with EDDIE+ was positive, negative or neutral, if EDDIE+ impacted the care of their loved one in a good way, and their views on whether EDDIE+ should be introduced into other RAC homes (see online supplemental file S3).
Supplemental material
Mechanisms of impact
As illustrated in the logic model in figure 1, the EDDIE+ intervention is expected to produce improvements in resident, staff and system level outcomes through mechanisms including enhanced staff knowledge and skills, increased staff confidence and sense of empowerment, and greater confidence of family members and external care providers in the ability of RAC home staff to provide appropriate clinical care for residents. These mechanisms will be explored through several data sources. RAC staff will be requested to complete a self-efficacy survey pre-EDDIE+ and post-EDDIE+ implementation using a validated self-efficacy questionnaire17 to evaluate reported changes in staff confidence and capability (online supplemental file S4). Questionnaire data will be supplemented with data from semi-structured interviews conducted with RAC staff, clinical facilitators, managers and external care providers, such as general practitioners, to assess mechanisms relating to confidence, staff empowerment and skills and knowledge development (online supplemental files S1 and S2).
Understanding barriers and enablers
Consistent with the i-PARIHS framework, barriers and enablers to implementation will be explored in relation to the EDDIE+ intervention (acceptability and feasibility), recipient characteristics (RAC staff ‘want to’ and ‘can do’ factors) and the inner and outer context. During semi-structured interviews, RAC staff and wider stakeholders will be asked to provide specific examples of barriers and enablers of EDDIE+, what worked well (or less well) in their own RAC home and what would need to be considered for future implementation in other facilities. Supplementary information related to barriers and enablers will be extracted from the baseline context mapping, communication and activity tracking spreadsheets and check in forms completed by clinical facilitators and the nurse educator and project implementation facilitator.
Setting and participant recruitment for process evaluation
Twelve Bolton Clarke Residential Aged Care Facilities in Queensland, Australia have been recruited to participate in the EDDIE+ study. The stepped wedge design involved four phases (preparation, baseline/usual care exposure, intervention introduction and intervention exposure) that took place from March 2021 to May 2022. The process evaluation will be conducted from May to September 2022 with data from all participating homes. This will include recruitment of RAC staff, clinical facilitators, family members of residents (where applicable), and local and external stakeholders including General Practitioners (GPs), home managers and allied health managers (see table 2).
Quantitative data
Quantitative data will be extracted from baseline context mapping, communication, activity tracking and check in sheets, and resident family awareness questionnaires (see table 2). These data will include the hours of EDDIE+ training, days of intervention exposure, home structure (bed number, staff, occupancy), local services and communication mechanisms. The evaluation of these data will inform intervention fidelity.
Preintervention and postintervention staff-efficacy surveys will be collected using a validated questionnaire.17 The questionnaire comprises three sections. Section 1 provides information about the staff member’s demographics, their role at the facility, years worked at the facility, years worked in aged care and their qualifications. Section 2 is a 5-point Likert scale with 10 statements related to job self-efficacy. The statements include job related confidence and ability, having the required skills to perform the job well and how they compare themselves to others in the field. Section 3 is a 5-point Likert scale with seven statements related to team self-efficacy. Section 3 has questions related to team members’ skills, abilities, and effectiveness in relation to completing their own tasks and functioning as a team.
Qualitative data
Qualitative data will be primarily collected from a series of semi-structured interviews with staff, family members and advocates of residents, EDDIE+ clinical facilitators, the nurse educator, project implementation facilitator and external stakeholders. Interviewees will be recruited by email and direct correspondence. Staff at participating RAC sites will be invited to participate in an interview by the project implementation facilitator during one of the end of intervention site visits. Relevant family members and stakeholders from the participating RAC homes will be identified by the EDDIE+ facilitator and BC investigators and details forwarded to the QUT project team. The QUT project team will then make contact through email correspondence. Once written consent is obtained, interviewee details will be passed on through email to investigators leading the process evaluation (EB and GH) who will coordinate a mutual time for the interview.
Participation will be voluntary and informed consent will be obtained prior to the conduct of the interview. Additional qualitative data will be extracted from communication tracking field notes, baseline context assessments and check in forms where relevant. These data will address multiple aims of the process evaluation such as the acceptability of EDDIE+, contextual barriers and enablers, and the mechanisms of action (table 2).
Staff, local and external Stakeholder interviews
At intervention completion the RAC staff, including those in managerial positions, and external stakeholders such as GPs and allied health providers, will be invited to participate in semi-structured interviews. Interviews will be up to 30 min in length and completed via telephone or Microsoft Teams. Topics to be covered during the interview include feasibility of implementation, adaptation and tailoring of EDDIE+, what worked and did not work, and factors to consider for sustainability and future scale up of EDDIE+ in other RAC homes (see online supplemental file). Additionally, an open-ended interview will be conducted with the nurse educator and project implementation facilitator after the completion of the trial to ascertain their reflections and experience of the EDDIE+ intervention and implementation process.
Family and nominated advocate interviews
At intervention completion, family members and nominated advocates of residents, including those who have and those who have not experienced clinical deterioration, will be invited to participate in a short interview either via telephone or using Microsoft Teams. Interviews with family members and advocates are anticipated to take around 15 min dependent on interviewee responses and knowledge of the programme. Questions will explore their awareness and experience of EDDIE+.
All interviewees who have signed the consent form and completed an interview will be allocated a unique identifier to maintain confidentiality. No identifiable information will be reported in the findings from these interviews. Interviews will take place up to 4 months post-trial with a maximum of 30 interviews per stakeholder group across the 12 sites.
Data analysis
Quantitative data
Descriptive statistics related to the process evaluation (counts, mean, SD) will be analysed in Microsoft Excel to determine the communication level and engagement from each site based on the quantity of emails, meetings and phone calls. Job-related and team-related self-efficacy data from nursing and personal care workers will be subject to descriptive and inferential analysis using SPSS version 14.0 to assess whether EDDIE+ improved staff’s perceived self-efficacy postintervention. The baseline self-efficacy survey will be completed immediately prior to the participant’s (RN, EN, PCW) first EDDIE+ training session while postintervention self-efficacy surveys will be provided to staff between the final 2 weeks of the intervention exposure and up to 2 weeks post-trial.
Internal consistency of job-related and team-related self-efficacy will be assessed separately using Cronbach’s alpha. Differences between mean baseline and postintervention scores on the self-efficacy measures will be assessed using t-tests, to determine if there is a statistically significant (p<0.05) change in job-related self-efficacy and team-related self-efficacy. Linear regression will be used to determine the contribution of staff-related factors including role, experience, age, gender and location, to changes in job-related and team-related self-efficacy scores. Missing outcome data from staff lost to follow-up will be treated as missing completely at random and handled using complete case analysis.
Qualitative data
Semi-structured interviews will be digitally recorded with consent from the interviewee and transcribed using Microsoft software. Once transcribed and checked for accuracy, interview transcripts will be mapped against the i-PARIHS constructs of innovation, recipients, context and facilitation using NVivo qualitative data software. Additionally, qualitative data will be extracted from the baseline context mapping as well as communication, activity tracking and check in forms where appropriate and mapped to the i-PARIHS framework. Data that do not align with the i-PARIHS framework will be analysed using a descriptive qualitative approach.18 Transcripts will be read by two members of the project team with qualitative research experience and content analysis will be used to code data, group codes into categories and identify major themes.19 The analysis will be complete once agreement between researchers is attained and no new themes emerge.
Integrating results of data analysis
Process evaluation data analysis will be undertaken independently of the analysis of the effectiveness data from the trial. Once the trial results are available, combined analysis will be undertaken to determine the extent to which the process evaluation helps explain the main trial findings.
Patient and public involvement
There is no planned resident or public involvement in the design of the process evaluation due to the COVID-19 pandemic and restricted access to RAC settings. While recognising this as a potential limitation to the study, family members and nominated advocates of residents will be invited to participate in interviews and surveys as part of the process evaluation.
Ethics and dissemination
Ethical approval for this study has been granted by the Bolton Clarke Human Research Ethics Committee (approval number: 170031) with administrative ethical approval granted by the Queensland University of Technology University Human Research Ethics Committee (2000000618). Full ethical approval includes a waiver of consent for access to residents’ demographic, clinical and health services de-identified data. A separate health services data linkage based on RAC home addresses will be sought through a Public Health Act application. Group or individual interviews will require written consent prior to commencement. Protocol amendments will be submitted as variations to the approving ethics committees at time of identification. Additionally, the project manager will notify committees in the circumstance of protocol deviations and adverse events in accordance with local procedures.
Study findings will be disseminated through traditional academic channels, such as journal publications and conference presentations, alongside more interactive strategies, including engagement with a stakeholder network established to embed knowledge translation within the research.
Discussion
Early detection and management of deterioration in residents of aged care homes could result in a decrease of avoidable and unnecessary hospital transfers. The original EDDIE programme was considered feasible, well received, and reduced total hospital bed days by 41%.6 7 However, these promising results were inferred using a relatively small sample size and a predesign and postdesign that did not control for external trends. Following the success of EDDIE in a single site, a modified version of the pilot (EDDIE+) was developed. A stepped wedge randomised controlled trial involving 12 RAC homes will evaluate the effectiveness and cost-consequences of EDDIE+ with the aim of confirming preliminary findings and strengthening the evidence base for wider implementation. The embedded process evaluation will explore whether the scaled-up intervention was delivered and implemented as originally proposed, if EDDIE+ was acceptable from the perspective of various stakeholders, the mechanisms of impact through which EDDIE+ improved outcomes (or not), and contextual barriers and enablers that may have influenced implementation. A mixed method, theory-informed approach will provide an in-depth evaluation of the EDDIE+ programme and valuable insights into determinants of implementation success across multiple sites. This could help to identify key factors to consider in the future development and implementation of hospital avoidance programmes such as EDDIE+.
Limitations
Direct resident involvement in the evaluation of EDDIE+ would strengthen the process evaluation, however, this is not achievable during a pandemic that has led to strict visitor lockdowns in RAC. As an alternative strategy, data to reflect residents’ experiences will be collected from family members and nominated advocates.
Another potential limitation is that EDDIE+ is being implemented and evaluated with a single aged care provider in Queensland which could compromise transferability to other aged care settings and providers. However, the RAC facilities involved in EDDIE+ represent a range of metropolitan and rural settings and different socioeconomic populations across Queensland. Furthermore, the original EDDIE intervention was undertaken with a different aged care provider allowing for some comparison. Applying the i-PARIHS framework to collect and analyse data at an individual facility level will enable us to identify the detailed relationships between contextual factors, implementation processes and outcomes, which could inform future scale-up of EDDIE+. Future studies and process evaluations could further explore the generalisability and applicability to other aged care facilities and directly involve residents in the feedback and evaluation of such programmes.
Ethics statements
Patient consent for publication
Acknowledgments
A collaborative research agreement is established between QUT and partnering institutions. We thank the following partnering institutions: Flinders University, Central Queensland University, Bolton Clarke, University of the Sunshine Coast, Metro North Hospital and Health Service and University of Newcastle.
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @Hannah_E_Carter, @GillHar26
Contributors HC, NG, XJL, GH, TD, LC, CM, FO conceived the EDDIE+ study. GH, EB and MA have led the development of the process evaluation. EB and GH drafted the manuscript with input from all contributing authors. All authors critically revised the manuscript and approved the final version.
Funding This project is funded by a National Health and Medical Research Council Medical Research Future Fund grant (GNT1177501) and led by Queensland University of Technology.
Disclaimer The funding body did not have a role in the study design and subsequent protocol paper, nor are the funders involved with ongoing data collection, management, analysis and interpretation.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.