Research & reporting methodology•

Development and pilot testing of quality improvement indicators for integrated primary dementia care

Abstract

Background Implementation of integrated primary care is considered an important strategy to overcome fragmentation and improve quality of dementia care. However, current quality indicator (QI) sets, to assess and improve quality of care, do not address the interprofessional context. The aim of this research was to construct a feasible and content-wise valid minimum dataset (MDS) to measure the quality of integrated primary dementia care.

Methods A modified Delphi method in four rounds was performed. Stakeholders (n=15) (1) developed a preliminary QI set and (2) assessed relevance and feasibility of QIs via a survey (n=84); thereafter, (3) results were discussed for content validity during a stakeholder and (4) expert consensus meeting (n=8 and n=7, respectively). The stakeholders were professionals, informal caregivers, and care organisation managers or policy officers; the experts were professionals and researchers. The final set was pilot-tested for feasibility by multidisciplinary dementia care networks.

Results The preliminary set consisted of 40 QIs. In the survey, mean scores for relevance ranged from 5.8 (SD=2.7) to 8.5 (SD=0.7) on a 9-point Likert scale, and 25% of all QIs were considered feasible to collect. Consensus panels reduced the set to 15 QIs to be used for pilot testing: 5 quality of care, 3 well-being, 4 network-based care, and 3 cost-efficiency QIs. During pilot testing, all QIs were fully completed, except for well-being QIs.

Conclusion A valid and feasible MDS of QIs for primary dementia care was developed, containing innovative QIs on well-being, network-based care and cost-efficiency, in addition to quality of care QIs. Application of the MDS may contribute to development and implementation of integrated care service delivery for primary dementia care.

Introduction

The complexity of primary dementia care, caused by the complex nature of this clinical syndrome, often associated comorbidity, and the multiple professionals involved poses a great challenge to healthcare systems.1 2 As a result, dementia care is often fragmented, which leads to poor quality and inefficient care, because different professionals provide unaligned services, resulting in patients’ and caregivers’ low satisfaction with the services offered.3–5 Care integration is considered important to overcome fragmentation and improve quality of care.6–9

Integrated care is defined as the delivery of a continuum of care, designed to meet multidimensional needs of the population and the individual, by a coordinated multidisciplinary team of professionals.10 To achieve dementia care integration, a transition towards network-based care is needed.11–13 DementiaNet, a healthcare innovation consisting of 36 multidisciplinary networks of primary care professionals, facilitates such a transition.11 14 The essential characteristics of DementiaNet are summarised in box 1. To evaluate the effectiveness of such an integrated care approach in practice and to identify areas for improvement, a tailored, feasible and valid set of quality indicators (QIs) is needed.15 This need is based on extensive literature search and expert consensus on QIs based on patient-reported outcomes of physiotherapy and geriatric patient care showing that, when connected firmly with feedback loops to the healthcare, it can improve outcomes and efficiency on the levels of direct patient care and health services.16 Several QI sets have been developed to assess quality of primary dementia care, including a set of six QIs used to assess DementiaNet networks’ quality of care.17–21 However, these sets do not sufficiently adopt the interprofessional context. Furthermore, existing integrated QI sets do not include well-being issues,22 or their large number of indicators limits feasibility.23 Therefore, the aims of this study were to construct a content-wise valid minimum dataset (MDS) of QIs to measure quality of integrated primary dementia care in co-creation with stakeholders and to pilot test its feasibility in practice.

Box 1

What is DementiaNet?

DementiaNet is a Dutch healthcare innovation focusing on primary network-based care. Networks of professionals from medical, care and social disciplines, caring for the same persons with dementia (PWDs), are formed.14 Currently, the DementiaNet approach is being applied in more than 40 networks. DementiaNet facilitates gradual development of self-organised primary care networks providing high-quality integrated dementia care. Strategies to achieve this include clinical leadership and quality improvement cycles. With DementiaNet, we address needs of professionals (increase knowledge, skills and collaboration); PWD and informal caregivers (personalised care, integration of care and welfare, care coordination and continuity of care); and the healthcare system (local level, tailor-made, system approach, cost-effectiveness and sustainable approach). Full description can be found in the paper of Nieuwboer et al,14 and the first quantitative evaluation of effectiveness is described by Richters et al.21

Methods

Design

Between May 2018 and July 2019, a modified Delphi method was conducted. This methodology is particularly useful to reach agreement on content validity and feasibility; inclusion of a large number of participants prevents domination during the consensus process.24 25 In four rounds, we (1) developed a preliminary QI set, (2) sent out a survey to stakeholders, (3) discussed results during a stakeholder consensus meeting and (4) discussed results during an expert consensus meeting. Thereafter, the final set was pilot-tested for feasibility.

Delphi method

Participants

Participants were stakeholders and experts in the field of primary dementia care. Stakeholders were healthcare professionals, informal caregivers, and care organisation managers or policy officers. Experts were professionals and researchers, including all authors, specialised in research or care. Professionals from each discipline were balanced for each round.

Stakeholders for the first physical meeting (round 1) and survey (round 2) were invited by email and recruited via the DementiaNet newsletter database to ensure their familiarity with integrated primary dementia care. Additional caregivers for the survey were recruited via the outpatient clinic or through project contacts. Stakeholders interested to participate in round 1 and participants who showed interest during round 2 were invited for the second physical meeting (round 3). Purposeful sampling was used for the meetings (rounds 1 and 3); we made sure all relevant primary care disciplines were represented, and the representatives were seen as experts by their colleagues. To include non-expert opinions as well, random sampling of a large group of stakeholders was used for the survey (round 2).

Participants for the expert consensus meeting (round 4) were selected based on their involvement in the DementiaNet project; all were employees at the Radboudumc Alzheimer Centre or at the General Practitioner Organisation of Nijmegen.

Delphi rounds

1. Development of the preliminary set

In a brainstorm session with stakeholders, relevant and easy-to-implement indicators were pre-selected, covering three predefined topics: (1) quality of dementia care (process indicators, patient level); (2) well-being of person with dementia (PWD) and their caregiver (outcome indicators)15 26; and (3) network-based care (process indicators network level).27 Additionally, the existing quality of care indicators used by DementiaNet networks were included in the discussion.11 21 Afterwards, the authors reviewed previous QI sets developed by their group17 21 and subsequently composed the preliminary set for the stakeholder survey.

2. Survey to stakeholders

In an electronic survey, the preliminary QI set was presented to stakeholders. Consent was asked at the beginning of the survey. Caregivers were only asked to comment on quality of care and well-being indicators. A glossary was included to explain medical jargon. Caregivers could request a paper version of the survey.

Stakeholders were asked to rate all indicators individually on relevance by using a 9-point Likert scale from 1 (not relevant) to 9 (highly relevant), to identify the three most important indicators per topic and to add missing indicators. Professionals were also asked to assess QIs on measurement feasibility (yes/no).

Coded questionnaire responses of professionals and caregivers were analysed separately to determine the mean relevance, top 3 indicators and feasibility of all indicators. An indicator was judged feasible when over 50% of respondents indicated the data feasible to collect.28 Descriptive analyses were performed using IBM SPSS V.25.0.

3. Stakeholder consensus meeting

Survey results, for professionals and caregivers separately, were presented to a stakeholder panel for their judgement of content validity. Inclusion of additional indicators, revision of QIs, and exclusion based on relevance and feasibility were discussed.

4. Expert consensus meeting

Results of the survey and stakeholder consensus meeting were distributed to the experts a week prior to the meeting. The expert panel composed the MDS for pilot testing, based on the rated relevance, arguments from the consensus meeting, content validity, feasibility and correspondence with current guidelines.

Pilot testing

Participants

DementiaNet networks were selected to pilot the new set of indicators (MDS) based on their proactive attitude and timing of their networks’ yearly assessment.11

Pilot test

Networks reported their experiences via an open-ended questionnaire, including QI relevance, feasibility of data collection and added value of outcomes.

The authors narratively analysed the questionnaires and evaluated the collected data to determine measurement feasibility. Acceptable measurement feasibility was defined as data collection per QI completed for over 80% of the cases. Thereafter, the set of indicators was adjusted accordingly.

Results

We invited 312 stakeholders for round 1; 19 were willing to participate, and 15 were able to participate. Thirty-one were invited for round 3; 8 were able to participate. Four stakeholders participated in both rounds. Diversity in stakeholders’ characteristics was large, with general practitioners, practice nurses, case managers, community nurses and caregivers being most represented. All invited experts agreed to participate. Table 1 describes participants’ characteristics.

Table 1
•
Participants’ background and years of experience in numbers (and percentages for round 2) defined per round

Indicator development

Figure 1 summarises the process of QI development.

Figure 1
Request permission

Subsequent stages of the QI development process for integrated primary dementia care. QI, quality indicator; QoC, quality of care.

1. Development of the preliminary set

Fifteen stakeholders developed the preliminary QI set. The meeting yielded a set of 40 indicators; 11 for quality of care, 12 for well-being of PWD and their caregiver and 17 for network-based care, including all 6 QIs previously used by DementiaNet.11 21 No indicators were added by the authors after reviewing previously developed QI sets.

2. Stakeholder survey

Eighty-four stakeholders fully, and 30 stakeholders partly completed the assessment of the 40 QIs (table 1). Mean relevance scores per indicator ranged from 6.0 (SD=2.1) to 8.5 (SD=0.7) for professionals and from 5.8 (SD=2.7) to 8.3 (SD=0.9) for caregivers. Professionals considered collection of 25% of the indicators feasible. Table 2 shows relevance, top 3 and feasibility scores per indicator.

Table 2
•
Relevance (by professionals and caregivers) and feasibility (by professionals and managers/policy officers) of QIs to measure integrated primary dementia care assessed via a survey and final consensus

3. Stakeholder consensus meeting

Eight stakeholders discussed the survey results. Stakeholders generally agreed with the survey respondents on relevance and prioritisation. Disagreement existed on the QI ‘recent pharmacotherapy check’, which had a low relevance and feasibility score, though stakeholders considered the indicator crucial for assessing quality of care. They argued that polypharmacy negatively influences quality of life and morbidity. Two new well-being QIs were proposed for inclusion: ‘discussing changing relationship between PWD and caregiver’ and ‘PWD satisfaction with daily activities’. The QI about PWDs’ goals was nominated for exclusion as it was considered not specific and not feasible.

4. Expert consensus meeting

A geriatrician, two general practitioners, a nursing scientist, two researchers and a coordinator of General Practitioner Organisation of Nijmegen decided on the composition of the MDS for pilot testing considering primarily relevance but also feasibility. The panel agreed with the stakeholders on adding the QI on PWD satisfaction with daily activities. Three network QIs about task division and communication were merged. Experts added the QI: ‘whether the network has formulated a well-being related goal for the PWD’, which replaced some of the excluded network and well-being QIs. During the discussion, cost-efficiency was added as a separate topic, and QIs were arranged accordingly. An additional QI was included for cost-efficiency: ‘whether care of PWD is reimbursed from the long-term care finance system’. Twenty-six QIs were excluded, resulting in a pilot-testing set of 15 QIs: five quality of care, three well-being, four network and three cost-efficiency QIs.

Pilot testing

Three DementiaNet networks voluntarily pilot-tested the QI set. Regarding relevance and added value, they preferred the new set above their currently used QIs, as they stated that it represented more aspects of the network collaboration. This information helped them to identify new improvement goals. The QIs focusing on PWD care wishes and on support for caregivers were considered of added value. Measurement feasibility was good; it took one to two hours, depending on the size of the caseload, to complete the QIs. QIs were fully completed, except for the well-being QIs; on average, these were completed for 14% (perseverance time) and 42% (satisfaction with daily activities) of their PWD (range: 11–31). These well-being indicators were difficult to collect as they required caregiver consultation as an additional time-consuming action. Participants suggested the case manager should complete these questions. Furthermore, pilot answers suggested ambiguity of two QIs, which were therefore reformulated (table 3).

Table 3
•
Final set of QI descriptions for integrated primary dementia care

Discussion

An MDS of 15 QIs was developed to measure the quality of integrated primary dementia care; QIs for quality of care, well-being, network-based care, and cost-efficiency were included. Most QIs were overall judged positively regarding content validity and feasibility by primary dementia care stakeholders. During pilot testing, most indicators were considered relevant, of added value and feasible to collect. However, collection of the well-being QIs still needs specific attention.

Strengths and limitations

An important novelty of this set is the inclusion of well-being, network-based care and cost-efficiency indicators, which supports the triple-aim ambitions; improve quality of care, population health and cost-effectiveness.29 Previous sets mainly included quality of care indicators.19 Moreover, this novel MDS for integrated primary dementia care is substantially shorter than existing sets.23 The comprehensiveness of the set, in combination with its high relevance and feasibility, make the MDS useful for daily practice; based on the pilot test future acceptance is expected to be high.

Another strength of the set is its fit with the validated Rainbow Model of Integrated Care.27 This model’s four levels of care provision, the service, professional, organisational and system levels, are all represented in the MDS. QIs on quality of care and well-being refer to the service level: processes and outcomes of care for the PWD and caregiver (eg, QIs on multidisciplinary meetings and satisfaction of PWD with daily activities). Network QIs refer to the professional and organisational levels, the required collaboration between professionals and organisations (eg, QIs on task division and communication). Cost-efficiency QIs refer to the system level, the impact of regulations on collaboration between professionals and organisations (eg, QI on PWD care reimbursement).

Last, stakeholder meetings and the survey included a diverse group of participants regarding background, age and informal caregiver’s roles. Their interaction and discussions created broad support for the QIs included; acceptability of the MDS is therefore expected to be high. However, the stakeholders were mainly active in DementiaNet networks; the MDS is therefore highly applicable to the Dutch network-based dementia care context. Adaptations to the MDS might be necessary when using the set for other healthcare systems in other countries.30 However, because of their generic content, the QIs can be highly transferable to other populations, for example, vulnerable older adults. Collection of the well-being QIs proved difficult. Evaluation of new strategies to collect these QIs is needed since their relevance is well recognised in practice.

Future implications for research and practice

By addressing various levels of integrated care, the MDS is highly useful for performance feedback in an interdisciplinary setting to encourage professionals to improve the quality of services for their joint caseload. Care coordination, interdisciplinary teamwork and personalised care can be improved by setting improvement goals and assess change over time. Moreover, the complexity in dementia care is captured by clustering QIs in levels related to each other; better network collaboration leads to better quality of care, better patient and caregiver well-being and/or lower healthcare costs.

The MDS can inform policy makers on best practices to achieve (cost-)effective network-based dementia care. As most QIs in the MDS are not dementia specific, it is applicable in other populations, for example, vulnerable older adults, which will increase acceptability by preventing fragmentation due to disease-specific QIs.

Future research should include the use of the MDS in long-term follow-up evaluations of integrated primary dementia care programmes such as DementiaNet. However, first reliability testing and assessing sensitivity for relevant change are needed.

Conclusion

A concise set of QIs with proven feasibility and content validity was developed to assess integrated primary dementia care. Innovative QIs on different relevant aspects of integrated care, well-being, network-based care and cost-efficiency were included next to quality of care QIs. By monitoring changes in QI scores over time and subsequent care improvement cycles, professionals in interdisciplinary primary dementia care collaborations can improve the quality of service for their joint caseload. As such, the set may contribute to the implementation of guidelines and care pathways for integrated primary dementia care.

  • Contributors: DO composed the preliminary set of quality indicators, was a member of the expert panel, was responsible for the research question and the design of this study, analysed the data, wrote the first draft and revised the manuscript. MN composed the preliminary set of quality indicators and was a member of the expert panel, commented on the design, and contributed to drafting and revising the manuscript. MOR was a member of the expert panel, commented on the design, and contributed to drafting and revising the manuscript. MP composed the preliminary set of quality indicators, was coresponsible for the research question and the design of this study, was a member of the expert panel, commented on the design, and contributed to drafting and revising the manuscript.

  • Funding: Funding was provided The Netherlands Organization for Health Research and Development.

  • Competing interests: None declared.

  • Patient and public involvement: Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.

  • Patient consent for publication: Not required.

  • Provenance and peer review: Not commissioned; externally peer reviewed.

  • Data availability statement: All data relevant to the study are included in the article.

Acknowledgements

We thank Chantal Hensens, Lonneke Wittenberg and Simone van Dulmen for their participation in the expert panel.

  1. close Wimo A, Jönsson L, Bond J, et al. The worldwide economic impact of dementia 2010. Alzheimers Dement 2013; 9:1–11.
    doi:10.1016/j.jalz.2012.11.006•Google Scholar•PubMed
  2. close Livingston G, Sommerlad A, Orgeta V, et al. Dementia prevention, intervention, and care. Lancet 2017; 390:2673–734.
    doi:10.1016/S0140-6736(17)31363-6•Google Scholar•PubMed
  3. close World Health Organization. Who global strategy on people-centred and integrated health services. 2015;
    Google Scholar
  4. close Enthoven AC. Integrated delivery systems: the cure for fragmentation. Am J Manag Care 2009; 15:S284–90.
    Google Scholar•PubMed
  5. close Reuben DB. Better care for older people with chronic diseases: an emerging vision. JAMA 2007; 298:2673–4.
    doi:10.1001/jama.298.22.2673•Google Scholar•PubMed
  6. close Goodwin N. How should integrated care address the challenge of people with complex health and social care needs? emerging lessons from international case studies. Int J Integr Care 2015; 15.
    doi:10.5334/ijic.2254•Google Scholar•PubMed
  7. close Kodner DL, Spreeuwenberg C. Integrated care: meaning, logic, applications, and implications--a discussion paper. Int J Integr Care 2002; 2.
    doi:10.5334/ijic.67•Google Scholar•PubMed
  8. close Lichtenstein BJ, Reuben DB, Karlamangla AS, et al. Effect of physician delegation to other healthcare providers on the quality of care for geriatric conditions. J Am Geriatr Soc 2015; 63:2164–70.
    doi:10.1111/jgs.13654•Google Scholar•PubMed
  9. close Callahan CM, Boustani MA, Unverzagt FW, et al. Effectiveness of collaborative care for older adults with Alzheimer disease in primary care: a randomized controlled trial. JAMA 2006; 295:2148–57.
    doi:10.1001/jama.295.18.2148•Google Scholar•PubMed
  10. close World Health Organization Europe. Integrated care models: an overview. 2016;
    Google Scholar
  11. close Richters A, Nieuwboer MS, Perry M, et al. Evaluation of DementiaNet, a network-based primary care innovation for community-dwelling patients with dementia: protocol for a longitudinal mixed methods multiple case study. BMJ Open 2017; 7.
    doi:10.1136/bmjopen-2017-016433•Google Scholar•PubMed
  12. close Wübbeler M, Thyrian JR, Michalowsky B, et al. How do people with dementia utilise primary care physicians and specialists within dementia networks? results of the dementia networks in Germany (DemNet-D) study. Health Soc Care Community 2017; 25:285–94.
    doi:10.1111/hsc.12315•Google Scholar•PubMed
  13. close Draper B, Low L-F, Brodaty H, et al. Integrated care for adults with dementia and other cognitive disorders. Int Rev Psychiatry 2018; 30:272–91.
    doi:10.1080/09540261.2018.1564021•Google Scholar•PubMed
  14. close Nieuwboer MS, Richters A, van der Marck MA, et al. Triple aim improvement for individuals, services and society in dementia care : The DementiaNet collaborative care approach. Z Gerontol Geriatr 2017; 50:78–83.
    doi:10.1007/s00391-017-1196-4•Google Scholar•PubMed
  15. close Donabedian A. Evaluating the quality of medical care. Milbank Mem Fund Q 1966; 44:166–206.
    doi:10.2307/3348969•Google Scholar
  16. close Olde Rikkert MGM, van der Wees PJ, Schoon Y, et al. Using patient reported outcomes measures to promote integrated care. Int J Integr Care 2018; 18.
    doi:10.5334/ijic.3961•Google Scholar•PubMed
  17. close Perry M, Drasković I, van Achterberg T, et al. Development and validation of quality indicators for dementia diagnosis and management in a primary care setting. J Am Geriatr Soc 2010; 58:557–63.
    doi:10.1111/j.1532-5415.2010.02726.x•Google Scholar•PubMed
  18. close Beerens HC, Sutcliffe C, Renom-Guiteras A, et al. Quality of life and quality of care for people with dementia receiving long term institutional care or professional home care: the European RightTimePlaceCare study. J Am Med Dir Assoc 2014; 15:54–61.
    doi:10.1016/j.jamda.2013.09.010•Google Scholar•PubMed
  19. close Joling KJ, van Eenoo L, Vetrano DL, et al. Quality indicators for community care for older people: a systematic review. PLoS One 2018; 13.
    doi:10.1371/journal.pone.0190298•Google Scholar•PubMed
  20. close Vasse E, Moniz-Cook E, Rikkert MO, et al. The development of quality indicators to improve psychosocial care in dementia. Int Psychogeriatr 2012; 24:921–30.
    doi:10.1017/S1041610211002523•Google Scholar•PubMed
  21. close Richters A, Nieuwboer MS, Olde Rikkert MGM, et al. Longitudinal multiple case study on effectiveness of network-based dementia care towards more integration, quality of care, and collaboration in primary care. PLoS One 2018; 13.
    doi:10.1371/journal.pone.0198811•Google Scholar•PubMed
  22. close Heckman GA, Boscart VM, Franco BB, et al. Quality of dementia care in the community: identifying key quality assurance components. Can Geriatr J 2016; 19:164–81.
    doi:10.5770/cgj.19.233•Google Scholar•PubMed
  23. close Kröger E, Tourigny A, Morin D, et al. Selecting process quality indicators for the integrated care of vulnerable older adults affected by cognitive impairment or dementia. BMC Health Serv Res 2007; 7.
    doi:10.1186/1472-6963-7-195•Google Scholar•PubMed
  24. close Boulkedid R, Abdoul H, Loustau M, et al. Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PLoS One 2011; 6.
    doi:10.1371/journal.pone.0020476•Google Scholar•PubMed
  25. close Powell C. The Delphi technique: myths and realities. J Adv Nurs 2003; 41:376–82.
    doi:10.1046/j.1365-2648.2003.02537.x•Google Scholar•PubMed
  26. close Austrom MG, Boustani M, LaMantia MA, et al. Ongoing medical management to maximize health and well-being for persons living with dementia. Gerontologist 2018; 58:S48–57.
    doi:10.1093/geront/gnx147•Google Scholar•PubMed
  27. close Valentijn PP, Schepman SM, Opheij W, et al. Understanding integrated care: a comprehensive conceptual framework based on the integrative functions of primary care. Int J Integr Care 2013; 13.
    doi:10.5334/ijic.886•Google Scholar•PubMed
  28. close Forum NQ. Measure evaluation criteria and guidance for evaluating measures for endorsement. DC, National Quality Forum Washington 2017;
    Google Scholar
  29. close Berwick DM, Nolan TW, Whittington J, et al. The triple aim: care, health, and cost. Health Aff 2008; 27:759–69.
    doi:10.1377/hlthaff.27.3.759•Google Scholar•PubMed
  30. close Steel N, Melzer D, Shekelle PG, et al. Developing quality indicators for older adults: transfer from the USA to the UK is feasible. Qual Saf Health Care 2004; 13:260–4.
    doi:10.1136/qshc.2004.010280•Google Scholar•PubMed

  • Received: 10 January 2020
  • Accepted: 8 May 2020
  • First published: 14 June 2020

Article metrics
Altmetricopen-url
Dimensionsopen-url