Practice•

Using implementation science theories and frameworks in global health

Abstract

In global health, researchers and decision makers, many of whom have medical, epidemiology or biostatistics background, are increasingly interested in evaluating the implementation of health interventions. Implementation science, particularly for the study of public policies, has existed since at least the 1930s. This science makes compelling use of explicit theories and analytic frameworks that ensure research quality and rigour. Our objective is to inform researchers and decision makers who are not familiar with this research branch about these theories and analytic frameworks. We define four models of causation used in implementation science: intervention theory, frameworks, middle-range theory and grand theory. We then explain how scientists apply these models for three main implementation studies: fidelity assessment, process evaluation and complex evaluation. For each study, we provide concrete examples from research in Cuba and Africa to better understand the implementation of health interventions in global health context. Global health researchers and decision makers with a quantitative background will not become implementation scientists after reading this article. However, we believe they will be more aware of the need for rigorous implementation evaluations of global health interventions, alongside impact evaluations, and in collaboration with social scientists.

Summary box

  • Global health researchers and decision makers tend to favour impact evaluations to the detriment of implementation evaluations of health interventions.

  • Research on global health interventions is not yet sufficiently supported by theories and analytic frameworks.

  • Theories and analytic frameworks ensure the quality and rigour of global health intervention implementation evaluations.

  • Impact evaluations should go hand in hand with implementation evaluations to understand implementation processes, causal mechanisms and contextual factors shaping outcomes of global health interventions.

Introduction

Since the publication in 2015 of the Medical Research Council framework for evaluating the processes of complex public health interventions,1 we have witnessed the increasing awareness of the importance of such evaluations among global health researchers. Our colleagues with epidemiology, statistics, demography or economics background steadily seek our advice and assistance to develop the implementation evaluation component of their intervention research. During our interdisciplinary discussions, we noticed that they rarely made use of theories or analytical frameworks, although the latter ensure research quality and rigour in implementation science (IS).

In this article, we draw from our experience to open the black box of IS for global health researchers who are unfamiliar with its theories and frameworks. We propose a reflection accessible to as many global health stakeholders as possible on how theories and analytical frameworks are used to understand global health interventions. We define a global health intervention as any action, whether local, national or international, implemented in a context where domestic resources are scarce and power issues resulting from dependence on international aid are present. We believe that this article will be useful to researchers new to this research branch and to field stakeholders who collaborate with researchers or plan such evaluations.

IS is the scientific investigation of factors associated with effective implementation,2 where the roles of context,3 4 actors, ideas, institutions and power are central to analysis.5–7 For instance, health workers’ ideas about healthcare user fees abolition influenced policy implementation in Africa.8–10 Similarly, Burkina Faso’s social context partly explains the heterogeneity of the childbirth user fee subsidy policy outcomes,11 as well as its implementation gap for reducing women’s out-of-pocket expenditures.12 Implementation is the process of putting an intervention (action/project/policy)—either evidence based or theory based—into use in a specific setting.13 Some authors have proposed 14 steps for effective implementation14 and others 23 factors that may influence it.15 The concept of implementation is now considered sufficiently mature16 to be investigated in greater depth in global health. Such studies are even more important in global health, as health interventions implemented in low-income and middle-income settings often originate from, and are funded by, stakeholders from high-income countries.17 18 This contrast results in power struggles and relationships among actors, institutions and contexts that inevitably influence the implementation of interventions.

There is now consensus that global health interventions are complex and that it is necessary to adopt methodological approaches to address this complexity. Understanding their implementation, while not easy, has become essential.19 As an example, a study in Mauritania showed no impact of an obstetrical risk insurance scheme,20 whereas the qualitative study revealed that its implementation had not been adapted to health system dysfunctions.21 Even advocates of randomised control trials (RCTs) are compelled to use qualitative methods to better understand the causal mechanisms of effective interventions.22

Analysis of the implementation of interventions originates as early as the 1930s23 and therefore largely predates the current renewed interest. The present enthusiasm for implementation has been boosted by the development of implementation research,24 which has a dedicated journal (Implementation Science25) and prompted the development of methodological guides.26 However, IS differs from implementation research, in that the latter focuses on methods for promoting the use of evidence in designing an intervention. It does not specifically aim to analyse its implementation. IS, however, is an umbrella term including the analysis of the processes of interventions (process evaluation), the analysis of the fidelity of implementation (fidelity assessment) and especially the relationships with social actors and context.1 27 IS has an instrumental objective, which is to understand the factors affecting the implementation of an intervention. IS is a research branch that mobilises both qualitative and quantitative data, for example, to measure the fidelity or acceptability of an intervention.28

Global health researchers, research funders and decision makers are increasingly interested in understanding why some interventions fail while other succeed in different contexts. There are at least three corollaries to this growing enthusiasm. First, impact evaluation researchers, who conduct efficacy studies (in controlled environments) rather than effectiveness studies (in real-life settings), often tend to quantify or measure rather than try to uncover the complexity of processes, successes or failures using qualitative or mixed methods. Most of these researchers are also not trained in other methodological approaches, particularly from the social sciences,29 and do not know the theories and analytical frameworks used in IS.30 Second, IS publications of global health interventions23 31 are still rare. There are few concrete examples and few reflective analyses of the challenges of IS in these specific contexts.31 Third, a recent review of studies from low-income and middle-income countries (LMICs) between 1998 and 2016 showed that ‘only five articles used an explicit or published (…) model or theory’.32 This scarcity inhibits the dissemination of ‘good practices’ and exposes the lack of robust studies.

The objective of this article is to raise awareness among global health researchers and decision makers about how theories and analytical frameworks can be used to make sense of health interventions and their implementation in context and conduct rigorous implementation evaluations. This article is not intended for social science experts or evaluators who used ‘theory as method’.33 It targets researchers and decision makers trained in quantitative methods who wish to deepen their understanding of global health interventions in context.

Using theory in implementation science

In global health intervention research, theory-based evaluation is frequently promoted.29 33 In the field of evaluation, theory refers to the intervention theory, that is, the description of how an intervention unfolds and brings about change and of the relationships between inputs, outputs and outcomes.34 35 In social science, theories explain, rather than describe, the causal relationships between a phenomenon and an outcome. Along with conceptual frameworks, they are used to guide the research process, especially for analysis and interpretation.36

A plethora of conceptual frameworks exist to analyse implementation.37 ‘[D]eliberately using conceptual or theoretical frameworks to deepen analysis’ is essential. However, ‘selecting an implementation framework is a challenging task’.38 The novice researcher can quickly become lost in the proliferation of existing approaches. A recent survey revealed the use of about 100 different approaches.39 Although this survey shows the abundance of opportunities, it especially underscores the challenges of selecting the appropriate framework or theory, particularly when there is no clear understanding of how they differ.39 In addition to this challenge, researchers may experience the ‘temptation (…) to try to make the data fit, thereby reducing both the analytical value and its burden’.40 They may also be lured into choosing the most fashionable theory or the most ‘off the shelf’,41 losing sight of the most relevant one.

Today, we are in the third generation of IS research, which promotes a ‘rigorous research design’.42 However, according to Saetren, ‘[w]e are not even close to a well-developed theory of policy implementation’.23 Franks and Schroeder2 confirmed that ‘[t]he theoretical base for implementation is relatively new and needs to be tested and operationalized in real-world settings’. Many researchers thus use ‘bricolages’.43 They amalgamate several theories or conceptual frameworks but rarely explain how choices were made. Moreover, several conceptual frameworks deal simultaneously with implementation (process evaluation or fidelity assessment) and outputs/outcomes of an intervention (reach, sustainability and impact), such as RE-AIM (http://www.re-aim.org) or EPIS (https://episframework.com). To summarise, there is no such thing as a miracle theory or magic bullet framework.7 44

To help disentangle the possible approaches, frameworks and theories,44–47 table 1 defines four models of causation commonly used in IS and suggests essential readings for each model. These models form a continuum on an abstraction and complexity ladder. However, they may overlap when, for example, researchers borrow concepts from middle-range or grand theories to build an intervention theory or expand a conceptual framework. Researchers may use these models for three main implementation studies: fidelity assessment, process evaluation and complex evaluation. Nilsen48 also proposed a taxonomy of theories, models, and frameworks to make sense of implementation. His taxonomy differs from ours in two aspects. First, our definition of IS encompasses all types of interventions and does not solely refer to knowledge translation interventions. Second, his taxonomy is organised according to the overarching aims of theoretical approaches, while ours uses as a starting point the consistent confusion about levels of abstraction and complexity. Recently, Kislov and colleagues25 published a commentary in Implementation Science, where they provide a similar account of three different levels of theories. Our classification of frameworks and theories is complementary because it introduces a fourth level so that frameworks, which are part of IS practice, are included. To our knowledge, it is also the first account of the application of models of causation in the context of global health. With this article, our aim is not to standardise IS research practices but rather to contribute to strengthening our global health research practices and reflexivity.

Table 1
•
Four models of causation

At the end of the continuum (table 1), the approaches are much more complex and call on grand theories or major social theories, such as symbolic interactionism.44 Not all social scientists agree on the existence and relevance of grand theories, which are sometimes associated with ideologies (eg, Marxism, socialism and positivism49). We nevertheless retain the term to support our pedagogical demonstration. To our knowledge, grand theories are not often used to study implementation in global health. Social scientists, however, may call on such theories, for instance, when using the concept of Foucault’s biopower to understand HIV interventions,50 or Sen’s capability theory to understand the implementation of user fees exemption interventions in Burkina Faso.51 In IS, the theory is used from the beginning of the research and supports the analysis. It may also be the research output when the aim is to refine the theory.30 52

In the remainder of the article, we explain the use of these four models for fidelity assessment, process evaluation and complex evaluation. We define these three implementation studies, explain how they relate to the four models of causation and provide illustrations from global health (table 2).

Table 2
•
Three main implementation studies

Fidelity assessment

Fidelity is the degree to which an intervention is implemented as intended. Ensuring fidelity increases the chance of achieving the intended effects, bearing in mind that, in real-life settings, it is inconceivable to control the factors that may influence them. A comprehensive framework for implementation fidelity involves: (1) measuring the intervention’s adherence to content, coverage, frequency and duration; (2) understanding the factors moderating the level of fidelity achieved (eg, intervention complexity, facilitation strategies, quality of delivery and participant responsiveness); and (3) identifying essential components.53 54

While high fidelity is desired, adaptation (ie, users’ modification of the original design of the intervention) is likely to occur.53 55 Moreover, certain interventions are adaptive, such that implementers are allowed, or encouraged, to make changes to the original design for better adjustment to the context, ownership and sustainability.56 However, some modifications may detract from the expected outcomes.57 Hence, it is advisable to apply this framework to analyse negative adaptations as well.56 58

At the beginning of the continuum (table 1) is the intervention theory, whose causal logic is used to guide research questions and data collection in order to understand implementation. It is a long-standing approach in the field of evaluation.34 59 Following this approach, researchers propose a model of how the intervention was planned and is supposed to work according to its designers. There are many guides and articles to support researchers in this process60 and to help them involve intervention stakeholders. The intervention theory is usually a visual representation, which comes with a narrative. It may be simple and linear or display multiple layers of causal pathways. Useful illustrations include the intervention theory of the free caesarean section policy in Benin61 or that of a WHO programme (figure 1) implemented in many countries,62 which is explained below (box 3).

Figure 1
Request permission

Modelling of the intervention theory of the universal health coverage partnership. Source: ref 62.

Fidelity assessment, using the intervention theory, makes it possible to explain, along with process evaluation, the production or absence of effects. We recently used fidelity assessment to analyse a results-based financing intervention in Burkina Faso, where a process evaluation and a fidelity assessment were also conducted.63 64 Some journals require authors who submit papers on intervention evaluation to use a grid describing the intervention.65 However, they do not request a description of the intervention theory. To fully understand an intervention’s theory, fidelity assessment is a compelling initial step for grasping the complexity and opening the black box of global health interventions.

Evaluation experts have long warned against type III error; epidemiologists and statisticians are well trained to deal with a type I error (rejecting a ‘true’ null hypothesis) or a type II error (failing to reject a ‘false’ null hypothesis), which results from evaluating an intervention that has not been entirely or adequately implemented.66 This is why implementation fidelity assessment (see box 1) is essential, although still underused. Of the 90 RCTs of public health interventions in LMICs with a study protocol published in a publicly available trial registry from January 2012 to May 2016, 28% did not include any implementation fidelity assessment.67 In Burkina Faso, we carried out an impact assessment of a community control intervention against Aedes aegypti, the vector for dengue fever,68 along with an assessment of its implementation fidelity.69 Other examples include assessing the implementation fidelity of a performance-based financing intervention in Malawi70 and Burkina Faso64 and of an arctic char distribution intervention in Nunavik (Canada).71

Box 1

Fidelity assessment in Cuba

We assessed the implementation fidelity of an evidence-based empowerment strategy for Aedes aegypti control that was replicated in 16 communities in Havana, Cuba. Due to the adaptive nature of the intervention, we focused on adaptation rather than on fidelity. The intervention components and subcomponents were classified as implemented, not implemented or modified, based on qualitative process data collected by implementers. Qualitative data were transformed into quantitative data. Frequencies were tabulated for all the communities, and the mean/average was calculated for each component. Semistructured interviews were also conducted with coordinators of the intervention at different levels to identify implementation challenges. The assessment showed implementation variations according to the communities and components of the strategy. It was not possible to identify negative adaptations nor to provide a detailed account of fidelity.94.

Lessons learnt:

  • Researchers should apply a comprehensive conceptual framework for implementation fidelity to categorise adaptations.

  • They should develop a comprehensive description of the intervention making explicit its functioning principles, that is, the intervention theory.

  • They should keep the intervention theory in mind to identify adaptations that might detract from outcomes.

Process evaluation

While fidelity assessment makes it possible to document what has been done compared with what was planned, process evaluation aims to understand how the intervention unfolded and how different factors influenced its implementation. Such factors include the internal dynamics of the intervention, organisational, socioeconomic or other contextual elements and stakeholders’ behaviours.27 30

Further along the continuum in terms of abstraction (table 1), process evaluations may rely on intervention theories and descriptive frameworks that divide the implementation of interventions into different categories or constructs. An example is the Consolidated Framework for Implementation Research (CFIR), which we used in Burkina Faso (box 2). A recent systematic review showed that the CFIR is increasingly used worldwide, including in LMICs, where it was used 27 times.72 Several researchers have adapted the CFIR to fit their contexts, showing that frameworks can be adjusted according to the research needs. The CFIR was for example adapted to study acceptability of a health intervention in Zambia73 or to investigate sustainability in Ghana.74 The CFIR may be mobilised to support data collection according to a deductive approach and/or at the data analysis stage to sort out data collected according to an inductive approach.72

Box 2

CFIR process evaluation in West Africa

In 2016, we used the Consolidated Framework for Implementation Research (CFIR) in Burkina Faso. We analysed the implementation of a community intervention to combat Aedes aegypti in addition to conducting impact studies,68 a process evaluation (not yet published) and a fidelity assessment.69 This triple evaluation was guided by the intervention theory developed with stakeholders during the evaluability assessment. The 16 CFIR constructs were chosen and adapted based on the context, the nature of the intervention and the availability of data. The data collected were qualitative (focus group, interviews and documentation). Like other researchers,95 we also asked stakeholders to quantitatively assess the influence of each construct on implementation (eg, network and leaders) on a scale of −2 to +2. These scores had no statistical value but helped research participants organise their ideas better. By comparing scores among the three intervention areas, we were able to collect statements to understand the heterogeneity of implementation, as was done in Mozambique.90 This process evaluation provided a better understanding of the contextual factors that influenced implementation and also underscored the factors that contributed to the intervention’s effectiveness. The use of the CFIR was particularly beneficial in that it was complementary to the fidelity assessment. Qualitative sociologists were initially reluctant to use a framework that was too precise. However, interdisciplinary discussions showed that the CFIR could be used openly and be successfully adapted. The participatory adaptation of the CFIR opened the discussions about the challenges of the intervention. Some elements of the CFIR were less appropriate or absent in the global health context.

Lessons learnt:

  • Researchers should adapt the CFIR to context and research needs.

  • They should explain why each construct is selected or eliminated.

  • Research team should discuss and reach a consensus on the meaning of each construct, including its translation into local/national languages.

  • Researchers should translate and operationalise each construct to facilitate data collection.

  • They should remain open and attentive to the emergence of empirical data that may not be related to previously defined constructs.

  • They should consider studying contextual disparities and heterogeneity of implementation and explanatory factors.

Besides descriptive frameworks, researchers may also use conceptual or theoretical frameworks, which are analytical. Such frameworks provide causal propositions for how different factors may influence—negatively or positively—implementation and outcomes. For example, researchers mobilised the theoretical literature on the determinants of access to skilled birth attendance to investigate heterogeneous outcomes of a maternal health policy in Burkina Faso.11 First, however, they analysed the policy implementation using the intervention theory.75

Complex evaluation

Complex evaluation is about analysing implementation and provides critical evidence about the implementation process and its outcomes in relation to, and not in isolation from, other elements of the context that may influence the intervention. Complex evaluation does not assume that an intervention is complex per se. Instead, complexity refers to ‘understanding the social systems within which interventions are implemented as complex’.76

The realist approach to evaluation, which is gaining interest in global health research,77 78 falls within the realm of complex evaluation. Pawson and Tilley52 propose to disentangle the complexity inherent in social interventions by uncovering interactions among an intervention, its stakeholders (implementers or beneficiaries), the multiple layers of context (eg, social and institution) within which they interact and (un)expected outcomes. The realist approach starts from the intervention theory and moves to a middle-range theory that considers multiple contextual influences to make sense of expected and unexpected outcomes of an intervention. In global health, an example would be the middle-range theory on user fee exemption policies in Africa proposed by Robert et al.79 This middle-range theory is based on Sen’s capability approach. It also considers theories and frameworks on access to healthcare to explain why such policies may lead to heterogeneous outcomes in different places or at different times. Another realist evaluation investigated a programme supporting health policy dialogue for universal health coverage (box 3), implemented by WHO in several countries under different implementation arrangements.62

Box 3

Realist evaluation of the Universal Health Coverage Partnership (UHC-P)

Supported by several stakeholders, the Universal Health Coverage Partnership (UHC-P) is a WHO-implemented programme that supports low-income and middle-income countries in organising health policy dialogues to produce robust and evidence-informed health policies for universal health coverage (https://uhcpartnership.net). The first step in the UHC-P evaluation was to design its intervention theory, which was informed by a literature review on policy dialogue96 and several meetings and interviews with key stakeholders. This initial theory was then divided into two subtheories to expose the different support strategies (eg, financial support, ongoing or ad hoc technical support, information and data generation), related mechanisms (eg, trust, empowerment of ministries of health and mutual understanding of values) and potential contextual influences.62 These subtheories guided data collection in six African countries, where qualitative case studies were conducted on health planning policy dialogue (Togo, Cape Verde and Niger), health financing policy dialogue (Burkina Faso and Democratic Republic of Congo) and aid coordination policy dialogue (Liberia). Data analysis consisted of: (1) a descriptive analysis of UHC-P implementation barriers and facilitators and (2) a realist analysis of interactions among the UHC-P components and outcomes, highlighting explanatory mechanisms, along a chain of causal events.

Lessons learnt:

  • Researchers should inform stakeholders, especially those who design the intervention, about the nature of the research and the methodological approach, and involve them in modelling the intervention theory.

  • They should consult as much conceptual and empirical scientific literature as possible to identify potential mechanisms and contextual influences.

  • They should identify intervention barriers and facilitators as a first step to uncovering Context Mechanisms Outcome (CMO) configurations.

The realist approach belongs to theory-based evaluation. It is based on the premise that interventions are complex because they are ‘theories incarnate’.52 It postulates that an intervention is effective because activities have been set up and because they mobilise social actors, whose choices influence the life of the intervention. Social actors’ reactions, reasoning and choices are called mechanisms. Mechanisms are hidden but real and may be triggered in a given context, producing outcomes.80 A realist study will uncover patterns of regularities in the interaction of contexts, mechanisms and outcomes, called CMO configurations. A website is dedicated to the realist approach (http://www.ramesesproject.org), and reporting standards have been published.81

Conclusion

This article is an introduction to IS and three main implementation studies for global health. Our aim was not to provide an exhaustive description of all the concepts, theories and examples in this research branch. Global health researchers with a quantitative background will not become implementation scientists after reading this article. However, we believe they will be more aware of the need for rigorous implementation evaluations of global health interventions, alongside impact evaluations. We encourage policy makers and practitioners to use this article to dialogue with researchers and ensure a better use of theories and analytic frameworks to plan and conduct rigorous global health intervention implementation evaluations. We also encourage all of them to study implementation in collaboration with colleagues from social science and to conduct intervention research collectively in interdisciplinary teams. ‘(B)y learning from other researchers one increases the possibilities of creative solutions’.82 The major contribution of this article is to enlighten policy makers, practitioners and quantitative researchers about the main implementation studies and models of causation, so that they actively contribute to more robust implementation evaluations of global health interventions.

  • Handling editor: Seye Abimbola

  • Twitter: @ValeryRidde, @emilie_robert_

  • Contributors: VR came up with the idea for the article, and then the three authors developed the content together on the basis of their collective and respective experiences.

  • Funding: The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests: VR and ER have been funded as research consultants by the Department of Health Systems Governance and Financing of WHO.

  • Patient consent for publication: Not required.

  • Provenance and peer review: Not commissioned; externally peer reviewed.

  • Data availability statement: No data for this article.

Acknowledgements

We wish to thank Lara Gautier, Annabel Desgrées du Loû and Joseph Larmarange for insightful input.

  1. close Moore GF, Audrey S, Barker M, et al. Process evaluation of complex interventions: medical Research Council guidance. BMJ 2015; 350.
    doi:10.1136/bmj.h1258•Google Scholar•PubMed
  2. close Franks B, Schroeder J. Implementation Science: What Do We Know and Where Do We Go From Here? In: Applying implementation science in early childhood programs and systems. Baltimore: : Paul H. Brookes Pub. Co 2013;
    Google Scholar
  3. close Pfadenhauer LM, Gerhardus A, Mozygemba K, et al. Making sense of complexity in context and implementation: the context and implementation of complex interventions (CICI) framework. Implement Sci 2017; 12:21.
    doi:10.1186/s13012-017-0552-5•Google Scholar•PubMed
  4. close Craig P, Di Ruggiero E, Frohlich KL, et al. Taking account of context in population health intervention research: guidance for producers, users and funders of research. 2018;
    Google Scholar
  5. close RIDDE VALÉRY, Ridde V. Policy implementation in an African state: an extension of Kingdon"s MULTIPLE-STREAMS approach. Public Adm 2009; 87:938–54.
    Google Scholar
  6. close Béland D. Kingdon reconsidered: ideas, interests and institutions in comparative policy analysis. Journal of Comparative Policy Analysis: Research and Practice 2016; 18:228–42.
    doi:10.1080/13876988.2015.1029770•Google Scholar
  7. close Gilson L, Orgill M, Shroff ZC, et al. Health policy analysis reader The politics of policy change in low- and middle-income countries. S.l.: : World Health Organization. 2018;
    Available: here
    Google Scholar
  8. close Walker L, Gilson L. "We are bitter but we are satisfied": nurses as street-level bureaucrats in South Africa. Soc Sci Med 2004; 59:1251–61.
    doi:10.1016/j.socscimed.2003.12.020•Google Scholar•PubMed
  9. close Ridde V, Samb O. Des politiques de gratuité confrontées aux idées pro-paiement des médicaments en Afrique de l’Ouest, Anthropologie du médicament au Sud. La pharmaceuticalisation ses marges. Paris, Éditions de l’IRD 2015;
    Google Scholar
  10. close Béland D, Ideas RV. And policy implementation: understanding the resistance against free health care in Africa. Glob Health Gov 2016; 10:9–23.
    Google Scholar
  11. close Belaid L, Ridde V. Contextual factors as a key to understanding the heterogeneity of effects of a maternal health policy in Burkina Faso? Health Policy Plan 2015; 30:309–21.
    doi:10.1093/heapol/czu012•Google Scholar•PubMed
  12. close Ridde V, Kouanda S, Yameogo M, et al. Why do women pay more than they should? a mixed methods study of the implementation gap in a policy to subsidize the costs of deliveries in Burkina Faso. Eval Program Plann 2013; 36:145–52.
    doi:10.1016/j.evalprogplan.2012.09.005•Google Scholar•PubMed
  13. close Damschroder LJ, Aron DC, Keith RE, et al. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009; 4:50.
    doi:10.1186/1748-5908-4-50•Google Scholar•PubMed
  14. close Meyers DC, Durlak JA, Wandersman A, et al. The quality implementation framework: a synthesis of critical steps in the implementation process. Am J Community Psychol 2012; 50:462–80.
    doi:10.1007/s10464-012-9522-x•Google Scholar•PubMed
  15. close Durlak JA. What Everyone Should Know About Implementation. In: Handbook of social and emotional learning: research and practice. New York, The Guilford Press 2015;
    Google Scholar
  16. close Pfadenhauer LM, Mozygemba K, Gerhardus A, et al. Context and implementation: a concept analysis towards conceptual maturity. Z Evid Fortbild Qual Gesundhwes 2015; 109:103–14.
    doi:10.1016/j.zefq.2015.01.004•Google Scholar•PubMed
  17. close Gautier L, Ridde V. Health financing policies in Sub-Saharan Africa: government ownership or donors" influence? A scoping review of policymaking processes. Glob Health Res Policy 2017; 2:23.
    doi:10.1186/s41256-017-0043-x•Google Scholar•PubMed
  18. close Olivier de Sardan J-P, Diarra A, Moha M, et al. Travelling models and the challenge of pragmatic contexts and practical norms: the case of maternal health. Health Res Policy Syst 2017; 15:60.
    doi:10.1186/s12961-017-0213-9•Google Scholar•PubMed
  19. close Durlak JA. Studying program implementation is not easy but it is essential. Prev Sci 2015; 16:1123–7.
    doi:10.1007/s11121-015-0606-3•Google Scholar•PubMed
  20. close Philibert A, Ravit M, Ridde V, et al. Maternal and neonatal health impact of obstetrical risk insurance scheme in Mauritania: a Quasi experimental before-and-after study. Health Policy Plan
    Google Scholar
  21. close Fauveau V, Diarra A, Amar Z, et al. Evaluation du projet d’appui l’extension du forfait obstétrical en Mauritanie. Paris, Agence française de développement 2016;
    Google Scholar
  22. close Mannell J, Davis K. Evaluating complex health interventions with randomized controlled trials: how do we improve the use of qualitative methods? Qual Health Res 2019; 29:623–31.
    doi:10.1177/1049732319831032•Google Scholar•PubMed
  23. close Saetren H. Facts and myths about research on public policy implementation: out-of-fashion, allegedly dead, but still very much alive and relevant. Policy Studies J 2005; 33:559–82.
    doi:10.1111/j.1541-0072.2005.00133.x•Google Scholar
  24. close Peters DH, Adam T, Alonge O, et al. Implementation research: what it is and how to do it. BMJ 2013; 347.
    doi:10.1136/bmj.f6753•Google Scholar•PubMed
  25. close Kislov R, Pope C, Martin GP, et al. Harnessing the power of theorising in implementation science. Implement Sci 2019; 14:103.
    doi:10.1186/s13012-019-0957-4•Google Scholar•PubMed
  26. close WHO TDR. Implementation research toolkit workbook. Geneva, World Health Organization on behalf of the Special Programme for Research and Training in Tropical Diseases 2014;
    Google Scholar
  27. close Patton MQ, Evaluation U-F. Thousand Oaks-London-New Delhi. Sage Publications 1997;
    Google Scholar
  28. close Sekhon M, Cartwright M, Francis JJ, et al. Acceptability of healthcare interventions: an overview of reviews and development of a theoretical framework. BMC Health Serv Res 2017; 17.
    doi:10.1186/s12913-017-2031-8•Google Scholar•PubMed
  29. close Van Belle S, van de Pas R, Marchal B, et al. Towards an agenda for implementation science in global health: there is nothing more practical than good (social science) theories. BMJ Glob Health 2017; 2.
    doi:10.1136/bmjgh-2016-000181•Google Scholar•PubMed
  30. close Linnan L, Steckler A. Process evaluation for public health interventions and research: an overview, Process evaluation for public health interventions and research. San Francisco, CA, Jossey-Bass 2002;
    Google Scholar
  31. close Ridde V. Need for more and better implementation science in global health. BMJ Glob Health 2016; 1.
    doi:10.1136/bmjgh-2016-000115•Google Scholar•PubMed
  32. close Alonge O, Rodriguez DC, Brandes N, et al. How is implementation research applied to advance health in low-income and middle-income countries? BMJ Glob Health 2019; 4.
    doi:10.1136/bmjgh-2018-001257•Google Scholar•PubMed
  33. close Lipsey MW. Theory as method: small theories of treatments. New Dir Eval 1993; 114:30–62.
    Google Scholar
  34. close Weiss CH. Evaluation: methods for studying programs and policies. Upper Saddle River, NJ, Prentice Hall 1998;
    Google Scholar
  35. close Chen H-T. Theory-driven evaluation, Encyclopedia of evaluation. Thousand Oaks, Sage Publications 2004;
    Google Scholar
  36. close Chen HT. Interfacing theories of program with theories of evaluation for advancing evaluation practice: Reductionism, systems thinking, and pragmatic synthesis. Eval Program Plann 2016; 59:109–18.
    doi:10.1016/j.evalprogplan.2016.05.012•Google Scholar•PubMed
  37. close Walt G, Shiffman J, Schneider H, et al. "Doing" health policy analysis: methodological and conceptual reflections and challenges. Health Policy Plan 2008; 23:308–17.
    doi:10.1093/heapol/czn024•Google Scholar•PubMed
  38. close Moullin JC, Sabater-Hernández D, Fernandez-Llimos F, et al. A systematic review of implementation frameworks of innovations in healthcare and resulting generic implementation framework. Health Res Policy Syst 2015; 13:16.
    doi:10.1186/s12961-015-0005-z•Google Scholar•PubMed
  39. close Birken SA, Powell BJ, Shea CM, et al. Criteria for selecting implementation science theories and frameworks: results from an international survey. Implement Sci 2017; 12:124.
    doi:10.1186/s13012-017-0656-y•Google Scholar•PubMed
  40. close Flemming K, Booth A, Garside R, et al. Qualitative evidence synthesis for complex interventions and Guideline development: clarification of the purpose, designs and relevant methods. BMJ Glob Health 2019; 4.
    doi:10.1136/bmjgh-2018-000882•Google Scholar
  41. close Moore GF, Evans RE. What theory, for whom and in which context? reflections on the application of theory in the development and evaluation of complex population health interventions. SSM - Population Health 2017; 3:132–5.
    doi:10.1016/j.ssmph.2016.12.005•Google Scholar
  42. close Sætren H, Hupe PL. Policy Implementation in an Age of Governance, The Palgrave Handbook of public administration and management in Europe. London, Palgrave Macmillan UK 2018;
    Google Scholar
  43. close Denzin NK, Lincoln YS. Strategies of qualitative inquiry. Thousand Oaks, CA, Sage 2003;
    Google Scholar
  44. close Anfara VA, Frameworks T. In: The SAGE Encyclopedia of Qualitative Research Methods. 2455 Teller Road, Thousand Oaks California 91320 United States: : SAGE Publications. Inc 2008;
    Google Scholar
  45. close Schlager E. A comparison of Frameworks, Theories, and Models of Policy Processes, Theories of the policy process. theoretical lenses on public policy. Boulder, Colo, Westview Press 2006;
    Google Scholar
  46. close Sinclair M. A guide to understanding theoretical and conceptual frameworks. Based Midwifery 2007; 5:39.
    Google Scholar
  47. close Imenda S. Is there a conceptual difference between theoretical and conceptual frameworks? J Soc Sci 2014; 38:185–95.
    doi:10.1080/09718923.2014.11893249•Google Scholar
  48. close Nilsen P. Making sense of implementation theories, models and frameworks. Implementation Sci 2015; 10:53.
    doi:10.1186/s13012-015-0242-0•Google Scholar
  49. close Wiarda H. Grand theories and ideologies in the social sciences. New York, Palgrave Macmillan 2010;
    Google Scholar
  50. close Nguyen V-K. The republic of therapy: triage and sovereignty in West Africa’s time of AIDS. Durham, NC, Duke University Press 2010;
    Google Scholar
  51. close Samb OM, Ridde V. The impact of free healthcare on women"s capability: a qualitative study in rural Burkina Faso. Soc Sci Med 2018; 197:9–16.
    doi:10.1016/j.socscimed.2017.11.045•Google Scholar
  52. close Pawson R, Tilley N. Realistic evaluation. London ; Thousand Oaks, Calif, Sage Publications 1997;
    Google Scholar
  53. close Carroll C, Patterson M, Wood S, et al. A conceptual framework for implementation fidelity. Implementation Sci 2007; 2:40.
    doi:10.1186/1748-5908-2-40•Google Scholar
  54. close Hasson H. Systematic evaluation of implementation fidelity of complex interventions in health and social care. Implementation Sci 2010; 5:67.
    doi:10.1186/1748-5908-5-67•Google Scholar
  55. close Pérez D, Van der Stuyft P, Ridde V, et al. To the bone: Comment on “I wanted a skeleton … they brought a prince”: A qualitative investigation of factors mediating the implementation of a Performance Based Incentive program in Malawi. SSM - Population Health 2019; 7.
    doi:10.1016/j.ssmph.2018.10.005•Google Scholar
  56. close Pérez D, Van der Stuyft P, Zabala MdelC, et al. A modified theoretical framework to assess implementation fidelity of adaptive public health interventions. Implementation Sci 2015; 11:91.
    doi:10.1186/s13012-016-0457-8•Google Scholar
  57. close Rebchook GM, Kegeles SM, Huebner D, et al. Translating research into practice: the dissemination and initial implementation of an Evidence–Based HIV prevention program. AIDS Education and Prevention 2006; 18:119–36.
    doi:10.1521/aeap.2006.18.supp.119•Google Scholar
  58. close von Thiele Schwarz U, Hasson H, Lindfors P, et al. Applying a fidelity framework to understand adaptations in an occupational health intervention. Work 2015; 51:195–203.
    doi:10.3233/WOR-141840•Google Scholar
  59. close Chen H-T. Practical program evaluation : assessing and improving planning, implementation, and effectiveness. Thousand Oaks, Calif. Sage 2005;
    Google Scholar
  60. close Renger R, Titcomb A. A three-step approach to teaching logic models. American Journal of Evaluation 2002; 23:493–503.
    doi:10.1177/109821400202300409•Google Scholar
  61. close Dossou J-P, De Brouwere V, Van Belle S, et al. Opening the ‘implementation black-box’ of the user fee exemption policy for caesarean section in Benin: a realist evaluation. Health Policy Plan 2020; 35:153–66.
    doi:10.1093/heapol/czz146•Google Scholar
  62. close Robert E, Ridde V, Rajan D, et al. Realist evaluation of the role of the universal health coverage partnership in strengthening policy dialogue for health planning and financing: a protocol. BMJ Open 2019; 9.
    doi:10.1136/bmjopen-2018-022345•Google Scholar
  63. close Ridde V, Yaogo M, Zongo S, et al. Twelve months of implementation of health care performance-based financing in Burkina Faso: a qualitative multiple case study. Int J Health Plann Manage
    Google Scholar
  64. close Bodson O, Barro A, Turcotte-Tremblay A-M, et al. A study on the implementation fidelity of the performance-based financing policy in Burkina Faso after 12 months. Arch Public Health 2018; 76.
    doi:10.1186/s13690-017-0250-4•Google Scholar
  65. close Campbell M, Katikireddi SV, Hoffmann T, et al. TIDieR-PHP: a reporting guideline for population health and policy interventions. BMJ 2018;
    doi:10.1136/bmj.k1079•Google Scholar
  66. close Dobson D, Cook TD. Avoiding type III error in program evaluation : Results from a field experiment. Eval Program Plann 1980; 3:269–76.
    Google Scholar
  67. close Pérez MC, Minoyan N, Ridde V, et al. Comparison of registered and published intervention fidelity assessment in cluster randomised trials of public health interventions in low- and middle-income countries: systematic review. Trials 2018; 19.
    doi:10.1186/s13063-018-2796-z•Google Scholar
  68. close Ouédraogo S, Benmarhnia T, Bonnet E, et al. Evaluation of effectiveness of a community-based intervention for control of dengue virus vector, Ouagadougou, Burkina Faso. Emerg Infect Dis 2018; 24:1859–67.
    doi:10.3201/eid2410.180069•Google Scholar
  69. close Saré D, Pérez D, Somé P-A, et al. Community-Based dengue control intervention in Ouagadougou: intervention theory and implementation fidelity. Glob Health Res Policy 2018;
    Google Scholar
  70. close McMahon SA, Muula AS, De Allegri M, et al. “I wanted a skeleton … they brought a prince”: A qualitative investigation of factors mediating the implementation of a Performance Based Incentive program in Malawi. SSM - Population Health 2018; 5:64–72.
    doi:10.1016/j.ssmph.2018.04.006•Google Scholar
  71. close Gautier L, Pirkle CM, Furgal C, et al. Assessment of the implementation fidelity of the Arctic char distribution project in Nunavik, Quebec. BMJ Glob Health 2016; 1.
    doi:10.1136/bmjgh-2016-000093•Google Scholar
  72. close Means A, Kemp C, Gwayi-Chore M-C, et al. Evaluating applications of the consolidated framework for implementation research (CFIR) in low- and middle-income country settings. Implement Sci Proc 11th Annu Conf Sci Dissem Implement Wash DC USA 3-5 2019;
    Google Scholar
  73. close Jones DL, Rodriguez VJ, Butts SA, et al. Increasing acceptability and uptake of voluntary male medical circumcision in Zambia: implementing and disseminating an evidence-based intervention. Transl Behav Med 2018; 8:907–16.
    doi:10.1093/tbm/iby078•Google Scholar
  74. close Gyamfi J, Allegrante JP, Iwelunmor J, et al. Application of the Consolidated Framework for Implementation Research to examine nurses’ perception of the task shifting strategy for hypertension control trial in Ghana. BMC Health Serv Res 2020; 20.
    doi:10.1186/s12913-020-4912-5•Google Scholar
  75. close Belaid L, Ridde V. An implementation evaluation of a policy aiming to improve financial access to maternal health care in Djibo district, Burkina Faso. BMC Pregnancy Childbirth 2012; 12.
    doi:10.1186/1471-2393-12-143•Google Scholar
  76. close Walton M. Applying complexity theory: a review to inform evaluation design. Eval Program Plann 2014; 45:119–26.
    doi:10.1016/j.evalprogplan.2014.04.002•Google Scholar
  77. close Marchal B, Dedzo M, Kegels G, et al. A realist evaluation of the management of a well- performing regional hospital in Ghana. BMC Health Serv Res 2010; 10.
    doi:10.1186/1472-6963-10-24•Google Scholar
  78. close Mukumbang FC, van Wyk B, Van Belle S, et al. Unravelling how and why the antiretroviral adherence Club intervention works (or not) in a public health facility: a realist explanatory theory-building case study. PLoS One 2019; 14.
    doi:10.1371/journal.pone.0210565•Google Scholar
  79. close Robert E, Samb OM, Marchal B, et al. Building a middle-range theory of free public healthcare seeking in sub-Saharan Africa: a realist review. Health Policy Plan 2017; 32:1002–14.
    doi:10.1093/heapol/czx035•Google Scholar
  80. close Lacouture A, Breton E, Guichard A, et al. The concept of mechanism from a realist approach: a scoping review to facilitate its operationalization in public health program evaluation. Implementation Sci 2015; 10:153.
    doi:10.1186/s13012-015-0345-7•Google Scholar
  81. close Wong G, Westhorp G, Manzano A, et al. RAMESES II reporting standards for realist evaluations. BMC Med 2016; 14.
    doi:10.1186/s12916-016-0643-1•Google Scholar
  82. close Bhaskar R, Danermark B, Price L, et al. Interdisciplinarity and wellbeing: a critical realist general theory of interdisciplinarity. Abingdon, Oxon ; New York, NY, Routledge 2017;
    Google Scholar
  83. Weiss CH. Theory-based evaluation: past, present, and future. New Dir Eval 1997; 1997:41–55.
    doi:10.1002/ev.1086•Google Scholar
  84. Durlak JA, DuPre EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol 2008; 41:327–50.
    Google Scholar
  85. Merton RK. Social theory and social structure., Enlarged ed., [Nachdr]. New York, NY, Free Press 1968;
    Google Scholar
  86. Pawson R. Middle range theory and programprogramme theory evaluation: from provenance to practice, Mind the gap: perspectives on policy evaluation and the social sciences. New York, Routledge 2010;
    Google Scholar
  87. Astbury B, Leeuw FL. Unpacking black boxes: mechanisms and theory building in evaluation. Am J Eval 2010; 31:363–81.
    Google Scholar
  88. Dossou J-P, Cresswell JA, Makoutodé P, et al. ‘Rowing against the current’: the policy process and effects of removing user fees for caesarean sections in Benin. BMJ Glob Health 2018; 3.
    doi:10.1136/bmjgh-2017-000537•Google Scholar
  89. Vareilles G, Marchal B, Kane S, et al. Understanding the motivation and performance of community health volunteers involved in the delivery of health programmes in Kampala, Uganda: a realist evaluation. BMJ Open 2015; 5.
    doi:10.1136/bmjopen-2015-008614•Google Scholar
  90. close Cole CB, Pacca J, Mehl A, et al. Toward communities as systems: a sequential mixed methods study to understand factors enabling implementation of a skilled birth attendance intervention in Nampula Province, Mozambique. Reprod Health 2018; 15.
    doi:10.1186/s12978-018-0574-8•Google Scholar
  91. Renmans D, Holvoet N, Criel B, et al. Combining Theory-Driven Evaluation and Causal Loop Diagramming for Opening the ‘Black Box’ of an Intervention in the Health Sector: A Case of Performance-Based Financing in Western Uganda. Int J Environ Res Public Health 2017; 14.
    doi:10.3390/ijerph14091007•Google Scholar
  92. Seppey M, Ridde V, Touré L, et al. Donor-funded project’s sustainability assessment: a qualitative case study of a results-based financing pilot in Koulikoro region, Mali. Glob Health 2017;
    Google Scholar
  93. Turcotte-Tremblay A-M, Gali-Gali IA, De Allegri M, et al. The unintended consequences of community Verifications for performance-based financing in Burkina Faso. Soc Sci Med
    Google Scholar
  94. close Perez D, Lefevre P, Castro M, et al. Process-oriented fidelity research assists in evaluation, adjustment and scaling-up of community-based interventions. Health Policy Plan 2011; 26:413–22.
    doi:10.1093/heapol/czq077•Google Scholar
  95. close Damschroder LJ, Lowery JC. Evaluation of a large-scale weight management program using the consolidated framework for implementation research (CFIR). Implementation Sci 2013; 8.
    doi:10.1186/1748-5908-8-51•Google Scholar
  96. close Robert E, Ridde V. ‘Policy dialogue’ (dialogue sur les politiques) : une synthèse exploratoire des écrits scientifique. 2018;
    Available: here
    Google Scholar

  • Received: 22 December 2019
  • Accepted: 27 March 2020
  • First published: 16 April 2020

Article metrics
Altmetricopen-url
Dimensionsopen-url