- Open Open Peer Review
An introduction to implementation science for the non-specialist
BMC Psychologyvolume 3, Article number: 32 (2015)
The movement of evidence-based practices (EBPs) into routine clinical usage is not spontaneous, but requires focused efforts. The field of implementation science has developed to facilitate the spread of EBPs, including both psychosocial and medical interventions for mental and physical health concerns.
The authors aim to introduce implementation science principles to non-specialist investigators, administrators, and policymakers seeking to become familiar with this emerging field. This introduction is based on published literature and the authors’ experience as researchers in the field, as well as extensive service as implementation science grant reviewers.
Implementation science is “the scientific study of methods to promote the systematic uptake of research findings and other EBPs into routine practice, and, hence, to improve the quality and effectiveness of health services.” Implementation science is distinct from, but shares characteristics with, both quality improvement and dissemination methods. Implementation studies can be either assess naturalistic variability or measure change in response to planned intervention. Implementation studies typically employ mixed quantitative-qualitative designs, identifying factors that impact uptake across multiple levels, including patient, provider, clinic, facility, organization, and often the broader community and policy environment. Accordingly, implementation science requires a solid grounding in theory and the involvement of trans-disciplinary research teams.
The business case for implementation science is clear: As healthcare systems work under increasingly dynamic and resource-constrained conditions, evidence-based strategies are essential in order to ensure that research investments maximize healthcare value and improve public health. Implementation science plays a critical role in supporting these efforts.
Background: what is implementation science and how did it develop?
The need for a science of implementation
It has been widely reported that evidence-based practices (EBPs) take on average 17 years to be incorporated into routine general practice in health care [1–3]. Even this dismal estimate presents an unrealistically rosy projection, as only about half of EBPs ever reach widespread clinical usage .
Historically, this research-to-practice gap has not been the concern of academic clinical researchers. The traditional academic business case for career success has primarily supported conducting descriptive or mechanism-oriented studies and intervention studies on highly selected, typically academic medical center-based populations, and publishing in—ideally—top quality academic journals. Whether these findings translate into public health impact has typically not been the concern of traditional healthcare researchers.
This paradigm for academic success has justly come under criticism in recent years. Payers for biomedical research have been concerned over the lack of public health impact of their research dollars . Moreover, decreasing research funding world-wide has led to debates over the trade-offs between investing in more conservative projects with predictable results versus more innovative research, including projects involving more real-world samples that could result in greater public health impact .
Recognition of the need for research that more directly impacts public health has broadened the academic mindset somewhat, from an exclusive emphasis on efficacy studies to more broadly generalizable effectiveness trials (Table 1) . Several overlapping names, and conceptual structures, have developed for these latter types of trials including “effectiveness trials” [7, 8] “pragmatic clinical trials” “practical clinical trials”  and “large simple trials” .
However, such trials in and of themselves provide little guarantee of public health impact. Effectiveness trials typically depend, like efficacy trials, on a research resources which are separate from the clinical infrastructure and which are externally funded, time-limited, and evanescent at the end of the protocol . Research-supported resources seldom remain at local sites to support continued use of even successful interventions; there is typically little institutional memory for the intervention, and no technology transfer. Moreover, EBPs often have characteristics that make unassisted translation into practice unlikely, especially if the intervention requires incorporating a change in a clinician’s routine .
A useful conceptualization of the biomedical research process has been as a “pipeline”  whereby an intervention moves from efficacy through effectiveness trials to sustained application in general practice. Blockages can appear at various stages, leading to quality gaps as EBPs are applied in less controlled settings. Many factors can impede EBP uptake, including competing demands on frontline providers; lack of knowledge, skills and resources; and misalignment of research evidence with operational priorities can all impede uptake. Accordingly, there is clear need to develop specific strategies to promote the uptake of EBPs into general clinical usage . Implementation science has developed to address these needs.
The organization of this summary
The aim of this review is to introduce non-specialist investigators, administrators, and policymakers to the principles and methods of implementation science. We compiled this review based on published literature, our experience as practicing implementation researchers, and extensive service on implementation science grant review committees. We include examples to illustrate critical principles, utilizing primarily studies of psychosocial EBPs. Most of these examples come from studies funded by the US Department of Veterans Affairs (USVA), since we are able to present these cases with a look “behind the scenes” to provide insights that may not be evident from simply reading the published results. However, it is important to note that the field of implementation science is a global field, as perusal of some of the major publication venues will make evident, such as Implementation Science, BMC Health Services Research, and a variety of specifically themed journals.
In this review we first define implementation science, comparing and contrasting it to the methods of quality improvement and dissemination. We then trace the development of implementation science as a field, illustrating this trajectory with experience in the USVA. We then drill down into the field, beginning with the basics of descriptive methods, including the pivotal role played by theories, models, and frameworks. Next we introduce the fundamentals of controlled intervention trials, including the quickly developing area of hybrid effectiveness-implementation designs. We next provide a detailed description of two real-world implementation studies, and close by placing implementation science into the context of global health policy.
Defining implementation science
Implementation science can be defined as “the scientific study of methods to promote the systematic uptake of research findings and other evidence-based practices into routine practice, and, hence, to improve the quality and effectiveness of health services” . This field incorporates a scope broader than traditional clinical research, focusing not only at the patient level but also at the provider, organization, and policy levels of healthcare (Table 2). Accordingly, implementation research requires trans-disciplinary research teams that include members who are not routinely part of most clinical trials such as health services researchers; economists; sociologists; anthropologists; organizational scientists; and operational partners including administrators, front-line clinicians, and patients.
Implementation sciences, quality improvement, and dissemination
Both implementation science and quality improvement (QI) efforts share the ultimate goal of improving the quality of healthcare. Methods used in the two fields often overlap, although there are some differences. QI efforts usually begin with a specific problem in a specific healthcare system, recognized at the level of the provider, clinic, or health system, and lead to the design and trial of strategies to improve a specific problem for that specific healthcare system. A variety of QI approaches have developed, often taken from other industries such as Toyota Lean, Six Sigma, and others [16, 17].
In contrast, implementation science typically begins with an EBP that is under-utilized, and then identifies and addresses resultant quality gaps at the provider, clinic, or healthcare system level. Additionally, implementation science, as a science, takes as part of its mission an explicit goal of developing generalizable knowledge that can be widely applied beyond the individual system under study. Nonetheless, both approaches share a common goal, value system, desire for rigor, focus on outcomes, and many common methods.
Dissemination, in contrast to implementation, refers to the spread of information about an intervention, assisted at most by educational efforts . Again, however, there is some overlap in that implementation efforts may incorporate dissemination techniques, but they are typically embedded in more comprehensive, targeted, and active efforts to spread the EBP.
Development of implementation science
Over the past 10–15 years healthcare systems that carry responsibility for managing fixed budgets to care for their populations have become increasingly attentive to problems of implementation in order to maximize the value of their healthcare funds . Such organizations include several countries’ national health services, and the USVA in the United States. More recently in the US, new laws such as the Affordable Care Act have led to the development of accountable care organizations, which align care providers, often across hospital systems, and which frequently manage care for a given population under a fixed or constrained reimbursement cap . As a result, US federal research funding organizations have recently developed an investment in implementation science [21, 22]. Another North American program, Knowledge Translation Canada , strives to improve the impact of research findings through implementation science strategies.
Among the oldest and most extensive scientific programs focusing on implementation is the USVA’s Quality Enhancement Research Initiative (QUERI; ), which we provide as a case study in the development of implementation science as a field. Since its inception in 1998, the goal of QUERI has been to improve military veterans’ health by supporting the more rapid movement of effective interventions into practice . QUERI combines scientific rigor provided by leading academic researchers with partnerships with USVA operations leaders who face pressing implementation issues. QUERI is uniquely positioned as an implementation science resource within a national healthcare system, and as a result provides significant academic-operations synergies: QUERI focuses on increasing the health system impact of EBPs of clinical and policy importance to VA operations, and through scientific rigor contributes foundational knowledge to the field of implementation science which can have impact well beyond the USVA system itself.
Reflecting the development of implementation science as a field, QUERI’s first evolutionary leap was its transition from investment in conventional health services research studies to research that focused on description of gaps in the use of EBPs and description of relevant barriers and facilitators. The second evolutionary leap, which is currently ongoing, is a transition from descriptive studies of barrier/facilitators to specifying and testing optimal strategies for implementation, including both controlled trials and observational studies that evaluate natural experiments of practice-based improvement initiatives. Most recently, QUERI’s mission has extended to support implementation of EBPs, as promoted by the US Office of Management and Budget  through the use of rigorous evaluation to study and scale-up programs or initiatives that have demonstrated effectiveness, or use randomized program design to test policies or programs at a population level [26, 27].
Discussion: the principles and methods of implementation science
Implementation Studies Differ from Clinical Studies
The first step in understanding implementation studies is to distinguish implementation processes from the EBPs they seek to implement. An implementation intervention is “a single method or technique to facilitate change,” while an implementation strategy is “an integrated set, bundle, or package of discreet implementation interventions ideally selected to address specific identified barriers to implementation success” . Implementation interventions may include, for example, efforts to change behavior at the patient, provider, system, or even policy level. Common examples include strategies at the provider level such as education/training, audit-feedback, and performance incentives. Strategies targeting the provider, team, or clinic levels may include QI techniques or other systems redesign efforts, team-based performance incentives, learning collaboratives, or community-engagement. Facilitation, guided efforts by internal or external organizational staff to support multiple levels of system change through provider or team-based coaching, is increasingly recognized as critical to the success of many implementation effects [28, 29].
In contrast to clinical research, in which typically focuses on the health effects of an EBP, implementation studies typically focus on rates and quality of use of EBPs rather than their effects. Such EBPs may be as “simple” as increasing use of a single medication such as beta-blockers in individuals who have experienced a myocardial infarction or the use of metabolic side effect monitoring for individuals taking antipsychotic medications, or as complex as instituting psychotherapies like cognitive-behavioral therapy or even multi-component care paradigms such as the collaborative chronic care model .
While the distinction between the implementation strategy and the EBP seems clear in concept, careful attention to their distinct roles is imperative in developing focused hypotheses and making evaluation design decisions. For instance, in studying the effects of a program to increase effective cognitive-behavioral therapy use for bipolar disorder, the impact of cognitive-behavioral therapy on health status would be an EBP outcome (and more typically a focus for a clinical study), while measuring the proportion of clinicians providing cognitive-behavioral therapy, or proportion of patients who attend a minimum level of cognitive-behavioral therapy sessions would be a more typical implementation study outcome.
Evaluating the implementation process and its impact
Thus the crux of implementation studies is their focus on evaluating the process of implementation and its impact on the EBP of interest. These studies can involve one or more of three broad types of evaluation, process evaluation, formative evaluation, and summative evaluation [13, 31].
Process evaluation simply describes the characteristics of use of an EBP (or lack thereof). Data are collected before, during, and/or after the implementation and analyzed by the research team without feedback to the implementation team and without intent to change the ongoing process. Process evaluation can be undertaken in a purely observational study (e.g., in preparation for developing an implementation strategy) or during the course of a spontaneous or planned system or policy change.
Formative evaluation utilizes the same methods as process evaluation, but differs in that data are fed back to the implementation team and/or staff in the target system during the study in order to adapt and improve the process of implementation during the course of the protocol. Formative evaluation is conceptually similar to fidelity monitoring that goes on as part of any traditional clinical trial, but differs in that it is specified a priori in a study hypothesis or research question.
A quantitative version of formative evaluation in clinical trials is the use of sequential multiple assignment randomized trials (SMART) or adaptive intervention designs, which are used to either augment or switch treatments at critical decision points where there is evidence of initial non-response [32, 33]. As we discuss below, the use of formative evaluation has implications for controlled trial validity, which may differ between clinical and implementation trials.
Summative evaluation is a compilation, at study end, of the impact of the implementation strategy. Summative evaluation measures typically assess impacts on processes of care (e.g., increased use or quality of targeted EBP). Another common component of summative evaluation is to characterize the economic impact of an implementation strategy and its effects. Implementation studies most commonly do not employ formal cost-effectiveness analyses but conduct focal “business impact analyses” . Such analyses focus on estimating the financial consequences of adoption of a clinical practice within a specific healthcare setting or system. Typically this includes costs to the system associated with both the implementation strategy and the utilization of the EBP.
Types of evaluation data
Data for the process, formative, and/or summative evaluation can come from various sources and can include either or both of quantitative and qualitative data. Data can be collected across various levels of observation including patients; providers; systems; and broader environmental factors such as community, policy, or economic indices.
Common quantitative measures include structured surveys and tools that assess, for example, organizational context, provider attitudes and behaviors, or patient receptivity to change. Administrative data are often utilized, either in focal target populations or at the broader system level to characterize, for example, baseline and change in rates of utilization of particular practices. Measures of fidelity to the EBP are often central components of the evaluation plan, and these can be quantitative, qualitative, or both.
Common qualitative data collection methods include semi-structured interviews with patients, providers, or other stakeholders; focus groups; direct observation of clinical processes; and document review. Qualitative data collection and analysis can either be structured as a hypothesis-free exploration using grounded theory or related approaches  or can utilize directed content analysis  to address pre-specified issues such as hypothesis-testing or measurement of intervention fidelity. Most implementation evaluation processes include mixed qualitative and quantitative measures, and require careful attention in the study design to the various ways to combine such data .
The role of theory and its link to specific design decisions
The need for theory-based implementation
Describing, implementing, and then sustaining any innovation is a complex undertaking  —complex because implementation strategies (a) are typically multi-component and (b) must adapt to local contexts. Contexts in which implementation efforts occur are themselves complex because of multiple interacting levels (e.g., patients, providers, teams, service units), with wide variation from setting to setting . Without a thorough understanding of the context, an EBP may either not be adopted or may be taken up in an adapted form with compromised fidelity due to contextual pressures. The endeavor requires clear, collective, consistent use of theory to build knowledge about what works, where, and why .
Theories, models, and frameworks
The terms theory, model, and framework are, unfortunately, often used interchangeably and imprecisely. Use of the term theory in this paper refers to any proposal about meaningful relationships between constructs (variables) or how a mechanism or construct may change the behavior of another construct or outcome [41, 42].
A theory may be operationalized within a model, which is a simplified depiction of a more complex world with relatively precise assumptions about cause and effect. For example, the model in Fig. 1 depicts a single hypothesized pathway of change for implementing a weight management program [43, 44]. This model hypothesizes that leadership engagement will result in sufficient resources for implementation, and in leaders’ articulating implementation goals that align with organizational priorities. These leadership actions serve to raise organizational perceptions of priority, which in turn leads to positive implementation outcomes (e.g., high fidelity of the program to its intended design), and thus to better weight loss for patients. Each construct in the model can be assessed and the model tested to affirm or disprove this pathway of change. Based on results, the model may need refinement or be rejected altogether.
Frameworks provide a broad set of constructs that organize concepts and data descriptively without specifying causal relationships. They may also provide a prescriptive series of steps summarizing how implementation should ideally be planned and carried out [13, 45]. For example, the Consolidated Framework for Implementation Research (CFIR, ) classifies 39 implementation constructs across five domains considered to be influential moderators or mediators of implementation outcomes, providing a structure by which to systematically assess context within which implementation occurs . Optimally, researchers will also circle back and utilize empirical data to assess the usefulness of the guiding theories, models, or frameworks used to design the study and offer recommendations to improve them . Consistent use of constructs across studies further allows more efficient syntheses through use of e.g., qualitative comparative analyses techniques . Explicit operationalization of theoretical constructs can also guide development of robust quantitative measures [50, 51].
Example: using a framework to guided data collection and analysis
An example will illustrate the role of such frameworks. Damschroder and Lowery conducted an evaluation of a weight management program in USVA, funded by the QUERI program described above [43, 44]. There was wide variation in program implementation 1.5 years after initial dissemination. CFIR was used to develop a guide to interview staff at local facilities to understand what contributed to this variation. Quantitative ratings were applied to qualitatively coded interview data  and used to identify constructs that distinguished between sites with better versus worse implementation outcomes. Ten constructs were found to be related to successful implementation. For example, the sites with the poorest implementation outcomes had lower ratings for “Goals and Feedback” because staff at these sites did not have the time or skills to identify program goals or monitor progress toward those goals. This information can be used to develop more specific models to guide future implementations and identify discrete implementation strategies (e.g., audit-feedback , identify champions ).
Controlled implementation trials
Types of controlled implementation trials
Many implementation studies seek to identify barriers and facilitators of EBP adoption under naturalistic conditions. However, other studies seek to enhance EBP adoption by employing specific implementation strategies in controlled trials. Controlled implementation trial designs are of two major types: parallel group and interrupted time series. Parallel group designs are randomized and prospective, similar to other types of health services or clinical trials. The unit of randomization depends on the strategy and the outcome of interest, and may be the patient, provider, clinic, facility, or system level. Some creative designs respond to real-world constraints by combining characteristics of historical control studies with true matched randomization . Additional innovative designs have been employed, such as stepped wedge designs in which all participating sites receive implementation support, though staggered in time [54, 55], resembling traditional incomplete block designs. Additionally, controlled implementation trials sometimes randomize a small number of sites and measure outcomes at the individual patient level, utilizing multivariate nested analyses . Finally, SMART randomization designs, as mentioned above , can bring an added degree of rigor to the use of within-study formative evaluation.
In the real world of policy roll-outs and program changes, parallel group designs may not be feasible for pragmatic, political, or ethical reasons. In such situations, interrupted time series designs can be utilized. These also resemble their counterparts among health services and clinical trials, optimally with the implementation outcome of interest measured at multiple time points prior to and after an implementation effort. Such designs are most feasible when the implementation strategy is a discrete event, such as a policy change  since more typical implementation strategies exert their effect over months, making the “interruption” in the time series more difficult to pinpoint.
Implementation versus traditional health services and clinical trials
Controlled implementation trials differ from other types of health services and clinical trials in two major ways. First and most basically, as noted above, they focus on the impact of the implementation strategy on the use of an EBP, rather than health impact of the EBP itself. Second, they take a fundamentally different approach to validity, as illustrated below.
Consider a hypothetical trial of motivational interviewing (MI) for substance use disorders in the homeless population (Table 3). Implementation designs differ from the intervention trials at a basic level, that of the hypothesis. While intervention trials focus on comparing the intervention to a comparison group, implementation trials focus on the ability of the research team to increase adoption of MI. While the population and setting may resemble that for effectiveness trials, the unit of observation in implementation trials may be providers or clinics—depending on the focus of the implementation efforts. As the population becomes more heterogeneous, outcome measures must become briefer and simpler in order to minimize the respondent burden and retain in the protocol subjects who are less research-tolerant. As one moves from left to right in Table 3, intervention clinicians become progressively less highly specialized and skill development efforts more closely resemble training that would typically be feasible under clinical conditions; measurement of fidelity to the intervention varies similarly.
The consideration of the context for the research also differs substantively. Efficacy trials value conducting a trial that is invariant from beginning to end. This entails sometimes extra-ordinary efforts at training clinicians and tracking subjects, so that the research team’s efforts amount at times to “crypto-case management”—which is not a problem since the focus of the trial is solely on testing the intervention under optimal conditions. However, since effectiveness studies are more concerned with performance of an efficacious intervention in real-world settings, careful attention is paid to limiting the amount of research-funded effort that is incorporated into the protocol, often “firewalling” research effort with carefully and explicitly defined parameters (e.g., 8). Further, for implementation trials, trying to optimize the natural context in which adoption is being measured is a threat to validity of the trial. Thus researcher involvement in the site of adoption is drastically circumscribed, with some implementation trials even limiting research support to training endogenous clinicians and utilizing remote, “light-touch” outcome assessments for patient and provider subjects.
These differences are driven by radically different priorities regarding trial validity . Internal validity prioritizes establishing a causal connection between the intervention and the outcome; in the service of this aim, sample, outcome measurements, intervention are all highly controlled without consideration of applicability to other considerations outside of the trial itself. External validity prioritizes the generalizability of trial results to other relevant populations and situations. While efficacy studies value internal over external validity, the balance shifts for effectiveness studies: Although the effectiveness trial must have sufficient internal validity to carry out the study successfully, explicit design decisions are made that value generalizability beyond the trial itself.
Finally, implementation trials, in contrast to both types of intervention trials, aim at arriving at the optimal implementation strategy, and maximal EBP impact, by the end of the study, rather than isolating specific mechanisms. Note, however, that in subtle ways this is the case for all types of studies, including efficacy trials, in which intervention fidelity data are assiduously collected throughout the study and adjustments in training and even personnel are made to ensure internal validity. The difference with implementation trials is that such modifications are planned for through formative evaluation, and often specifically hypothesized about .
Hybrid effectiveness-implementation trial designs
Although an implementation research protocol can demonstrate improvements in the uptake of EBPs, the impact of such practices on health outcomes has not classically been a concern of implementation science. However, it is clear from many studies that the assumption that improved processes mean improved health outcomes is flawed [59, 60]. Additionally, even if EBPs are implemented as planned with adequate fidelity, there may be a “voltage drop” as an EBP moves from the effectiveness clinical trials setting into implementation in real-world clinical practices. Moreover, it is important to know the degree to which an EBP, when utilized in general clinical practice, retains the active mechanisms while adapting to local site needs and priorities [61, 62].
Cognizant of these needs, a group of implementation and clinical trial researchers codified the concept of hybrid effectiveness-implementation designs . “Codify” is the correct term, rather than “develop,” since all the concepts and methodologies involved in hybrid designs already existed across often disparate scientific fields; the purpose of the hybrid design concept paper was to bring together those concepts and methodologies into a single framework and explore the utility of the framework for the needs of real-world researchers and public health stakeholders. In a few short years this framework has begun to be adopted by researchers and funders .
Hybrid research designs are defined as trial designs that take a dual focus a priori of assessing both the effectiveness of the implementation strategy in enhancing the use of the EBP and the health impact of the EBP . Thus hybrid designs pose hypotheses regarding both EBP and implementation effects, typically measuring both healthcare processes (implementation outcomes) and health status (intervention outcomes). Not all hybrid designs apply controlled trial methodologies to both aspects of outcome simultaneously, but all incorporate a priori the assessment of both of these domains explicitly during the development of the research design. There are three types of hybrid designs.
Hybrid Type I designs test the health impact of an EBP while explicitly collecting data on the implementation process to facilitate subsequent implementation efforts. Such designs are typically utilized preliminary to an implementation trial. Some advocates say that all effectiveness trials should be Hybrid Type Is.
Hybrid Type III designs test the ability of an implementation strategy to enhance use of an EBP while collecting data on health impact of the EBP during implementation. These designs are useful when the effectiveness of an intervention is well established, but it is unclear how robust the effects are likely to be under implementation conditions. Some advocates say that all implementation trials that involve processes to improve health outcomes should be Hybrid Type IIIs.
Hybrid Type II designs are a hybrid of hybrids: They test both the EBP effects on health outcome and the implementation strategy effects on EBP use. Such designs are utilized when there is substantial “implementation momentum” within a clinical system, but intervention effectiveness data are somewhat less extensive or only indirect. Thus both the impact of the implementation strategy on EBP usage and the impact of the EBP on health status are the dual foci of Hybrid Type II designs.
Two real-world implementation study examples
A brief description of two USVA QUERI projects will illustrate some of the concepts covered in this review.
Incentives in substance use disorders (Illustrating a type I hybrid design)
The first exemplar [65, 66] is an effectiveness trial with an observational implementation component fully integrated into its design (Hybrid Type I). The rationale behind adding the implementation component to this trial was that if the intervention proved to be effective, information on the most feasible strategies for implementation and the most significant barriers likely to be encountered would already be collected, informing future implementation efforts. As the methods of the randomized, controlled effectiveness trial portion of the study are standard and familiar to most researchers (see also Table 3), this description will focus on the process evaluation. The process evaluation was guided by two frameworks. RE-AIM  guided the investigator to examine the Reach, Effectiveness, Adoption, Implementation and Maintenance of the intervention, posing questions such as how many patients will be interested in receiving the intervention, what information will a clinic require to consider adoption of the intervention, which training, tools and/or resources will the clinic require to implement the proposed intervention with high fidelity and what resources will be required for the clinic to maintain the intervention long-term. In complementary fashion, the PARIHS framework (Promoting Action on Research Implementation in Health Services ; recently revised ) proposes that successful implementation is a function of the strength of the evidence supporting the intervention, the context in which the intervention is implemented and the facilitation provided to support implementation. These constructs guided the investigators to determine how the evidence base for the intervention is perceived by practitioners, which characteristics of the context impeded or facilitated implementation, and which tools and supports facilitated dealing with identified barriers.
The process evaluation for this study combined multiple data collection tools and strategies in an attempt to identify key barriers and facilitators related to each of these constructs. Data collection tools included administrative data, patient surveys, staff surveys and interviews, clinic leadership interviews and cost tracking data. Information from these multiple sources allowed the identification of key barriers and facilitators likely to be encountered in any efforts to implement the incentive intervention in additional clinics and allowed for the creation of recommended implementation strategies (Table 4).
Implementing pharmacotherapy for alcohol use disorders (illustrating formative evaluation)
The second example is an ongoing implementation trial that uses formative evaluation techniques to refine and test a theory-driven implementation strategy to increase access to alcohol use disorder pharmacotherapy in primary care settings. The rationale for including the formative evaluation was that no matter how carefully an implementation strategy is planned in advance, the experience of actual implementation leads to insights which can be utilized to improve the strategy even further. The development of the initial strategy was based on prior formative QUERI work as well as the Theory of Planned Behavior . Earlier qualitative interviews with providers had documented: (1) lack of provider confidence in medication effectiveness, (2) lack of provider skills and knowledge about using the medications, and (3) perceived low patient demand as the top three barriers to implementation. This work suggested that implementation interventions would have to focus not only on education but also on “marketing” and that the interventions would need to focus not only on providers but also on patients.
The Theory of Planned Behavior was used as a guiding theory because the desired behavior change is primarily at the individual level, e.g., convincing providers and patients to consider pharmacotherapy as a viable intervention option for alcohol use disorders. The Theory of Planned Behavior states that behavioral intentions are guided by attitudes toward the behavior, subjective peer norms, and perceived behavioral control and implementation interventions were selected to address each of these constructs. Implementation strategies selected for providers include social marketing, access to peer expert consultants, feedback on prescribing rates, and education and decision support tools. Direct-to-consumer mailings were selected as the intervention to address lack of patient awareness of medication options and potential negative peer attitudes toward pharmacological treatment of alcohol use disorder and to encourage patients to discuss alcohol use with their primary care provider. The implementation strategy is being evaluated using an interrupted time-series design with controls. Prescribing rates are being monitored for 9-month pre-implementation, implementation, and post-implementation phases for intervention facilities and matched control facilities. Already in the study, formative evaluation has led to refinements of the implementation strategy (Table 5).
Healthcare environments around the world are increasingly dynamic, resource-constrained, and interconnected—and are driven by equally complex political and economic environments. Accordingly, maximizing healthcare value  has become a policy imperative globally. Healthcare sciences must evolve in parallel to serve this need.
To this end, implementation science is becoming a critical tool for promoting what the US Institute of Medicine has called the “Learning Healthcare System.” The learning healthcare system can be defined as a healthcare system that employs a set of ongoing processes to provide higher value healthcare through systematic review of health care data analytics, and application of such data to inform promotion of overall strategies and targeted improvement efforts [71, 72].
To date, the bulk of attention to learning healthcare systems has focused on the acquisition and analysis of large datasets gathered from real-world clinical practices in as close to real-time as possible . However, such data-gathering represents only the input arm of the learning healthcare organization. Healthcare systems also need to act on such data to improve practice. Implementation science provides a systematic set of principles and methods by which to accomplish this.
Provider and system behavior do not change by themselves, and unimodal interventions do not typically produce effective or lasting changes in systems as complex as healthcare . Without concerted attention to evidence-based implementation strategies, learning healthcare systems risk developing massive, expensive repositories of information without adequate strategies to actually utilize such data for system change. As with earlier assumptions that mere publication of an efficacy or effectiveness trial results would lead to automatic EBP adoption, the assumption that comprehensive, elegant, real-time datasets will, in and of themselves, lead to practice change is at best an optimistic aspiration and at worst a massively expensive error.
Rather, sophisticated data-gathering systems must be paired with equally sophisticated implementation strategies if learning healthcare systems are to make good on their promise. The emerging science of implementation provides a systematized approach to identifying and addressing barriers and facilitators to system change, and thus represents a critical component of any learning healthcare system.
Availability of data and materials
Consolidated Framework for Implementation Research
United States National Institutes of Health
Promoting action on research Implementation in health services (an implementation framework)
Quality Enhancement Research Initiative
Reach, effectiveness, adoption, implementation, and maintenance (an implementation framework)
Sequential multiple assignment randomized trials
- US VA:
United States Department of Veterans Affairs, Veterans Health Administration
Balas EA, Boren SA. Managing clinical knowledge for health care improvement. Yearbook of Medical Informatics. Schattauer: Stuttgart; 2000. p. 65–70.
Grant J, Green L, Mason B. Basic research and health: a reassessment of the scientific basis for the support of biomedical science. Res Eval. 2003;12:217–24.
Morris ZS, Wooding S, Grant J. The answer is 17 years, what is the question: understanding time lags in translational research. J Roy Soc Med. 2011;104:510–20.
Moses 3rd H, Matheson DH, Cairns-Smith S, George BP, Palisch C, Dorsey ER. The anatomy of medical research: US and international comparisons. JAMA. 2015;313(2):174–89.
Alberts B, Kirschner MW, Tilghman S, Varmus H. Rescuing US biomedical research from its systemic flaws. Proc Natl Acad Sci U S A. 2014;111(16):5773–7.
US Institute of Medicine. Assessing Medical Technologies. Washington: National Academy Press; 1985.
Wells KB. Treatment Research at the Crossroads: The Scientific Interface of Clinical Trials and Effectiveness Research. Am J Psychiatry. 1999;156:5–10.
Bauer MS, Williford W, Dawson E, Akiskal H, Altshuler L, Fye C, et al. Principles of effectiveness trials and their implementation in VA Cooperative Study #430, “Reducing the Efficacy-Effectiveness Gap in Bipolar Disorder.”. J Affect Disord. 2001;67:61–78.
Zwarenstein M, Treweek S, Gagnier JJ, Altman DG, Tunis S, Haynes B, et al. Improving the reporting of pragmatic trials: an extension of the CONSORT statement. BMJ. 2008;337:a2390.
March JS, Silva SG, Compton S, Shapiro M, Califf R, Krishnan R. The case for practical clinical trials in psychiatry. Am J Psychiatry. 2005;162(5):836–46.
Yusuf S, Collins R, Peto R. Why do we need some large, simple randomized trials? Stat Med. 2006;3:409–20.
Rogers EJ. Diffusion of Innovations. 5th ed. New York: Free Press; 2003.
USVA QUERI (US Department of Veterans Affairs; Quality Enhancement Research Initiative, QUERI). http://www.queri.research.va.gov/implementation/default.cfm. Last accessed June 1, 2015.
Woolf SH. The meaning of translational research and why it matters. JAMA. 2008;299:211–13.
Eccles MP, Mittman BS. Welcome to implementation science. ImplementationSci. 2006;1:1.
Mason SE, Nicolay CR, Darzi A. The use of Lean and Six Sigma methodologies in surgery: A systematic review. Surgeon. 2015;13(2):91–100.
D'Andreamatteo A, Ianni L, Lega F, Sargiacomo M. Lean in healthcare: A comprehensive review. Health Policy. 2015; epub ahead of print.
Health Services Research Information Central, US National Institutes of Health. http://www.nlm.nih.gov/hsrinfo/implementation_science.html. Last accessed April 27, 2015.
Berwick DM, Nolan TW, Whittington J. The triple aim: care, health, and cost. Health Aff (Millwood). 2008;27:759–69.
Shields MC, Patel PH, Manning M, Sacks L. A model for integrating independent physicians into accountable care organizations. Health Aff (Millwood). 2011;30(1):161–72.
US National Institutes of Health. http://www.fic.nih.gov/News/Events/implementation-science/Pages/faqs.aspx & http://obssr.od.nih.gov/scientific_areas/translation/dissemination_and_implementation/index.aspx. Last accessed June 1, 2015.
US National Cancer Institute. http://cancercontrol.cancer.gov/IS/. Last accessed March 28, 2015.
Canadian Institute of Health Research. http://ktclearinghouse.ca/ktcanada. Last accessed March 28, 2015.
Demakis JG, McQueen L, Kizer KW, Feussner JR. Quality Enhancement Research Initiative (QUERI): A collaboration between research and clinical practice. Med Care. 2000;38:I17–25.
Bridgeland J, Orszag P. Can the government play Moneyball? The Atlantic Monthly, July/August. 2013.
Allen H, Baicker K, Taubman S, Wright B, Finkelstein A. The Oregon health insurance experiment: when limited policy resources provide research opportunities. J Health Polit Policy Law. 2013;38:1183–92.
Kilbourne AM, Goodrich DE, Lai Z, Almirall D, Nord KM, Bowersox NW, et al. Reengaging veterans with serious mental illness into care: preliminary results from a national randomized trial. Psychiatr Serv. 2015;66:90–3.
Kirchner E, Ritchie M, Pitcock A, Parker LE, Curran GM, Fortney JC. Outcomes of a partnered facilitation strategy to implement primary care-mental health. J Gen Intern Med. 2014;29 suppl 4:S904–12.
Waxmonsky J, Kilbourne AM, Goodrich DE, Nord KM, Lai Z, Laird C, et al. Enhanced fidelity to treatment for bipolar disorder: results from a randomized controlled implementation trial. Psychiatr Serv. 2014;65:81–90.
Woltmann E, Grogan-Kaylor A, Perron B, Georges H, Kilbourne AM, Bauer MS. Comparative effectiveness of collaborative chronic care models for mental health conditions across primary, specialty, and behavioral health settings. systematic review and meta-analysis. Am J Psychiatry. 2012;169:790–804.
Stetler CB, Legro MW, Wallace CM, Bowman C, Guihan M, Hagedorn H, et al. The role of formative evaluation in implementation research and the QUERI experience. J Gen Intern Med. 2006;21 Suppl 2:S1–8.
Kilbourne AM, Almirall D, Eisenberg D, Waxmonsky J, Goodrich DE, Fortney JC, et al. Protocol: Adaptive Implementation of Effective Programs Trial (ADEPT): cluster randomized SMART trial comparing a standard versus enhanced implementation strategy to improve outcomes of a mood disorders program. Implement Sci. 2014;9:132.
Almirall D, Compton SN, Gunlicks-Stoessel M, Duan N, Murphy SA. Designing a pilot sequential multiple assignment randomized trial for developing an adaptive treatment strategy. Stat Med. 2012;31:1887–902.
Smith MW, Barnett PG. The role of economics in the QUERI program: QUERI Series. Implement Sci. 2008;3:20.
Miles MB, Huberman AM. Qualitative Data Analysis: An Expanded Sourcebook. Thousand Oaks: Sage; 1994.
Hsieh HF, Shannon SE. 3 approaches to qualitative content analysis. Qual Health Res. 2005;15:1277–88.
Creswell JW, Klassen AC, Plano Clark VL, Clegg Smith K. Best Practices for Mixed Methods Research in the Health Sciences. In: Research NIH OBSS. Bethesda: National Institute of Health; 2011.
Conklin J. Dialogue Mapping: Building Shared Understanding of Wicked Problems. England, Wiley: West Sussex; 2005.
Plsek P. Redesigning health care with insights from the science of complex adaptive systems. In: Institute of Medicine, Crossing the Quality Chasm. Washington: National Academy Press; 2001. p. 309–22.
Damschroder LJ, Hagedorn HJ. A guiding framework and approach for implementation research in substance use disorders treatment Psychology of addictive behaviors. J Soc Psychol Addic Behav. 2011;25(2):194–205.
Foy R, Ovretveit J, Shekelle PG, Pronovost PJ, Taylor SL, Dy S, et al. The role of theory in research to develop and evaluate the implementation of patient safety practices. Qual Saf Health Care. 2011;20(5):453–9.
Davidoff F, Dixon-Woods M, Leviton L, Michie S. Demystifying theory and its use in improvement. BMJ Qual Saf. 2015;24(3):228–38.
Damschroder L, Goodrich D, Robinson C, Fletcher C, Lowery J. A systematic exploration of differences in contextual factors related to implementing the MOVE! weight management program in VA: A mixed methods study. BMC Health Serv Res. 2011;11(1):248.
Damschroder LJ, Lowery JC. Evaluation of a large-scale weight management program using the consolidated framework for implementation research (CFIR). Implement Sci. 2013;8:51.
Meyers DC, Durlak JA, Wandersman A. The quality implementation framework: a synthesis of critical steps in the implementation process. Am J Community Psychol. 2012;50(3–4):462–80.
Damschroder L, Aron D, Keith R, Kirsh S, Alexander J, Lowery J. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50.
Tabak RG, Khoong EC, Chambers DA, Brownson RC. Bridging research and practice: models for dissemination and implementation research. Am J Prev Med. 2012;43(3):337–50.
Ilott I, Gerrish K, Booth A, Field B. Testing the Consolidated Framework for Implementation Research on health care innovations from South Yorkshire. J Eval Clin Pract. 2012;19:915–24.
Kane H, Lewis MA, Williams PA, Kahwati LC. Using qualitative comparative analysis to understand and quantify translation and implementation. Translational Behav Med. 2014;4(2):201–8.
Chaudoir SR, Dugan AG, Barr CH. Measuring factors affecting implementation of health innovations: A systematic review of structural, organizational, provider, patient, and innovation level measures. Implement Sci. 2013;8(1):22.
Aarons GA, Ehrhart MG, Farahnak LR. The implementation leadership scale (ILS): development of a brief measure of unit level implementation leadership. Implement Sci. 2014;9(1):45.
Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;6:CD000259.
Leeman J, Baernholdt M, Sandelowski M. Developing a theory-based taxonomy of methods for implementing change in practice. J Adv Nurs. 2007;58(2):191–200.
Hussey MA, Hughes JP. Design and analysis of stepped wedge cluster randomized trials. Contemp Clin Trials. 2007;28:182–91.
Parchman ML, Noel PH, Culler SD, Lanham HJ, Leykum LK, Romero RL, et al. A randomized trial of practice facilitation to improve the delivery of chronic illness care in primary care. Impl Sci. 2013;8:93.
Verbeke G, Molenberghs G. Linear Mixed Models for Longitudinal Data. New York: Springer; 2009.
Wagner A, Soumerai S, Zhang F, Ross‐Degnan D. Segmented regression analysis of interrupted time series studies in medication use research. J Clin Pharm Ther. 2002;27(4):299–309.
Loudeon K, Treweek S, Sullivan F, Donnan P, Thorpe K, Zwarenstein M. The PRECIS-2 tool: designing trials that are fit for purpose. BMJ. 2015;350:h2147.
Andriole GL, Crawford ED, Grubb 3rd RL, Buys SS, Chia D, Church TR, et al. Mortality results from a randomized prostate-cancer screening trial. N Engl J Med. 2009;360(13):1310–9.
Kilbourne AM, Goodrich DE, Nord KM, VanPoppelen C, Kyle J, Bauer MS, Waxmonsky JA, Lai Z, Kim HM, Eisenberg D, Thomas MR. Long-term clinical outcomes from a randomized controlled trial of two implementation strategies to promote collaborative care attendance in community practices. Admin Policy Mental Health 2014. epub ahead of print.
Baker R, Camosso-Stefinovic J, Gillies C, Hearnshaw H, Flottorp S, Robertson N, et al. Tailored interventions to overcome identified barriers to change: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2010;3:CD005470.
Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci. 2013;8:117.
Curran G, Bauer MS, Stetler CB, Mittman BS. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012;50:217–26.
US Department of Veterans Affairs. http://www.queri.research.va.gov/about/strategic_plans/QUERIStrategicPlan.pdf. Last accessed July 25, 2015.
Hagedorn HJ, Noorbaloochi S, Baines Simon A, Bangerter A, Stitzer ML, Stetler CB, et al. Rewarding early abstinence in Veterans Health Administration addiction clinics. J Subst Abuse Treat. 2013;45:109–17.
Hagedorn HJ, Stetler CB, Bangerter A, Noorbaloochi S, Stitzer ML, Kivlahan D. An implementation-focused process evaluation of an incentive intervention effectiveness trial in substance use disorders clinics at two Veterans Health Administration medical centers. Addict Sci Clin Pract. 2014;9:12.
Gaglio B, Glasgow RE. Evaluation approaches for dissemination and implementation research. In: Brownson R, Colditz G, Proctor E, editors. Dissemination and Implementation Research in Health: Translating Science into Practice. New York: Oxford University Press; 2012. p. 327–56.
Kitson A, Rycroft-Malone J, Harvey G, McCormack B, Seers K, Titchen A. Evaluating the successful implementation of evidence into practice using the PARIHS framework: Theoretical and practical challenges. Implement Sci. 2008;3:1.
Harvey G, Kitson A. Implementing Evidence-Based Practice in Healthcare. Routledge: A Facilitation Guide; 2015.
Azjen I. The theory of planned behavior. Organ Behav Hum Decis Process. 1991;50:179–211.
US Institute of Medicine. The Learning Helathcare System: Workshop Summary. Washington: National Academy Press; 2007.
US Institute of Medicine Committee on the Learning Health Care System in America. Best Care at Lower Cost. Washington: National Academy Press; 2013.
Fihn SD, Francis J, Clancy C, Nielson C, Nelson K, Rumsfeld J, et al. Insights from advanced analytics at the Veterans Health Administration. Health Aff (Millwood). 2014;33:1203–11.
This work was supported in part by a variety of grants from the USVA Health Services Research and Development Program and USVA QUERI. The opinions of the authors are theirs solely and do not necessarily reflect the opinions of USVA.
The authors wish to express appreciation to Drs. Karen Quigley, A. John Rush, and Dawne Vogt who read and commented on earlier versions of this manuscript, and to the executive editors of BMC Psychology who invited this submission after the workshop which presented this content at the Annual Meeting of the American Psychological Association on August 10, 2014, in Washington, DC.
The authors’ research is supported by the US Department of Veterans Affairs Quality Enhancement Research Initiative (USVA QUERI), and have no competing financial conflicts of interests.
Each listed author was involved in drafting the manuscript and revising it critically for intellectual content and style. Each has given final approval of the version to be published.