Advertisement

Contextual Factors Affecting Implementation of Pediatric Quality Improvement Programs

      Abstract

      Objective

      Context is a critical determinant of the effectiveness of quality improvement programs. We assessed the role of contextual factors in influencing the efforts of 5 diverse quality improvement projects as part of the Pediatric Quality Measure Program (PQMP) directed by the Agency for Health Care Research and Quality.

      Methods

      We conducted a mixed methods study of 5 PQMP grantees involving semistructured interviews followed by structured worksheets to identify influential contextual factors. Semistructured interviews and worksheets were completed between August and October 2020. Participants were comprised of PQMP grantee teams (2–4 team members per team for a total of 15 participants). Coding and analysis was based on the Tailored Implementation for Chronic Diseases (TICD) framework.

      Results

      Despite heterogeneity in the process and outcome targets of the PQMP initiatives, professional interactions, incentives and resources, and capacity for organizational change were the domains most commonly identified as influential across the grantees. While social, political, and legal factors was not commonly referenced as an important domain, payer or funder policies (a factor within this domain) was highlighted as one of the most influential factors. Overall, the incentives and resources domain was identified as the most influential.

      Conclusions

      We found that using a determinant framework, such as the TICD, is valuable in facilitating comparisons across heterogeneous projects, allowing us to identify key contextual factors influencing the implementation of pediatric quality measures across a diverse range of clinical topics and settings. Future quality improvement work should account for this and include resources to support infrastructure development in addition to program implementation.

      Keywords

      What's New
      A determinants framework was used to identify barriers and facilitators for 5 pediatric quality improvement projects. Incentives and resources were determined to be most influential, suggesting a need for greater investment to improve the efficiency of future quality improvement efforts.
      Context is a critical determinant of the effectiveness of any quality improvement (QI) program.
      • Ovretveit J
      Understanding the conditions for improvement: research to discover which context influences affect improvement success.
      ,
      • Kaplan HC
      • Brady PW
      • Dritz MC
      • et al.
      The influence of context on quality improvement success in health care: a systematic review of the literature.
      By systematically assessing the role of contextual factors in a program's implementation, we can gain a better understanding of how the intervention worked, with lessons for others conducting similar efforts.
      • McDonald KM
      Considering context in quality improvement interventions and implementation: concepts, frameworks, and application.
      Historically, implementation science frameworks have been used to provide a conceptual underpinning to analyze the influence of contextual factors for a range of interventions, including pediatric QI, primary care practice transformation, prevention and wellness projects, among others.
      • McDonald KM
      Considering context in quality improvement interventions and implementation: concepts, frameworks, and application.
      • Stover AM
      • Haverman L
      • van Oers HA
      • et al.
      Using an implementation science approach to implement and evaluate patient-reported outcome measures (PROM) initiatives in routine care settings.
      • Tomoaia-Cotisel A
      • Scammon DL
      • Waitzman NJ
      • et al.
      Context matters: the experience of 14 research teams in systematically reporting contextual factors important for practice change.
      • Lee RM
      • Ramanadhan S
      • Kruse GR
      • et al.
      A mixed methods approach to evaluate partnerships and implementation of the Massachusetts Prevention and Wellness Trust Fund.
      As part of the first phase of the Pediatric Quality Measure Program (PQMP) directed by the Agency for Health Care Research and Quality (AHRQ), 7 Centers of Excellence developed new and enhanced pediatric quality measures covering a range of clinical conditions. These quality measures were developed to increase the portfolio of measures available to public and private purchasers of children's health care services under the Children's Health Insurance Program Reauthorization Act of 2009, Title IV, Section 401. In the second phase of the PQMP, 6 grantees assessed the feasibility and usability of the newly developed measures for QI. Each of the grantees launched and led one or more quality improvement collaboratives (QICs) to support QI activities targeting their respective measures.
      QICs are a commonly used QI or implementation strategy that brings together multiple health care organizations to focus on the same goal and includes opportunities to share learnings and best practices in support of improving performance.
      • Zamboni K
      • Baker U
      • Tyagi M
      • et al.
      How and under what circumstances do quality improvement collaboratives lead to better outcomes? A systematic review.
      Given that the focus of each grantee's QIC was tied to one of the quality measures developed in the first PQMP phase, the QI work supported by the QICs reflected a variety of clinical conditions, measure types (process, outcome, experience), and data sources (claims, electronic health record [EHR], survey). Depending on the measure, the QICs engaged with individuals and organizations at different, and sometimes multiple, levels of the health care system—spanning patient, provider, health system, health plan, and state levels. Despite differences in the clinical focus and the level of the health care system at which each QIC was operating, all of the QICs aimed to improve systems, processes, and/or long-term outcomes of pediatric care among QIC participants.
      The purpose of this analysis was to use an implementation science determinants framework to identify which, if any, contextual factors were common barriers or facilitators to improving pediatric care across a group of diverse QICs conducted as part of phase II of the AHRQ PQMP program.
      • Flottorp SA
      • Oxman AD
      • Krause J
      • et al.
      A checklist for identifying determinants of practice: a systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice.

      Methods

       Overview

      We conducted a mixed methods study to identify the factors that facilitated or impeded the successful implementation of system and process changes within the PQMP QICs using the Tailored Implementation for Chronic Diseases (TICD) framework.
      • Flottorp SA
      • Oxman AD
      • Krause J
      • et al.
      A checklist for identifying determinants of practice: a systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice.
      The TICD is a comprehensive, integrated checklist of determinants of implementation success organized into 7 domains that include factors that might impact implementation. The 7 domains include: guideline or quality measure factors, individual health professional factors, patient factors, professional interactions, incentives and resources, capacity for organizational change, and social/political/legal factors. Consolidating other determinants frameworks, the TICD offers specificity and flexibility as it can be applied across different levels of organizations or systems, which was a key component of this analysis necessitated by how the QICs were organized.
      We carried out data collection in 2 phases: first, we conducted semistructured interviews to obtain information about the QICs and respective system and process changes implemented by the QIC participants; second, we collected information to capture details about contextual factors influencing implementation of the system and process changes using structured worksheets based on the TICD domains. The first phase of data collection was used to inform the second phase. This paper focuses on the findings from the second phase of data collection.

       Selection of QIC Projects

      While some grantees executed more than one QIC to support QI targeted to multiple quality measures, for the purpose of this evaluation, we focused on activities related to one QIC and one measure from 5 of the grantees. We excluded one grantee because neither of their QICs advanced to a phase of active QI during our evaluation period. In consultation with the grantees and AHRQ, we identified specific system and process changes implemented by QIC participants as part of the QI activities within each QIC. Selection was driven by measures/system changes that progressed the most during the period of performance. Characteristics of the included measures and QICs are highlighted in Table 1. We focused our inquiry on key contextual barriers and facilitators of implementation of these specific system and process changes.
      Table 1QICs Featured in the IS Analysis
      PQMP GranteeQIC GoalsQIC ParticipantsQIC DurationMeasure Targeted for ImprovementIntervention Selection ProcessMain System and Process Changes
      CEPQMDevelop and implement a shared intervention across sites to garner shared learnings for reducing hospital readmissionsHospitals (n = 5)36 monthsPediatric 30-day all-condition hospital readmissions (NQF # 2393)All QIC participants agreed to implement the same intervention through a consensus-based decision-making processParental confidence assessment: Hospitals developed and implemented a parental confidence assessment—focused on assessing parents’ confidence in discharge instructions—that could be administered to parents before their child was discharged from the hospital
      IMPLEMENT- Successfully establish and continue engagement in the collaborative

      - Improve asthma outcomes, specifically the number of child emergency department visits for asthma
      Primary care practices (n = 11)12 months, plus 9 months extensionRate of emergency department visit use for children managed for persistent asthmaGrantee presented QIC participants with a menu of intermediate measures; QIC participants selected from these measures and then developed their own interventionsElectronic health record (EHR) improvements for child asthma-related process measures: Participating practices focused mostly on improvements related to implementing decision support features and improving the sites’ documentation capabilities to help capture and track the process measures. Strategies included: reconfigured visit report pull from EHR, smart forms, dot/smart phrases, asthma patient identification
      NCINQ IIImprove metabolic monitoring in children and adolescents prescribed an antipsychoticHealth plans (n = 5)18 monthsMetabolic monitoring for children and adolescents on antipsychotics (NQF # 2800)Grantee presented QIC participants with a menu of intervention options that could be tailored to participant specificationsHealth plan-level metabolic monitoring improvement strategies: All the plans focused on implementing safe and judicious practices for antipsychotic prescribing among youth—they selected the strategy that was best suited to their context. Strategies included: member outreach, standing lab order, data exchange, provider outreach, incentive program
      P-HIPImprove participating hospitals’ performance on counseling parents/caregivers regarding restricting access to potentially lethal means of suicide from their child/ adolescent treated for suicidality prior to discharge home and documentation in the medical record that the counseling was completedHospitals (n = 8)12 monthsChildren/adolescents admitted to the hospital for dangerous self-harm or suicidality, should have documentation in the hospital record that their caregiver was counseled on how to restrict the child's/adolescent's access to potentially lethal means of suicide (eg, firearms, medications, car, etc.) prior to dischargeGrantee presented QIC participants with a menu of intervention options that could be tailored to participant specificationsImplementing a standardized documentation strategy for counseling through their EHR: Teams introduced a statement in the discharge template about counseling on the restriction of lethal means to serve as a reminder/prompt, and to ensure that documentation was completed
      Q-METRIC- Improve care for children with sickle cell disease and specifically improve rates of appropriate antibiotic prophylaxis for children with sickle cell anemia

      - Successfully identify target patient population

      - Continued engagement in the collaborative
      State (n = 1)

      Health plans (n = 3)

      Clinics (n = 2)
      12 monthsAppropriate antibiotic prophylaxis for children with sickle cell anemia (NQF # 3166)Grantee presented QIC participants with a menu of intervention options that could be tailored to participant specificationsDeveloping patient registries/databases for children with sickle cell anemia: Sites developed approaches to identifying children with sickle cell anemia in their clinics or plans
      QIC indicates quality improvement collaborative; IS, implementation science; and PQMP, Pediatric Quality Measure Program.

       Semistructured Interviews

      In August 2020, we conducted semistructured telephone interviews (supplemental file available upon request) with each of the grantees to elicit details about the nature of the QIC they convened. Grantees self-selected team members (from 2 to 4 members) to participate in the interviews. All teams included the principal investigator. Each interview was led by an experienced researcher (E.C. or M.J.). Interviews were recorded and transcribed. Interview notes and transcripts were independently reviewed by 2 research team members (M.J. and M.K.), with each extracting descriptive information about the QICs and system and process changes implemented. Results were compared and discrepancies resolved through consensus. Extracted information was used to prepopulate the first section of the worksheet for grantee review and confirmation and to provide background for the research team to better understand how each QIC functioned.

       Worksheets

      Between September and October 2020, each grantee was asked to complete a structured worksheet to identify relevant contextual factors that impacted QIC participants’ implementation of system/process changes (supplemental file available upon request). Worksheet design was informed by the approach of Tomoaia-Cotisel et al,
      • Tomoaia-Cotisel A
      • Scammon DL
      • Waitzman NJ
      • et al.
      Context matters: the experience of 14 research teams in systematically reporting contextual factors important for practice change.
      in which authors used structured worksheets to collect information on contextual factors from 14 distinct research teams. Wording leveraged supplementary materials published by Flottorp et al,
      • Flottorp SA
      • Oxman AD
      • Krause J
      • et al.
      A checklist for identifying determinants of practice: a systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice.
      the developers of the TICD checklist. Worksheets were tailored to each grantee by incorporating descriptive details of their QICs from the interviews. Grantees were asked to elaborate on how relevant factors in each TICD domain applied in their QIC context. Separately, grantees were asked to select the contextual factors they deemed “most influential” overall in the ability of QIC participants to successfully make system and process changes (allowing for several). Grantees completed the worksheets using free text. Typically, worksheets were completed by the same grantee team members as those who participated in the semistructured interviews.
      All 5 worksheets were collected from grantees and independently coded by 2 research team members (M.J. and M.K.). The research team performed deductive content analysis of free text data from the worksheets using the TICD as a template. Specifically, contextual factors were coded using TICD domains, subdomains, and individual factors as a priori codes, noting whether factors were barriers and/or facilitators. Coding results were reviewed and compared. Any coding discrepancies were discussed and resolved through consensus (E.C., M.J., and M.K.). Factors that were indicated by grantees as “most influential,” that is, those that had the most impact on implementation efforts, among their QIC participants were noted. Domains with the highest proportion of factors identified as relevant or “most influential” across grantee QIC participants were considered most salient and are referred to as such in the results. The team identified quotations from the worksheet responses to illustrate key themes or observations, particularly among the most salient domains.
      This work was a component of a learning collaborative effort associated with the grantees’ cooperative agreement awards, interviewees were only grantees and asked only about topics associated with the specific QICs established under their cooperative agreement awards. Interviewees were fully informed of how the information would be used and no sensitive information was collected.

      Results

      We interviewed a total of 15 participants (2–4 per grantee team). All grantee teams included the principal investigator. Additional participants included one or more of the following depending on the grantee team: project manager, implementation lead, QI lead, and/or lead learning collaborative facilitator. In some cases, the same person served multiple roles. These interviewees were also responsible for completing the structured worksheets.
      There were contextual factors within each of the TICD domains that were identified as important barriers and facilitators of successful implementation of system and process changes across the 5 QICs as shown in the data collected from the grantees’ worksheets (Table 2).
      Table 2Factors Identified by Grantees
      FactorDefinition
      Definitions were adapted using the supplementary Tailored Implementation for Chronic Diseases materials published by Flottorp et al8.
      CEPQMIMPLEMENTNCINQ IIP-HIPQ-METRIC
      1. Guideline factors
       A. Quality measure
        Quality of evidence supporting the measureHow confident we are in the evidence base, reliability, and validity of the quality measure
      Indicates factor was identified by grantee as being among the “most influential” factors for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being among the “most influential” factors for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        Strength of the quality measureHow confident we are that the desirable effects of improving the quality measure outweigh the undesirable effects
      Indicates factor was identified by grantee as being among the “most influential” factors for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        ClarityThe clearness of the target population, the settings in which the quality measure is to be used, and the recommended change strategy
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        Cultural appropriatenessThe extent to which the quality measure is suitable in the social context where it is being implemented
        Accessibility of the quality measureHow accessible the specifications are
        Source of the quality measureThe organization(s) and people that developed the quality measure
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        Consistency with other practicesThe extent to which the quality measure and associated change strategy are consistent with other guidelines with which the targeted health care professionals might be familiar
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
       B. Recommended intervention
        FeasibilityThe extent to which implementing the quality measure and recommended change strategy is practical
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        Accessibility of the interventionThe extent to which the quality measure and recommended change strategy is accessible
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
       C. Recommended behavior
        CompatibilityThe extent to which the recommended behaviors associated with the change strategy fits with current practices
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        EffortThe amount of effort required to change or adhere
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        TrialabilityThe ability to try out the recommended behaviors associated with the change strategy
        ObservabilityThe degree to which benefits of the change strategy are visible
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      2. Individual health professional factors
       A. Knowledge and skills
        Domain knowledgeThe extent to which the targeted health care professionals have pre-existing knowledge or expertise about the targeted condition/practice
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        Awareness and familiarity with the quality measureThe extent to which the targeted health care professionals are aware of and familiar with the change strategy
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        Knowledge about own practiceThe extent to which the targeted health care professionals are aware of their own practice in relationship to the change strategy
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        Skills needed to adhereThe extent to which the targeted health professionals have skills that they need to adhere
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
       B. Cognitions
        Agreement with the quality measureThe extent to which the targeted health care professionals agree with the quality measure and associated change strategies
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        Attitudes toward interventions in generalThe perceptions that the targeted health care professionals have regarding improvement change strategies in general
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        Expected outcomeThe extent to which the targeted health care professionals believe that adherence with the change strategy will lead to desired outcomes
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        Intention and motivationThe extent to which the targeted health care professionals intend to adhere and are motivated to do so
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being among the “most influential” factors for one or more of their quality improvement collaborative participants.
        Self-efficacyThe targeted health care professionals’ self-perceived competence or confidence in their abilities
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        Learning styleThe preferred ways in which the targeted health care professionals learn
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        EmotionsThe extent to which emotions affect adherence
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
       C. Professional behavior
        Nature of the behaviorCharacteristics of the behavior, including: frequency of performance for a patient, frequency of performance for a population of patients, the degree of habit or automaticity, whether it is within a sequence of other behaviors that have to be performed, and whether it is performed by one person or by different people
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being among the “most influential” factors for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        Capacity to plan changeThe extent to which the targeted health care professionals have the capacity to plan necessary changes in order to adhere
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        Self-monitoring or feedbackThe extent to which the targeted health care professionals have the capacity for self-monitoring or feedback to reinforce adherence with the change strategy
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      3. Patient and family factors
        Patient and family needsReal or perceived needs and demands of the patient/family
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        Patient and family beliefs and knowledgePatients’/families’ beliefs or knowledge or ability to learn, or the targeted health care professionals’ ability or perceived ability to inform or teach patients/families necessary knowledge and skills
      Indicates factor was identified by grantee as being among the “most influential” factors for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        Patient and family preferencesPatients’/families’ values in relationship to professional values or those in the change strategy
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        Patient and family motivationThe targeted health care professionals’ ability or perceived ability to motivate patients/families to adhere
      Indicates factor was identified by grantee as being among the “most influential” factors for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being among the “most influential” factors for one or more of their quality improvement collaborative participants.
        Patient and family behaviorPatient/family behaviors that motivate or demotivate adherence with the change strategy
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      4. Professional interactions
      Indicates domain was identified as most salient during the analysis. Domains with the highest proportion of factors identified as relevant or “most influential” across grantees were considered most salient.
        Communication and influenceThe extent to which the targeted health care professionals’ adherence is influenced by professional opinions and communication
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        Team processesThe extent to which professional teams or groups have the skills needed to adhere and interact in ways that facilitate or hinder adherence
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being among the “most influential” factors for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        Referral processesProcesses for transferring patients and communication between different levels of care, between health and social services, and between the targeted health care professionals and targeted patients
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being among the “most influential” factors for one or more of their quality improvement collaborative participants.
      5. Incentives and resources
      Indicates domain was identified as most salient during the analysis. Domains with the highest proportion of factors identified as relevant or “most influential” across grantees were considered most salient.
        Availability of necessary resourcesThe extent to which the resources that are needed to adhere are available
      Indicates factor was identified by grantee as being among the “most influential” factors for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being among the “most influential” factors for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being among the “most influential” factors for one or more of their quality improvement collaborative participants.
        Financial incentives and disincentivesThe extent to which patients/families, individual health professionals and organizations have financial incentives or disincentives to adhere
      Indicates factor was identified by grantee as being among the “most influential” factors for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being among the “most influential” factors for one or more of their quality improvement collaborative participants.
        Nonfinancial incentives and disincentivesThe extent to which patients/families, individual health professionals and organizations have nonfinancial incentives or disincentives to adhere
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being among the “most influential” factors for one or more of their quality improvement collaborative participants.
        Information systemThe extent to which the information system facilitates or hinders adherence
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being among the “most influential” factors for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being among the “most influential” factors for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        Quality assurance and patient safety systemsThe extent to which existing quality assurance or patient safety systems facilitate or hinder adherence
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        Continuing education systemThe extent to which the continuing education system facilitates or hinders adherence
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        Assistance for cliniciansThe extent to which clinicians have the assistance they need to adhere
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being among the “most influential” factors for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      6. Capacity for organizational change
      Indicates domain was identified as most salient during the analysis. Domains with the highest proportion of factors identified as relevant or “most influential” across grantees were considered most salient.
        Mandate, authority, accountabilityThe mandate, authority, and mechanisms of accountability for making necessary changes
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being among the “most influential” factors for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        Capable leadershipThe extent to which clinical leaders or managers are capable of making necessary changes
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being among the “most influential” factors for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        Relative strength of supporters and opponentsThe extent of support and/or opposition to necessary changes
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        Regulations, rules, policiesThe extent to which organizational regulations, rules, or policies facilitate or hinder necessary changes
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        Priority of necessary changeThe relative priority given to making necessary changes
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being among the “most influential” factors for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        Monitoring and feedbackThe extent to which monitoring and feedback are needed at the organizational level and available to sustain necessary changes (including evaluations of improvement programs)
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        Assistance for organizational changesThe extent to which external support is needed and available for necessary changes
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      7. Social, political, and legal factors
        Economic constraints on the health care budgetLimits on the total health care budget or its growth
        ContractsThe extent to which contracts may affect implementation of necessary changes
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        LegislationThe extent to which legislation may affect implementation of necessary changes
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        Payer or funder policiesThe extent to which payer or funder policies may affect implementation of necessary changes
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being among the “most influential” factors for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being among the “most influential” factors for one or more of their quality improvement collaborative participants.
        Malpractice liabilityThe extent to which malpractice liability may affect implementation of necessary changes
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        Influential peopleThe extent to which influential people may affect implementation of necessary changes
      Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
        CorruptionThe extent to which corruption may affect implementation of necessary changes
        Political stabilityThe extent to which political stability may affect implementation of necessary changes
      low asterisk Indicates factor was identified by grantee as being relevant as a barrier or facilitator for one or more of their quality improvement collaborative participants.
      Indicates factor was identified by grantee as being among the “most influential” factors for one or more of their quality improvement collaborative participants.
      Indicates domain was identified as most salient during the analysis. Domains with the highest proportion of factors identified as relevant or “most influential” across grantees were considered most salient.
      § Definitions were adapted using the supplementary Tailored Implementation for Chronic Diseases materials published by Flottorp et al
      • Flottorp SA
      • Oxman AD
      • Krause J
      • et al.
      A checklist for identifying determinants of practice: a systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice.
      .

       Most Salient Domains

      The “incentives and resources,” “professional interactions,” and “capacity for organizational change” domains were identified as most salient across the QICs. Importantly, contextual factors within these domains were identified as important despite the heterogeneity in the level of the health care system where changes were made, the types of system and process changes implemented, and the complexity of the clinical areas of focus.

       Incentives and Resources

      Contextual factors within this domain were identified as important barriers and facilitators of successful implementation of system and process changes universally by grantees in their worksheet responses. Not only was this domain commonly discussed by grantees in free text responses, but several factors under this domain were also designated as “most influential.” The 2 factors most commonly identified as having an impact on QIC implementation efforts were: the availability of necessary resources and information systems. Availability of necessary resources was reported as both a barrier and facilitator. For example, in the CEPQM QIC, as related to their strategy for developing and implementing a parental confidence discharge assessment, the grantee commented that “ultimately, resources—staff, QI expertise, and IRB expertise—ended up being critically important for getting this intervention off the ground. Even if there is leadership buy-in, without the additional resources, hospitals were unable to participate.” Another grantee, Q-METRIC, reported that, within their QIC, without adequate resources—financial and staffing—it was challenging to make short-term progress and even more difficult to sustain long-term improvements in appropriate antibiotic prophylaxis for children with sickle cell anemia. This grantee noted that a lack of extrinsic incentives exacerbated the issue of resource availability, commenting “Without incentives or an extrinsic requirement to report, it is unlikely that adequate organizational resourcing and support will be made available to improve performance rates.” The grantee supporting the NCINQ QIC had a similar observation, noting, “One of the most impactful factors in implementing the measures and strategies [for improving metabolic monitoring] was the existence of state mandate and existing infrastructure/mechanism for reporting … this created an incentive for providers and health plans to focus resources on this small, but highly vulnerable population….” Information systems were also noted as a key factor. In nearly all cases, information systems were facilitators to implementation efforts, particularly in cases where the information systems were able to be provided centrally by the QIC. For example, both the CEPQM and NCINQ QICs used their existing information systems to run performance reports to monitor their QIC participants’ progress. Others, like the P-HIP and IMPLEMENT QICs leveraged and built out their information systems as part of the QIC activities to support the sustainability of improvement efforts.

       Professional Interactions

      Within this domain, the most commonly cited factor was team processes, which is the extent to which professional teams have the skills needed to facilitate or impede implementation. Although the clinical focus of each QIC varied (Table 1), all of the QICs were focused on making system and process changes that involved members of different departments and/or disciplines. Within the P-HIP QIC which focused on caring for children and adolescents admitted to the hospital for dangerous self-harm or suicidality, the grantee noted: “Setting expectations and clearly defining roles for conducting and documenting the counseling was essential given the many staff and departments involved…. Teams accomplished this in a variety of ways: having a dedicated role staffed 24/7 that was responsible for the counseling; …required training program for new residents who were responsible for educating parents/caregivers; and creating accountability for doing both the counseling and the documentation.” Another grantee, highlighted an innovative approach that one health plan participating in the NCINQ QIC undertook to ensure a team-based approach for metabolic monitoring for children and adolescents on antipsychotics: “having a pharmacist on their QI team, which the other health plans had not done, but learned was a valuable approach through the QIC and explored incorporating it in their own QI work.” The pharmacist had developed a corporate-wide strategy to encourage safer monitoring that involved, “standing lab orders … and pharmacy soft edits (to encourage discussion of alternate treatment options).”

       Capacity for Organizational Change

      Contextual factors within this domain were the most commonly identified barriers and facilitators of successful implementation of system and process changes, specifically capable leadership; priorities of necessary change; mandate, authority, accountability; and monitoring and feedback. Referencing the importance of capable leadership, grantees identified this factor as both a facilitator and a barrier. Grantees stated that having engaged and “bought-in” leaders improved resource allocation and focus on the QIC efforts. This was noted specifically by the grantee team supporting the NCINQ QIC. On the other hand, grantees reported that those participants with less involved leaders suffered from fewer resources, and at one of the participating sites, the project risked being de-prioritized (IMPLEMENT). Grantees also noted that priorities of necessary change, impacted their QICs’ success. The IMPLEMENT QIC, which focused on EHR improvements for child asthma-related processes, reported that this factor was both a facilitator and a barrier. They shared that participants’ “ability to modify or customize an EHR requires time and resources… Unless the particular measure is a priority for an organization, changes to the EHR may require a practice to ‘wait their turn in line’.” Similarly, other grantees reported that QIC participants struggled with competing priorities drawing resources and support away from their QI work as part of the QIC. On the other hand, within the P-HIP QIC, priorities of necessary change was identified as a facilitator. The grantee speculated that the pressure the QIC participants felt to reduce rates of suicide among children led to this QI work being prioritized by participating organizations.

       Other Important Domains

      While the other domains of the TICD were less consistently reported as relevant among grantees, there were individual contextual factors within these domains that appeared to be critical barriers and facilitators of successful implementation of system and process changes in both the qualitative and quantitative (Table 2) data provided by grantees, though findings were more heterogeneous. The importance of these factors appeared to be dependent on the specific context of the QIC, and there were few cross-QIC patterns based broadly on clinical topic (eg, behavioral health) or QIC participant composition (eg, health system level).
      Social, political, and legal factors were the least referenced domain; however, 2 grantees reported that payer or funder policies was among their “most influential” factors impacting their QICs. The fact that the IMPLEMENT QIC was supported as part of the AHRQ PQMP program made the collaborative stand-out among other QI project options, facilitating recruitment of primary care practice participants: “…this group came together because of the unique opportunity to test the usability of a new pediatric quality measure.” Conversely, the Q-METRIC QIC attributed some of their limited measure uptake to a lack of funder policies, “it is very difficult to convince systems or plans to invest resources when other required priorities exist.”
      Within the domain of Guideline Factors, the most commonly cited factors were quality of evidence supporting the measure and source of the quality measure. While the domain overall was not identified as one of the most influential among grantees, the specific factor of quality of evidence supporting the measure was identified among the “most influential” factors impacting implementing success. This is not surprising given that a core criterion of a PQMP measure is having a robust evidence base.
      Among the most commonly cited factors related to Individual Health Professional Factors were domain knowledge, awareness and familiarity with the quality measure, and intention and motivation. Nearly all grantees noted that dedicated and motivated health professionals who were already familiar with the quality measure were well-positioned to facilitate implementation within their QIC participating sites. Nature of the behavior and capacity to plan change were also highlighted as key factors, with one grantee in particular noting that their QIC's strategy to “hardwire changes” into their team processes served as a facilitator for easier adoption.
      Finally, the most commonly cited factor within the domain of patient factors was patient and family beliefs and knowledge. This factor was reported to be important particularly among QICs where system and process changes required engagement directly with patients and families, and principally when timing was an important part of the process. For example, participants in the CEPQM QIC administered their parental discharge readiness assessment near the time of discharge, when parents may have been reluctant to acknowledge their lack of confidence in the discharge instructions to expedite taking their child home. In another case, the grantee found that the counseling strategy implemented by participants in the P-HIP QIC worked best within the first 3 days of admission, allowing parents time to adjust to the circumstances.

      Discussion

      Our study sought to identify common influential contextual factors across a set of 5, highly heterogeneous pediatric QI initiatives. These QI initiatives differed in terms of type of participants, clinical topics, measures of focus, and specific system and process changes being implemented. While each initiative supported participants’ QI efforts using a QIC model, the differing approaches used by these QIC's to support improvement varied and added to the heterogeneity. Within QICs, participants were generally given the freedom to identify system and process changes that would be best suited for their own setting—meaning specific system and process changes varied within as well as across QICs. In addition, the timeframe allowed for QIC activity varied across these initiatives with some QIC's occurring over 12 months and others over 36 months. While this amount of heterogeneity generates substantial complexity in the evaluation, it allowed us to identify a number of key contextual factors that are common barriers and facilitators of pediatric quality initiatives across the board, regardless of QI approach or clinical focus.
      The key contextual factors across domains that seemed to transcend the heterogeneity across QICs and were consistently highlighted as important barriers and facilitators included team processes, availability of necessary resources, information systems, capable leadership and priorities of necessary change. The need for organizational systems and infrastructure to support improvement was consistently identified in the QI initiatives, a finding that is aligned with the broader literature demonstrating the critical role of organizational, systemic, and cultural factors affecting implementation.
      • Tomoaia-Cotisel A
      • Scammon DL
      • Waitzman NJ
      • et al.
      Context matters: the experience of 14 research teams in systematically reporting contextual factors important for practice change.
      ,
      • Shea CM
      • Turner K
      • Albritton J
      • et al.
      Contextual factors that influence quality improvement implementation in primary care: the role of organizations, teams, and individuals.
      Additionally, these findings reinforce the central need for available resources to enable successful implementation, consistent with other studies.
      • Higgins A
      • Murphy R
      • Downes C
      • et al.
      Factors impacting the implementation of a psychoeducation intervention within the mental health system: a multisite study using the consolidation framework for implementation research.
      • Sarkies M
      • Long JC
      • Pomare C
      • et al.
      Avoiding unnecessary hospitalisation for patients with chronic conditions: a systematic review of implementation determinants for hospital avoidance programmes.
      • Weiss CH
      Why do we fail to deliver evidence-based practice in critical care medicine?.
      • Breland JY
      • Asch SM
      • Slightam C
      • et al.
      Key ingredients for implementing intensive outpatient programs within patient-centered medical homes: a literature review and qualitative analysis.
      Not surprisingly, because QI, at its core, is data-driven, having a data infrastructure to monitor changes in processes and outcomes of care related to the target quality metric was identified as important in all QICs. By creating a centralized and automated structure, information systems provide a powerful tool for organizations to support staff in carrying out their QI initiatives. As noted by Karltun et al, a consistent information and knowledge management infrastructure is crucial for clinical QI as it creates reinforcing enabling conditions, even across diverse health care settings.
      • Karltun A
      • Sanne JM
      • Aase K
      • et al.
      Knowledge management infrastructure to support quality improvement: a qualitative study of maternity services in four European hospitals [published correction appears in Health Policy. 2020 May;124:575].
      Given its relevance across all QICs, information systems clearly represent a key factor for similar QI initiatives to consider—and leverage—as a way to systematically support implementation.
      Observations from our evaluation also provide insights not only into the important role that extrinsic incentives, such as state mandates or required reporting of quality measures, play in procuring resources, but also into how external sponsors (like QICs or state agencies) can provide infrastructure for data collection and reporting of pediatric quality measures. In addition, reflections on the role of payer or funder policies by some grantees underscore the role that entities within the sociopolitical environment play in either facilitating or hindering successful improvement. Particularly important for pediatric QI is the observation that measures in the CMS Child Core Set or those that are aligned with QI focused incentive programs received greater attention by QIC participant leadership.
      Ensuring the local context is set up to support QI efforts targeting pediatric quality measures is critical and, ideally, should be addressed before beginning the QI initiative. Based on our analysis, it is clear that resources—human, infrastructure, financial—are needed to support effective pediatric focused QI activities. However, securing, and having dedicated resources can be challenging. There is an opportunity for Federal and state level governments to play a more substantial role in providing resources, whether through the creation of grants to support more pediatric QI work (eg, PQMP-like opportunities), by advancing value-based payment opportunities through Medicaid and CHIP that incentivize engagement in pediatric QI work, or through supporting registries that link varied data sources and can be used to track key quality measures. Lastly, the decision to adequately resource a QI effort is directly linked to whether that initiative was prioritized by the organization, and, in many organizations, pediatric quality is not identified as a top priority. Prioritization of pediatric quality by payers and health systems is an important first step in securing the resources needed for effective pediatric QI work.
      There were some notable limitations to this analysis. First, we collected data from the individuals who managed and led the QICs, so our findings may not represent the full range of experiences from the QIC participants. The retrospective nature of the project may also have led to underreporting of some factors. Also, due to the variation both in how the QICs defined success and in the maturity of the projects, we were not able to adequately assess success as measured by improvement in the pediatric quality measure across the projects, so we focused on those factors that influenced the implementation of system and process changes. Finally, the small sample size and high heterogeneity precludes inferential analysis, such as assessing associations between influential contextual factors and QIC characteristics.
      Despite the difficulties inherent in the retrospective data collection and analysis used in this study, it is important to make sense out of complex implementation projects to facilitate future implementation efforts across disparate sites. This is particularly important in pediatrics. Compared to the adult care setting, pediatrics is characterized by less funding and fewer resources, for a number of reasons including its comparatively smaller and less burdened population, as well as the reality that a higher proportion of children come from low-income families and communities, compared with adults.
      • Koball H
      • Moore A
      • Hernandez J
      Basic Facts About Low-Income Children: Children Under 9 Years, 2019.
      As we found, these resource limitations can have real impact on the ability to carry out effective QI. While QI in pediatrics has been an active area for some time,
      • Schwartz S
      • Rehder K
      Quality improvement in pediatrics: past, present, and future.
      this study is the first of its kind to explore commonly salient contextual factors that affect QI functioning across heterogeneous health system actors, clinical foci, measures, and change strategies. Based on our findings, as well as those in other studies, ensuring adequacy of resources at both an individual and collective (QIC) level will be essential to supporting effective activities in the future.
      • Zamboni K
      • Baker U
      • Tyagi M
      • et al.
      How and under what circumstances do quality improvement collaboratives lead to better outcomes? A systematic review.
      • Shea CM
      • Turner K
      • Albritton J
      • et al.
      Contextual factors that influence quality improvement implementation in primary care: the role of organizations, teams, and individuals.
      • Coles E
      • Anderson J
      • Maxwell M
      • et al.
      The influence of contextual factors on healthcare quality improvement initiatives: a realist review.

      Conclusions

      We found that using a determinant framework is valuable in facilitating comparisons and learning across heterogeneous QI projects. Through the use of semistructured interviews and worksheets based on the TICD framework, we were able to identify key contextual factors influencing the implementation of pediatric quality measures across a diverse range of clinical topics and settings. We found that factors related to the professional interactions, incentives and resources, and capacity for organizational change domains appeared to transcend measure topics and clinical settings as important drivers of implementation efforts.

      Acknowledgments

      The authors extend their gratitude to the individuals from the Pediatric Quality Measures Program grantee teams featured in this article; the authors' team is very grateful for their contributions, including technical guidance, support with data collection, and review of this manuscript. The authors also thank Robyn Sagatov, PhD, MHS, RDN from the Agency for Healthcare Quality and Research for her insightful reviews.
      Financial statement: This project was conducted under contract to Agency for Healthcare Research and Quality (contract number HHSA290201400003I / 75Q80119F32002 ).
      The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.
      This article is published as part of a supplement sponsored by the US Department of Health and Human Services, the Centers for Medicare and Medicaid Services, and the Agency for Healthcare Research and Quality.

      References

        • Ovretveit J
        Understanding the conditions for improvement: research to discover which context influences affect improvement success.
        BMJ Qual Saf. 2011; 20: i18-i23
        • Kaplan HC
        • Brady PW
        • Dritz MC
        • et al.
        The influence of context on quality improvement success in health care: a systematic review of the literature.
        Milbank Q. 2010; 88: 500-559https://doi.org/10.1111/j.1468-0009.2010.00611.x
        • McDonald KM
        Considering context in quality improvement interventions and implementation: concepts, frameworks, and application.
        Acad Pediatr. 2013; 13: S45-S53https://doi.org/10.1016/j.acap.2013.04.013
        • Stover AM
        • Haverman L
        • van Oers HA
        • et al.
        Using an implementation science approach to implement and evaluate patient-reported outcome measures (PROM) initiatives in routine care settings.
        Qual Life Res. 2020; ([e-pub ahead of print])https://doi.org/10.1007/s11136-020-02564-9
        • Tomoaia-Cotisel A
        • Scammon DL
        • Waitzman NJ
        • et al.
        Context matters: the experience of 14 research teams in systematically reporting contextual factors important for practice change.
        Ann Fam Med. 2013; 11: S115-S123https://doi.org/10.1370/afm.1549
        • Lee RM
        • Ramanadhan S
        • Kruse GR
        • et al.
        A mixed methods approach to evaluate partnerships and implementation of the Massachusetts Prevention and Wellness Trust Fund.
        Front Public Health. 2018; 6 (Published 2018 Jun 5): 150https://doi.org/10.3389/fpubh.2018.00150
        • Zamboni K
        • Baker U
        • Tyagi M
        • et al.
        How and under what circumstances do quality improvement collaboratives lead to better outcomes? A systematic review.
        Implement Sci. 2020; 15: 27https://doi.org/10.1186/s13012-020-0978-z
        • Flottorp SA
        • Oxman AD
        • Krause J
        • et al.
        A checklist for identifying determinants of practice: a systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice.
        Implement Sci. 2013; 8 (Published 2013 Mar 23): 35https://doi.org/10.1186/1748-5908-8-35
        • Shea CM
        • Turner K
        • Albritton J
        • et al.
        Contextual factors that influence quality improvement implementation in primary care: the role of organizations, teams, and individuals.
        Health Care Manage Rev. 2018; 43: 261-269https://doi.org/10.1097/HMR.0000000000000194
        • Higgins A
        • Murphy R
        • Downes C
        • et al.
        Factors impacting the implementation of a psychoeducation intervention within the mental health system: a multisite study using the consolidation framework for implementation research.
        BMC Health Serv Res. 2020; 20 (Published 2020 Nov 9): 1023https://doi.org/10.1186/s12913-020-05852-9
        • Sarkies M
        • Long JC
        • Pomare C
        • et al.
        Avoiding unnecessary hospitalisation for patients with chronic conditions: a systematic review of implementation determinants for hospital avoidance programmes.
        Implement Sci. 2020; 15 (Published 2020 Oct 21): 91https://doi.org/10.1186/s13012-020-01049-0
        • Weiss CH
        Why do we fail to deliver evidence-based practice in critical care medicine?.
        Curr Opin Crit Care. 2017; 23: 400-405https://doi.org/10.1097/MCC.0000000000000436
        • Breland JY
        • Asch SM
        • Slightam C
        • et al.
        Key ingredients for implementing intensive outpatient programs within patient-centered medical homes: a literature review and qualitative analysis.
        Healthc (Amst). 2016; 4: 22-29https://doi.org/10.1016/j.hjdsi.2015.12.005
        • Karltun A
        • Sanne JM
        • Aase K
        • et al.
        Knowledge management infrastructure to support quality improvement: a qualitative study of maternity services in four European hospitals [published correction appears in Health Policy. 2020 May;124:575].
        Health Policy. 2020; 124: 205-215https://doi.org/10.1016/j.healthpol.2019.11.005
        • Koball H
        • Moore A
        • Hernandez J
        Basic Facts About Low-Income Children: Children Under 9 Years, 2019.
        National Center for Children in Poverty, Bank Street College of Education, New York2021
        • Schwartz S
        • Rehder K
        Quality improvement in pediatrics: past, present, and future.
        Pediatr Res. 2017; 81: 156-161https://doi.org/10.1038/pr.2016.192
        • Coles E
        • Anderson J
        • Maxwell M
        • et al.
        The influence of contextual factors on healthcare quality improvement initiatives: a realist review.
        Syst Rev. 2020; 9 (Published 2020 Apr 26): 94https://doi.org/10.1186/s13643-020-01344-3