Advertisement

PQMP Phase 2: Implementation and Dissemination

      Keywords

      In 2009, the Children's Health Insurance Program Reauthorization Act provided an opportunity to consider and enhance the measurement of children's health care quality as a means to improve child health.

      Children's Health Insurance Program Reauthorization Act of 2009. Public Law 111–113. Available at: https://www.govinfo.gov/content/pkg/PLAW-111publ3/pdf/PLAW-111publ3.pdf. Accessed January 19, 2022.

      The legislation required the identification of an initial Child Core Set of pediatric quality measures for voluntary use by Medicaid/Child Health Insurance Programs. The initial child core set was published in 2009 and is updated annually with 2024 being the first year of mandatory reporting.

      Centers for Medicare and Medicaid Services, Centers for Medicaid and CHIP Services. 2020. CMCS informational bulletin, subject: 2021 updates to the Child and Adult Core Health Care Quality Measurement sets. 7500 Security Boulevard, Mail Stop S2-26-12, Baltimore, Maryland 21244-1850. Available at:https://www.medicaid.gov/federal-policy-guidance/downloads/cib111920.pdf. Accessed January 19, 2022.

      The legislation further established the Children's Health Insurance Program Reauthorization Act Pediatric Quality Measures Program (PQMP), the first such focused federal investment in addressing longstanding gaps in the science, tools, and use of health care quality measurement for children and youth. Through a more than a decade-long partnership, the Agency for Healthcare Research and Quality and Centers for Medicare and Medicaid Services developed and have led two phases of PQMP work. The first phase (PQMP 1.0) focused on development of a portfolio of new, evidence-based, pediatric measures of which more than 25 were endorsed by the National Quality Forum.
      • Kuhlthau KA
      • Mistry KB
      • Forrest CB
      • et al.
      Advancing the science of measurement in pediatric quality of are.
      The work by the initial Centers of Excellence (Centers) in PQMP 1.0 provided the foundation for a second phase (PQMP 2.0) that deepened the evidence base regarding the implementation of measures in real-world settings with a particular focus on usability and feasibility which often serve as barriers to measure use in quality improvement efforts. In achieving this focus, the work of PQMP 2.0 Centers was explicitly designed to be at the intersection of both quality improvement and implementation science. This led to an integration that went beyond just “testing” to understanding the contextual factors that explain the how and why undergirding improvements in pediatric care processes or outcomes.
      The 15 articles in this supplement examine the opportunities and challenges associated with the PQMP 2.0 Centers' work and how findings may advance the science for pediatric quality measurement and improvement, and ultimately child health outcomes. In their brief report, Mistry et al describe the structure of the PQMP 2.0, the unique aspects of the Pediatric Quality Measures Program Learning Collaborative (PQMP LC), and the set of “research foci” that served as a cornerstone to look both within and across projects to assess the role of measure usability and feasibility in pediatric measure implementation and quality improvement initiatives.
      • Mistry KB
      • Sagatov RDF
      • Schur C
      • et al.
      Design and implementation of the pediatric quality measures program 2.0.
      The remaining articles in this supplement fall into 4 overarching themes: importance of stakeholder engagement; barriers and facilitators of quality measure implementation; challenges of measure implementation across hierarchical levels of complex health systems; and, digitization of quality measures. Overall, the Centers not only implemented new measures within quality improvement projects across a variety of health system levels and settings but unpacked and increased the evidence-base regarding realizing and sustaining improvements in quality of care for children.

      Importance of Stakeholder Engagement

      Several articles highlight the critical need to directly invest both time and resources in stakeholder collaboration in order to make research questions and findings more meaningful and relevant. A key goal of all PQMP Centers' activities was to ensure that end-users were integrally involved in both developing measures and the process of implementing measures in quality improvement efforts. In fact, at the outset, the PQMP learning collaborative was designed to continually engage with experts and stakeholders to drive a deeper understanding of the research foci and how to use them to answer “pain points” in both measuring and improving care. The article by Schur and colleagues outlines a broad set of insights that allowed the Centers to balance research rigor with the “real world” policy environment that prioritizes speed, feasibility, and usability.
      • Schur C
      • Johnson M
      • Doherty J
      • et al.
      Real-World considerations for implementing pediatric quality measures: insights from key stakeholders.
      Shenkman and co-authors also describe a process for stakeholder engagement that aimed to bridge research and policy maker needs in the development of a new dental sealant measure for the Medicaid Core Set.
      • Shenkman E
      • Mistry KB
      • Davis D
      • et al.
      Stakeholder engagement: bridging research and policy to improve measurement and dental care for children in Medicaid.
      The Dental Managed Care Organizations had a broad team of researchers join with end-users, which was critical in identifying real world challenges and addressing limitations which ultimately led to a more feasible and accurate measure. Work by Thorp et al emphasizes the benefits (and lessons learned) in deliberately incorporating end-user feedback not only in measure development but also in designing and implementing quality improvement initiatives.
      • Thorp K
      • Van CM
      • Olin SS
      • et al.
      Integrating youth voice in health plan quality improvement.
      In the article, the team discusses their experience working with multiple health plans to improve delivery of mental health care for adolescents. Plans invested substantial time and dedicated resources to building relationships with youth and families as trusted advisors with valuable lived experiences. The plans observed the deep value in sustaining and even expanding their outreach efforts to engage youth and their caregivers to make the work more meaningful and impactful.

      Barriers and Facilitators of Quality Measure Implementation: Context Matters

      A second overarching theme of articles featured in this supplement is the importance of context to quality improvement. Paying attention to contextual or environmental factors is especially important for pragmatic implementation intentionally designed for real world health care settings. The findings included in the articles highlight how implementation science approaches support the generation of in-depth analyses that can be instrumental in designing and conducting targeted improvement strategies. As Cope and colleagues describe, the Centers prioritized contextual factors that affect effectiveness of quality improvement initiatives.
      • Cope E
      • Johnson M
      • Khan M
      • et al.
      Contextual factors affecting implementation of pediatric quality improvement programs.
      Using an established implementation framework, the PQMP learning collaborative compared and contrasted multiple quality measure implementations in order to better assess the most influential contextual factors. Despite the heterogeneity in the settings and clinical focus areas for improvements across the Center projects, a core set of environmental factors emerged that influenced implementation. For example, the article by Leyenaar et al reflects on both advances and limitations in pediatric quality measures for hospitalized children.
      • Leyenaar JK
      • Esporas M
      • Mangione-Smith R.
      How does pediatric quality measure development reflect the real world needs of hospitalized children?.
      The authors note that the majority of pediatric hospitalizations occur in general hospitals yet quality improvement frameworks and strategies are often developed and tested in free-standing children's hospitals. The discussion provides examples of contextual factors that vary across the two settings and illustrates the potential impacts on the usability of existing pediatric hospital-based quality measures. Parast, Shenkman, and colleagues demonstrate variability in the receipt of appropriate pediatric mental health care using a newly developed comprehensive typology of pediatric psychiatric disorders.
      • Parast L
      • Burkhart Q
      • Bardach NS
      • et al.
      Development and testing of an emergency department quality measure for pediatric suicidal ideation and self-harm.
      ,
      • Shenkman E
      • Huo T
      • Li Q
      • et al.
      Florida Medicaid Children’s Receipt of First Line Psychosocial Care Prior to Antipsychotic Initiation.
      Findings from both articles highlight concerning intersecting vulnerabilities among children and youth based on sociodemographic factors and mental health characteristics. The detailed analyses focused on examining disparities and help pinpoint targets for improvement initiatives.
      Relatedly, Bucholz and co-authors also highlight the need to carefully consider how to incorporate compositional (ie, patient sociodemographic and clinical factors) and environmental factors into measurement and improvement and point out the uncertainties in current case-mix approaches.
      • Bucholz EM
      • Toomey SL
      • McCulloch CE
      • et al.
      Adjusting for social risk factors in pediatric quality measures: adding to the evidence base.
      Using the pediatric asthma emergency department use measure as an example, they discuss the merits, feasibility, and methodological challenges of adjusting for patient factors in pediatric quality assessment.

      Challenges Across Hierarchical Levels of Complex Health Systems

      Several articles focus on challenges and barriers to using measurement and improvement strategies across multiple levels of health systems. This is not surprising given the explicit focus of the “research foci” on understanding the process of folding measures up and down and the potential benefits of alignment of measures across levels to quality improvement work. Taken together, the articles make abundantly clear the critical need for a broader systems science orientation, one that relies on mutually beneficial collaborations to drive improvements in care across levels. Olin and coauthors describe levers across multiple levels of care delivery (such as aligned priorities, common data platforms, champions, clear QI methods, common financial incentives), that, when addressed a priori, facilitate lasting and durable improvements.
      • Olin SS
      • Freed GL
      • Scholle SH
      • et al.
      Aligning to improve pediatric health care quality.
      Additionally, Reeves et al discuss the role of attribution in aggregating data used in measuring quality at varying levels of the health care system.
      • Reeves SL
      • Dombkowski KJ
      • Madden B
      • et al.
      Considerations when aggregating data to measure performance across levels of the health care system.
      Using case studies based on the work of two Centers, they evaluate two differing attribution models and assess the impact of the alternate models across population characteristics such as managed care enrollment, continuous enrollment criteria, or enrollment in a program. However, in applying the models, they noted inherent challenges and even unintentional consequences such as widening of health disparities, that are worthy of consideration. Bardach and coauthors examine the association between outpatient 14-day follow-up, per evidence-based guidelines, and asthma-related ED revisits in a multistate analysis.
      • Bardach NS
      • Harder VS
      • McCulloch C
      • et al.
      Follow-up after asthma emergency department visits and its relationship with subsequent asthma-related utilization.
      Their findings indicate a protective relationship yet also highlight an opportunity for improvement, with only 23% of those with asthma-related ED visits having 14-day follow-up and a lower likelihood of follow up in Medicaid children compared to those with commercial health insurance. The authors discuss the complexity of any intervention to mitigate this problem, as it would require a complex systems approach allowing for connections between levels of care, going beyond ED visit and discharge processes. Relatedly, Cabana and colleagues also illuminate the need for moving beyond a single level or practice-based approach and contend that improving outcomes for children with asthma will require a systems-approach, with health care delivery viewed holistically and broadly.
      • Cabana M
      • Robinson K
      • Plavin J.
      Partnering to improve pediatric asthma quality.

      Toward Digitization of Quality Measures

      An important theme related the challenges Centers faced in working across multiple levels of care delivery relates to the development and testing of quality measures for children on the basis of electronic clinical information, including de novo measures and specification of existing measures designed for other data sources. While Centers for Medicare and Medicaid Services has a goal of all quality measure to be digital by 2030, drawing on the experience of the Centers, there remain both structural and pragmatic considerations in harnessing the true value of clinical information and data quality management systems. Morden et al describe the challenges faced by health plans to report quality measures for adolescent and perinatal depression due to limited availability of clinical data usable for quality improvement work.
      • Morden E
      • Byron S
      • Roth L
      • et al.
      Health plans struggle to report on depression quality measures that require clinical data.
      The findings echo those noted by Olin and point to a need to incentivize collaborative data collection and sharing to realize quality improvement goals for depression. Huo and coauthors also describe the promise and difficulty of data linkages (EHR with claims and enrollment data) to acquire clinical information on body mass index, prescribed and dispensed medications, and sociodemographic characteristics for their examination of variations in receipt of psychosocial care before antipsychotic medication among children in Florida Medicaid.
      • Huo T
      • Li Q
      • Cardel M
      • et al.
      Enhancing quality measurement with clinical information: a use case of body mass index change among children taking second generation antipsychotics.
      The analysis provided valuable information about how clinical data can be used to identify children most at-risk for antipsychotic-induced weight gain, which cannot be done using process-focused measures. Yet results also demonstrated the importance of addressing gaps in available clinical information by carefully assessing underlying causes given findings indicating that missing clinical information disproportionately affects different subgroups of children, potentially contributing to inequities in assessing the quality of care.

      Conclusion

      The articles in this supplement richly illustrate the current state of quality measurement and implementation in support of pediatric care improvement. They identify a number of real-world successes and challenges that are useful and pragmatic lessons learned for the future. In addition, they speak to the importance of designing the PQMP learning collaborative model from the outset to harness evidence generated within and across the Centers. Employing an implementation science lens throughout allowed for a deeper look into the how and the why of both successes and challenges encountered, further advancing our knowledge thus generating enhanced value to the field. Finally, while the successes provide insights into best practices for the field, the challenges, some of which were insurmountable within the scope of PQMP 2.0, are equally important to acknowledge as they shine the light on priority areas for future work to continue the drive for excellence in quality of care for all children.

      Disclaimer

      The views expressed in this article are those of the authors, and no official endorsement by the Agency for Healthcare Research and Quality (AHRQ), the Centers for Medicare and Medicaid Services (CMS), or the Department of Health and Human Services (DHHS) is intended or should be inferred. The Pediatric Quality Measures Program 2.0 was funded by AHRQ and CMS.

      Acknowledgments

      The authors thank the Pediatric Quality Measures Program Centers of Excellence, L&M Policy Research, and AcademyHealth for their contributions to this important effort. The authors also thank Robyn D.F. Sagatov, PhD, MHS, RDN from the Agency for Healthcare Research and Quality for her contributions.
      The views expressed in this article are those of the authors, and no official endorsement by the Agency for Healthcare Research and Quality (AHRQ), the Centers for Medicare and Medicaid Services (CMS), or the Department of Health and Human Services (DHHS) is intended or should be inferred.
      This article is published as part of a supplement sponsored by the US Department of Health and Human Services, the Centers for Medicare and Medicaid Services, and the Agency for Healthcare Research and Quality.

      References

      1. Children's Health Insurance Program Reauthorization Act of 2009. Public Law 111–113. Available at: https://www.govinfo.gov/content/pkg/PLAW-111publ3/pdf/PLAW-111publ3.pdf. Accessed January 19, 2022.

      2. Centers for Medicare and Medicaid Services, Centers for Medicaid and CHIP Services. 2020. CMCS informational bulletin, subject: 2021 updates to the Child and Adult Core Health Care Quality Measurement sets. 7500 Security Boulevard, Mail Stop S2-26-12, Baltimore, Maryland 21244-1850. Available at:https://www.medicaid.gov/federal-policy-guidance/downloads/cib111920.pdf. Accessed January 19, 2022.

        • Kuhlthau KA
        • Mistry KB
        • Forrest CB
        • et al.
        Advancing the science of measurement in pediatric quality of are.
        Acad Pediatr. 2014; 14: S1-S3
        • Mistry KB
        • Sagatov RDF
        • Schur C
        • et al.
        Design and implementation of the pediatric quality measures program 2.0.
        Acad Pediatr. 2022; 22: S59-S64
        • Schur C
        • Johnson M
        • Doherty J
        • et al.
        Real-World considerations for implementing pediatric quality measures: insights from key stakeholders.
        Acad Pediatr. 2022; 22: S76-S80
        • Shenkman E
        • Mistry KB
        • Davis D
        • et al.
        Stakeholder engagement: bridging research and policy to improve measurement and dental care for children in Medicaid.
        Acad Pediatr. 2022; 22: S65-S67
        • Thorp K
        • Van CM
        • Olin SS
        • et al.
        Integrating youth voice in health plan quality improvement.
        Acad Pediatr. 2022; 22: S68-S69
        • Cope E
        • Johnson M
        • Khan M
        • et al.
        Contextual factors affecting implementation of pediatric quality improvement programs.
        Acad Pediatr. 2022; 22: S81-S91
        • Leyenaar JK
        • Esporas M
        • Mangione-Smith R.
        How does pediatric quality measure development reflect the real world needs of hospitalized children?.
        Acad Pediatr. 2022; 22: S70-S72
        • Parast L
        • Burkhart Q
        • Bardach NS
        • et al.
        Development and testing of an emergency department quality measure for pediatric suicidal ideation and self-harm.
        Acad Pediatr. 2022; 22: S92-S99
        • Shenkman E
        • Huo T
        • Li Q
        • et al.
        Florida Medicaid Children’s Receipt of First Line Psychosocial Care Prior to Antipsychotic Initiation.
        Acad Pediatr. 2022; 22: S100-S107
        • Bucholz EM
        • Toomey SL
        • McCulloch CE
        • et al.
        Adjusting for social risk factors in pediatric quality measures: adding to the evidence base.
        Acad Pediatr. 2022; 22: S108-S114
        • Olin SS
        • Freed GL
        • Scholle SH
        • et al.
        Aligning to improve pediatric health care quality.
        Acad Pediatr. 2022; 22: S115-S118
        • Reeves SL
        • Dombkowski KJ
        • Madden B
        • et al.
        Considerations when aggregating data to measure performance across levels of the health care system.
        Acad Pediatr. 2022; 22: S119-S124
        • Bardach NS
        • Harder VS
        • McCulloch C
        • et al.
        Follow-up after asthma emergency department visits and its relationship with subsequent asthma-related utilization.
        Acad Pediatr. 2022; 22: S125-S132
        • Cabana M
        • Robinson K
        • Plavin J.
        Partnering to improve pediatric asthma quality.
        Acad Pediatr. 2022; 22: S73-S75
        • Morden E
        • Byron S
        • Roth L
        • et al.
        Health plans struggle to report on depression quality measures that require clinical data.
        Acad Pediatr. 2022; 22: S133-S139
        • Huo T
        • Li Q
        • Cardel M
        • et al.
        Enhancing quality measurement with clinical information: a use case of body mass index change among children taking second generation antipsychotics.
        Acad Pediatr. 2022; 22: S140-S149