Advertisement
Academic Pediatrics
Advanced searchSave search
Overview| Volume 14, ISSUE 5, SUPPLEMENT , S1-S3, September 2014

Advancing the Science of Measurement in Pediatric Quality of Care

DOI:https://doi.org/10.1016/j.acap.2014.06.016
Advancing the Science of Measurement in Pediatric Quality of Care
Next ArticleA CHIPRA Quality Demonstration Grantee's Perspective on the Child Core Measure Set and Quality Measurement
      The Children's Health Insurance Program Reauthorization Act (CHIPRA) of 2009 provides an opportunity to consider and enhance the measurement of children's health care quality as a means to improve child health.

      Children's Health Insurance Program Reauthorization Act of 2009. Available at: http://www.gpo.gov/fdsys/pkg/PLAW-111publ3/html/PLAW-111publ3.htm. Accessed June 17, 2014.

      The legislation required the identification of an initial child core set of measures for voluntary use by Medicaid/Child Health Insurance (CHIP) programs. The initial child core set was published in 2009, and after its most current update in 2014, the list now includes 23 pediatric measures.

      Medicaid. 2014 core set of children's health care quality measures for Medicaid and CHIP. Available at: http://www.medicaid.gov/Medicaid-CHIP-Program-Information/By-Topics/Quality-of-Care/Downloads/ChildCoreMeasures.pdf. Accessed June 17, 2014.

      The legislation further mandated a CHIPRA Pediatric Quality Measures Program (PQMP), which in turn, funded 7 centers of excellence across the United States to develop and test pediatric quality measures.
      • Mistry K.B.
      • Chesley F.
      • LLanos K.
      • et al.
      Advancing children's healthcare through the pediatric quality measures program.
      Acad Pediatr. 2014; 14: s19-s26
      The PQMP is tasked with increasing the “portfolio of evidence-based, consensus pediatric quality measures available to public and private purchasers of children's health care services, providers and consumers.”

      Children's Health Insurance Program Reauthorization Act of 2009. Available at: http://www.gpo.gov/fdsys/pkg/PLAW-111publ3/html/PLAW-111publ3.htm. Accessed June 17, 2014.

      The articles in this supplement examine the opportunities and challenges associated with the PQMP centers of excellence work focused on advancing the measurement science for pediatric quality measures. The articles in this supplement fall into 3 broad themes: the value of pediatric quality measures to stakeholders; the scope of the PQMP measurement initiative; and challenges in developing and testing pediatric quality measures.

      Value of Measurement in Pediatric Care

      The value of measuring health care quality is no longer debated. Measurement is a critical step toward achieving the triple aim of better care, better population health, and more affordable care.
      • Conway P.F.
      • Mostashari F.
      • Clancy C.
      The future of quality measurement for improvement and accountability.
      JAMA. 2013; 309: 2215-2216
      , 

      US Department of Health and Human Services. 2013 annual progress report to Congress: national strategy for quality improvement in health care. Available at: http://www.ahrq.gov/workingforquality/nqs/nqs2013annlrpt.htm. Accessed June 16, 2014.

      , 
      • Institute of Medicine
      Child and Adolescent Health and Health Care Quality: Measuring What Matters.
      National Academies Press, Washington, DC2011
      This recognition of the value of measurement occurs at a time of great change and potential opportunity in children's health care. As the primary exemplar, the 2010 Patient Protection and Affordable Care Act (ACA) will expand coverage of children primarily through the expansion of Medicaid, health insurance exchanges, mandatory coverage, improvements in CHIP, and expansion of dependent coverage up to age 26, and also includes multiple opportunities and billions in funding for testing promising quality improvement strategies.

      Kaiser Family Foundation. Focus on health reform: summary of the Affordable Care Act. Available at: http://kaiserfamilyfoundation.files.wordpress.com/2011/04/8061-021.pdf. Accessed June 17, 2014.

      , 

      Centers for Medicare and Medicaid Services. The CMS innovation center. Available at: http://innovation.cms.gov. Accessed June 16, 2014.

      , 

      Patient Centered Outcomes Research Institute (PCORI). PCORnet: The National Patient-Centered Clinical Research Network. December 28, 2013. Available at: http://www.pcori.org/funding-opportunities/pcornet-national-patient-centered-clinical-research-network/. Accessed June 16, 2014.

      Additionally, CHIPRA provisions have established state-level quality improvement projects.
      • Dougherty D.
      • Conway P.
      The 3Ts roadmap to transform US health care: the “how” of high-quality care.
      JAMA. 2008; 299: 2319-2321
      Independent of federal legislation, quality improvement efforts are becoming more widely adopted by hospitals and insurers. The American Board of Pediatrics requires participation in quality improvement for its maintenance of certification program.
      • Miles P.V.
      • Moyer V.A.
      Quality improvement and maintenance of certification.
      Acad Pediatr. 2013; 13: S14-S15
      Meaningful use of health information technology may further stimulate interest in and opportunities for pediatric performance measurement.
      A number of articles in this supplement highlight both the critical need for and promise of pediatric quality measurement from a variety of stakeholder perspectives. As the commentaries by Doetsch and Smith
      • Doetsch J.
      • Smith G.
      A CHIPRA quality demonstration grantee's perspective on the child core measure set and quality measurement.
      Acad Pediatr. 2014; 14: s4-s5
      and by Bannister et al
      • Bannister L.
      • Singer J.A.
      • Lawthers A.G.
      A CHIPRA quality demonstration grantee's perspective on the CHIPRA core measures set: uses, challenges, and the future.
      Acad Pediatr. 2014; 14: s6-s7
      report, states appreciate the national measurement effort because uniform reporting requirements improve the uptake of measures and help with the ability to use them to compare quality across states and across entities within states. Carolyn Allshouse of Family Voices reports that family advocates serve as key partners in the efforts to improve quality of care; she and other center authors document the important role that families play in the PQMP measurement development process.
      • Allshouse C.
      The importance of the family voice in quality measure development for children's health care.
      Acad Pediatr. 2014; 14: s8-s9
      , 
      • Byron S.C.
      • Gardner W.
      • Kleinman L.C.
      • et al.
      Developing measures for pediatric quality: methods and experiences of the CHIPRA pediatric quality measures program grantees.
      Acad Pediatr. 2014; 14: s27-s32
      As noted by Sachdeva et al
      • Sachdeva R.
      • McInerny T.
      • Perrin J.M.
      Quality measures and the practicing pediatrician: perspectives from the American Academy of Pediatrics.
      Acad Pediatr. 2014; 14: s10-s11
      and by Miller,
      • Miller M.R.
      • Schwalenstocher E.
      • Weitcha M.
      Importance of the pediatric quality measurement program in advancing children's health care: a view from children's hospitals.
      Acad Pediatr. 2014; 14: s12-s14
      pediatricians and children's hospitals are interested in clinically relevant measures as a part of the process to improve care; particularly measures that can serve multiple purposes such as quality improvement and accountability. Additionally, Silber and Forrest
      • Silber J.H.
      • Forrest C.B.
      Concept and measurement of pediatric value.
      Acad Pediatr. 2014; 14: s33-s38
      add to the national conversation about assessing value by discussing the importance of both taking the child/family perspective into account and using appropriate methods of standardization in the analysis of data from quality measurement measures designed to help patients and families.

      Scope of PQMP Measurement Initiative

      The majority of articles in this supplement focus on specific measurement topics and illustrate the broad scope of the PQMP measurement effort, as shaped by the breadth of topics in the CHIPRA legislation and the priorities of the Centers for Medicare and Medicaid Services. Articles by Nakamura et al
      • Nakamura M.M.
      • Toomey S.L.
      • Zaslavsky A.M.
      • et al.
      An overview of pediatric hospital readmission rates as a quality of care measure.
      Acad Pediatr. 2014; 14: s39-s46
      and Lorch et al
      • Lorch S.A.
      • Passarella M.
      • Zeigler A.
      Challenges to determining state-level variation in readmission rates of neonatal intensive care patients.
      Acad Pediatr. 2014; 14: s47-s53
      are related to the controversial topic of measuring hospital readmissions in neonatal and general pediatric care. Several articles address development of measures related to care for chronic conditions that are both common, such as the article on attention-deficit/hyperactivity disorder outcomes by Woods et al,
      • Woods D.
      • Wolraich M.
      • Pierce K.
      • et al.
      Considerations and evidence for an ADHD outcome measure.
      Acad Pediatr. 2014; 14: s54-s60
      and relatively low prevalence, such as measures of the quality of sickle cell care by Reeves et al
      • Reeves S.L.
      • Garcia E.
      • Kleyn M.
      • et al.
      Identifying sickle cell disease cases using administrative data.
      Acad Pediatr. 2014; 14: S61-S67
      and the quality of use of antipsychotic drugs by Kealey et al.
      • Kealey E.
      • Scholle S.H.
      • Byron S.C.
      • et al.
      Quality concerns in antipsychotic prescribing for youth: a review of treatment guidelines.
      Acad Pediatr. 2014; 14: s68-s75
      Cross-cutting articles by Byron and colleagues
      • Byron S.C.
      • Gardner W.
      • Kleinman L.C.
      • et al.
      Developing measures for pediatric quality: methods and experiences of the CHIPRA pediatric quality measures program grantees.
      Acad Pediatr. 2014; 14: s27-s32
      and Mistry and colleagues
      • Mistry K.B.
      • Chesley F.
      • LLanos K.
      • et al.
      Advancing children's healthcare through the pediatric quality measures program.
      Acad Pediatr. 2014; 14: s19-s26
      provide details about measures being developed on a wide range of additional topics across the centers.

      Challenges in Developing and Testing Pediatric Quality Measures

      A number of the topic-specific articles noted above, as well as the cross-topic articles in the supplement, detail the centers' experiences with various steps in the measure development process and challenges the centers encountered. As noted by Mistry et al
      • Mistry K.B.
      • Chesley F.
      • LLanos K.
      • et al.
      Advancing children's healthcare through the pediatric quality measures program.
      Acad Pediatr. 2014; 14: s19-s26
      and Byron et al,
      • Byron S.C.
      • Gardner W.
      • Kleinman L.C.
      • et al.
      Developing measures for pediatric quality: methods and experiences of the CHIPRA pediatric quality measures program grantees.
      Acad Pediatr. 2014; 14: s27-s32
      although measure topics were assigned to the centers of excellence, the assignments were often quite broad. Literature reviews can be helpful in narrowing a topic to specific measure foci, based on where the best evidence exists. Several of the articles include extensive reviews of the evidence.
      • Nakamura M.M.
      • Toomey S.L.
      • Zaslavsky A.M.
      • et al.
      An overview of pediatric hospital readmission rates as a quality of care measure.
      Acad Pediatr. 2014; 14: s39-s46
      , 
      • Woods D.
      • Wolraich M.
      • Pierce K.
      • et al.
      Considerations and evidence for an ADHD outcome measure.
      Acad Pediatr. 2014; 14: s54-s60
      , 
      • Kealey E.
      • Scholle S.H.
      • Byron S.C.
      • et al.
      Quality concerns in antipsychotic prescribing for youth: a review of treatment guidelines.
      Acad Pediatr. 2014; 14: s68-s75
      After the focus of a measure has been identified, the measures must be clearly specified and subsequently (and iteratively) tested, typically with real data.
      • Byron S.C.
      • Gardner W.
      • Kleinman L.C.
      • et al.
      Developing measures for pediatric quality: methods and experiences of the CHIPRA pediatric quality measures program grantees.
      Acad Pediatr. 2014; 14: s27-s32
      Several articles point out data-related challenges and associated implications for measure implementation. For example, Bannister and colleagues
      • Bannister L.
      • Singer J.A.
      • Lawthers A.G.
      A CHIPRA quality demonstration grantee's perspective on the CHIPRA core measures set: uses, challenges, and the future.
      Acad Pediatr. 2014; 14: s6-s7
      point out that slight differences state-to-state in Medicaid administration or data collection can lead to differences in the implementation of measures and can reduce the comparability of measures. Gidengil and colleagues
      • Gidengil C.
      • Mangione-Smith R.
      • Bailey C.
      • et al.
      Using Medicaid and CHIP claims data to support pediatric quality measurement: lessons from three centers of excellence in measure development.
      Acad Pediatr. 2014; 14: s76-s81
      examine issues relevant to Medicaid and CHIP claims data and recommend steps to make these systems more helpful to quality measurement. Some, but not all, challenges are remediable by measure developers. Reeves et al
      • Reeves S.L.
      • Garcia E.
      • Kleyn M.
      • et al.
      Identifying sickle cell disease cases using administrative data.
      Acad Pediatr. 2014; 14: S61-S67
      relate an innovative approach to challenges in validating Medicaid claims data by using data from a state's newborn screening program. Looking to the future, Bailey and colleagues
      • Bailey C.
      • Tinoco A.
      • Mistry K.B.
      • et al.
      Addressing electronic clinical information in the construction of quality measures.
      Acad Pediatr. 2014; 14: s82-s89
      discuss the advantages and disadvantages of using claims data and electronic medical records in their current state of development and address the challenges as well as the promise of future electronic clinical information for measurement.
      Bevans et al
      • Bevans K.
      • Moon J.
      • Carle A.C.
      • et al.
      Patient-reported outcomes as indicators of pediatric health care quality.
      Acad Pediatr. 2014; 14: s90-s96
      report on the potential of Patient-Reported Outcome-Performance Measures (PRO-PMs) for making measurement more meaningful by integrating the patient perspective and engaging more families in the quality enterprise. Despite attention to these measures under the Patient Reported Outcome Measurement Information System (PROMIS), PRO-PMs have not been extensively incorporated into quality measurement sets thus far. As noted by Silber and Forrest,
      • Silber J.H.
      • Forrest C.B.
      Concept and measurement of pediatric value.
      Acad Pediatr. 2014; 14: s33-s38
      prioritizing high-value outcomes that matter most to the patient (and family) may further support the use of PRO-PMs, and increased support may move the field toward resolution of technical issues such as the use of PRO-PMs in electronic health records.
      However, as Austin and colleagues
      • Austin J.M.
      • Miller M.R.
      • Pronovost P.J.
      Young and reckless? Greater standardization and transparency of performance is needed for pediatric performance measures.
      Acad Pediatr. 2014; 14: s15-s16
      note in their commentary, more information about the performance of performance measures is a vital consideration for the future because it will guide the choices about which measures to use. In addition, Dougherty and colleagues
      • Dougherty D.
      • Mistry K.
      • Lindy O.
      • et al.
      A systematic evidence-informed quality measurement life-cycle approach to measure retirement in CHIPRA.
      Acad Pediatr. 2014; 14: s97-s104
      discuss the role of periodic assessment of measures for possible retirement as part of a life-cycle approach to measurement that again highlights the need to continually evaluate use and implementation of measures over time.

      Conclusion

      The articles in this supplement are intended to be useful to a variety of readers, including individuals and organizations choosing measures to assess and monitor health care quality, individuals and systems whose care is being measured, future developers of quality measures, and child-focused clinical and health services researchers. As noted by the Institute of Medicine in 2005
      • Institute of Medicine
      Performance Measurement: Accelerating Improvement.
      National Academies Press, Washington, DC2005
      http://www.iom.edu/reports/2005/performance-measurement-accelerating-improvement.aspx
      and 2011,

      Kaiser Family Foundation. Focus on health reform: summary of the Affordable Care Act. Available at: http://kaiserfamilyfoundation.files.wordpress.com/2011/04/8061-021.pdf. Accessed June 17, 2014.

      and more recently by Berenson et al,

      Berenson RA, Pronovost PJ, Krumholz HM. Achieving the potential of health care performance measures. May 2013. Available at: http://www.rwjf.org/en/research-publications/find-rwjf-research/2013/05/achieving-the-potential-of-health-care-performance-measures.html. Accessed June 22, 2014.

      the high stakes often associated with quality measurement have intensified the need for transparent measure development and testing. Understanding the careful and challenging process of developing pediatric quality measures will help consumers and providers understand the inherent strengths and limitations of different measures and encourage the use of useful and comprehensive sets of measures to monitor improvements in quality and outcomes.
      The implementers of CHIPRA were specifically tasked with improving the quality and outcomes of care for children, with a special focus on the more than 42 million children enrolled in Medicaid and CHIP.

      Medicaid. FY 2010: number of children ever enrolled in Medicaid and CHIP. Available at: http://www.medicaid.gov/Medicaid-CHIP-Program-Information/By-Topics/Childrens-Health-Insurance-Program-CHIP/Downloads/2010_enrollment_datapdf.pdf. Accessed June 18, 2014.

      Improving the measurement of quality in pediatrics is an important step in the process of improving the care and well-being of US children. The articles in this supplement illustrate the considerations necessary for creating good measure sets and provide strategies for overcoming challenges encountered in the measurement development process.

      Acknowledgments

      A Pediatric Quality Measures Program grant (U18HS20408) from the Agency for Healthcare Research and Quality supported Dr Forrest's work.

      References

      1. Children's Health Insurance Program Reauthorization Act of 2009. Available at: http://www.gpo.gov/fdsys/pkg/PLAW-111publ3/html/PLAW-111publ3.htm. Accessed June 17, 2014.

        View in Article

      2. Medicaid. 2014 core set of children's health care quality measures for Medicaid and CHIP. Available at: http://www.medicaid.gov/Medicaid-CHIP-Program-Information/By-Topics/Quality-of-Care/Downloads/ChildCoreMeasures.pdf. Accessed June 17, 2014.

        View in Article
        • Mistry K.B.
        • Chesley F.
        • LLanos K.
        • et al.
        Advancing children's healthcare through the pediatric quality measures program.
        Acad Pediatr. 2014; 14: s19-s26
        View in Article
        • Conway P.F.
        • Mostashari F.
        • Clancy C.
        The future of quality measurement for improvement and accountability.
        JAMA. 2013; 309: 2215-2216
        View in Article

      5. US Department of Health and Human Services. 2013 annual progress report to Congress: national strategy for quality improvement in health care. Available at: http://www.ahrq.gov/workingforquality/nqs/nqs2013annlrpt.htm. Accessed June 16, 2014.

        View in Article
        • Institute of Medicine
        Child and Adolescent Health and Health Care Quality: Measuring What Matters.
        National Academies Press, Washington, DC2011
        View in Article

      7. Kaiser Family Foundation. Focus on health reform: summary of the Affordable Care Act. Available at: http://kaiserfamilyfoundation.files.wordpress.com/2011/04/8061-021.pdf. Accessed June 17, 2014.

        View in Article

      8. Centers for Medicare and Medicaid Services. The CMS innovation center. Available at: http://innovation.cms.gov. Accessed June 16, 2014.

        View in Article

      9. Patient Centered Outcomes Research Institute (PCORI). PCORnet: The National Patient-Centered Clinical Research Network. December 28, 2013. Available at: http://www.pcori.org/funding-opportunities/pcornet-national-patient-centered-clinical-research-network/. Accessed June 16, 2014.

        View in Article
        • Dougherty D.
        • Conway P.
        The 3Ts roadmap to transform US health care: the “how” of high-quality care.
        JAMA. 2008; 299: 2319-2321
        View in Article
        • Miles P.V.
        • Moyer V.A.
        Quality improvement and maintenance of certification.
        Acad Pediatr. 2013; 13: S14-S15
        View in Article
        • Doetsch J.
        • Smith G.
        A CHIPRA quality demonstration grantee's perspective on the child core measure set and quality measurement.
        Acad Pediatr. 2014; 14: s4-s5
        View in Article
        • Bannister L.
        • Singer J.A.
        • Lawthers A.G.
        A CHIPRA quality demonstration grantee's perspective on the CHIPRA core measures set: uses, challenges, and the future.
        Acad Pediatr. 2014; 14: s6-s7
        View in Article
        • Allshouse C.
        The importance of the family voice in quality measure development for children's health care.
        Acad Pediatr. 2014; 14: s8-s9
        View in Article
        • Byron S.C.
        • Gardner W.
        • Kleinman L.C.
        • et al.
        Developing measures for pediatric quality: methods and experiences of the CHIPRA pediatric quality measures program grantees.
        Acad Pediatr. 2014; 14: s27-s32
        View in Article
        • Sachdeva R.
        • McInerny T.
        • Perrin J.M.
        Quality measures and the practicing pediatrician: perspectives from the American Academy of Pediatrics.
        Acad Pediatr. 2014; 14: s10-s11
        View in Article
        • Miller M.R.
        • Schwalenstocher E.
        • Weitcha M.
        Importance of the pediatric quality measurement program in advancing children's health care: a view from children's hospitals.
        Acad Pediatr. 2014; 14: s12-s14
        View in Article
        • Silber J.H.
        • Forrest C.B.
        Concept and measurement of pediatric value.
        Acad Pediatr. 2014; 14: s33-s38
        View in Article
        • Nakamura M.M.
        • Toomey S.L.
        • Zaslavsky A.M.
        • et al.
        An overview of pediatric hospital readmission rates as a quality of care measure.
        Acad Pediatr. 2014; 14: s39-s46
        View in Article
        • Lorch S.A.
        • Passarella M.
        • Zeigler A.
        Challenges to determining state-level variation in readmission rates of neonatal intensive care patients.
        Acad Pediatr. 2014; 14: s47-s53
        View in Article
        • Woods D.
        • Wolraich M.
        • Pierce K.
        • et al.
        Considerations and evidence for an ADHD outcome measure.
        Acad Pediatr. 2014; 14: s54-s60
        View in Article
        • Reeves S.L.
        • Garcia E.
        • Kleyn M.
        • et al.
        Identifying sickle cell disease cases using administrative data.
        Acad Pediatr. 2014; 14: S61-S67
        View in Article
        • Kealey E.
        • Scholle S.H.
        • Byron S.C.
        • et al.
        Quality concerns in antipsychotic prescribing for youth: a review of treatment guidelines.
        Acad Pediatr. 2014; 14: s68-s75
        View in Article
        • Gidengil C.
        • Mangione-Smith R.
        • Bailey C.
        • et al.
        Using Medicaid and CHIP claims data to support pediatric quality measurement: lessons from three centers of excellence in measure development.
        Acad Pediatr. 2014; 14: s76-s81
        View in Article
        • Bailey C.
        • Tinoco A.
        • Mistry K.B.
        • et al.
        Addressing electronic clinical information in the construction of quality measures.
        Acad Pediatr. 2014; 14: s82-s89
        View in Article
        • Bevans K.
        • Moon J.
        • Carle A.C.
        • et al.
        Patient-reported outcomes as indicators of pediatric health care quality.
        Acad Pediatr. 2014; 14: s90-s96
        View in Article
        • Austin J.M.
        • Miller M.R.
        • Pronovost P.J.
        Young and reckless? Greater standardization and transparency of performance is needed for pediatric performance measures.
        Acad Pediatr. 2014; 14: s15-s16
        View in Article
        • Dougherty D.
        • Mistry K.
        • Lindy O.
        • et al.
        A systematic evidence-informed quality measurement life-cycle approach to measure retirement in CHIPRA.
        Acad Pediatr. 2014; 14: s97-s104
        View in Article
        • Institute of Medicine
        Performance Measurement: Accelerating Improvement.
        National Academies Press, Washington, DC2005 (Available at:)
        http://www.iom.edu/reports/2005/performance-measurement-accelerating-improvement.aspx
        (Accessed June 22, 2014)
        View in Article

      30. Berenson RA, Pronovost PJ, Krumholz HM. Achieving the potential of health care performance measures. May 2013. Available at: http://www.rwjf.org/en/research-publications/find-rwjf-research/2013/05/achieving-the-potential-of-health-care-performance-measures.html. Accessed June 22, 2014.

        View in Article

      31. Medicaid. FY 2010: number of children ever enrolled in Medicaid and CHIP. Available at: http://www.medicaid.gov/Medicaid-CHIP-Program-Information/By-Topics/Childrens-Health-Insurance-Program-CHIP/Downloads/2010_enrollment_datapdf.pdf. Accessed June 18, 2014.

        View in Article

      Article info

      Publication history

      Accepted: June 24, 2014
      Received: June 20, 2014

      Footnotes

      Publication of this article was supported by the US Department of Health and Human Services and the Agency for Healthcare Research and Quality.

      The views expressed in this report are those of the authors and do not necessarily represent those of the US Department of Health and Human Services, the Agency for Healthcare Research and Quality.

      The authors declare that they have no conflict of interest.

      Identification

      DOI: https://doi.org/10.1016/j.acap.2014.06.016

      Copyright

      © 2014 Academic Pediatric Association. Published by Elsevier Inc. All rights reserved.

      ScienceDirect

      Access this article on ScienceDirect
      We use cookies to help provide and enhance our service and tailor content. To update your cookie settings, please visit the for this site.
      Copyright © 2023 Elsevier Inc. except certain content provided by third parties. The content on this site is intended for healthcare professionals.


      RELX