Advertisement

Advancing the Science of Measurement in Pediatric Quality of Care

      The Children's Health Insurance Program Reauthorization Act (CHIPRA) of 2009 provides an opportunity to consider and enhance the measurement of children's health care quality as a means to improve child health.

      Children's Health Insurance Program Reauthorization Act of 2009. Available at: http://www.gpo.gov/fdsys/pkg/PLAW-111publ3/html/PLAW-111publ3.htm. Accessed June 17, 2014.

      The legislation required the identification of an initial child core set of measures for voluntary use by Medicaid/Child Health Insurance (CHIP) programs. The initial child core set was published in 2009, and after its most current update in 2014, the list now includes 23 pediatric measures.

      Medicaid. 2014 core set of children's health care quality measures for Medicaid and CHIP. Available at: http://www.medicaid.gov/Medicaid-CHIP-Program-Information/By-Topics/Quality-of-Care/Downloads/ChildCoreMeasures.pdf. Accessed June 17, 2014.

      The legislation further mandated a CHIPRA Pediatric Quality Measures Program (PQMP), which in turn, funded 7 centers of excellence across the United States to develop and test pediatric quality measures.
      • Mistry K.B.
      • Chesley F.
      • LLanos K.
      • et al.
      Advancing children's healthcare through the pediatric quality measures program.
      The PQMP is tasked with increasing the “portfolio of evidence-based, consensus pediatric quality measures available to public and private purchasers of children's health care services, providers and consumers.”

      Children's Health Insurance Program Reauthorization Act of 2009. Available at: http://www.gpo.gov/fdsys/pkg/PLAW-111publ3/html/PLAW-111publ3.htm. Accessed June 17, 2014.

      The articles in this supplement examine the opportunities and challenges associated with the PQMP centers of excellence work focused on advancing the measurement science for pediatric quality measures. The articles in this supplement fall into 3 broad themes: the value of pediatric quality measures to stakeholders; the scope of the PQMP measurement initiative; and challenges in developing and testing pediatric quality measures.

      Value of Measurement in Pediatric Care

      The value of measuring health care quality is no longer debated. Measurement is a critical step toward achieving the triple aim of better care, better population health, and more affordable care.
      • Conway P.F.
      • Mostashari F.
      • Clancy C.
      The future of quality measurement for improvement and accountability.

      US Department of Health and Human Services. 2013 annual progress report to Congress: national strategy for quality improvement in health care. Available at: http://www.ahrq.gov/workingforquality/nqs/nqs2013annlrpt.htm. Accessed June 16, 2014.

      • Institute of Medicine
      Child and Adolescent Health and Health Care Quality: Measuring What Matters.
      This recognition of the value of measurement occurs at a time of great change and potential opportunity in children's health care. As the primary exemplar, the 2010 Patient Protection and Affordable Care Act (ACA) will expand coverage of children primarily through the expansion of Medicaid, health insurance exchanges, mandatory coverage, improvements in CHIP, and expansion of dependent coverage up to age 26, and also includes multiple opportunities and billions in funding for testing promising quality improvement strategies.

      Kaiser Family Foundation. Focus on health reform: summary of the Affordable Care Act. Available at: http://kaiserfamilyfoundation.files.wordpress.com/2011/04/8061-021.pdf. Accessed June 17, 2014.

      Centers for Medicare and Medicaid Services. The CMS innovation center. Available at: http://innovation.cms.gov. Accessed June 16, 2014.

      Patient Centered Outcomes Research Institute (PCORI). PCORnet: The National Patient-Centered Clinical Research Network. December 28, 2013. Available at: http://www.pcori.org/funding-opportunities/pcornet-national-patient-centered-clinical-research-network/. Accessed June 16, 2014.

      Additionally, CHIPRA provisions have established state-level quality improvement projects.
      • Dougherty D.
      • Conway P.
      The 3Ts roadmap to transform US health care: the “how” of high-quality care.
      Independent of federal legislation, quality improvement efforts are becoming more widely adopted by hospitals and insurers. The American Board of Pediatrics requires participation in quality improvement for its maintenance of certification program.
      • Miles P.V.
      • Moyer V.A.
      Quality improvement and maintenance of certification.
      Meaningful use of health information technology may further stimulate interest in and opportunities for pediatric performance measurement.
      A number of articles in this supplement highlight both the critical need for and promise of pediatric quality measurement from a variety of stakeholder perspectives. As the commentaries by Doetsch and Smith
      • Doetsch J.
      • Smith G.
      A CHIPRA quality demonstration grantee's perspective on the child core measure set and quality measurement.
      and by Bannister et al
      • Bannister L.
      • Singer J.A.
      • Lawthers A.G.
      A CHIPRA quality demonstration grantee's perspective on the CHIPRA core measures set: uses, challenges, and the future.
      report, states appreciate the national measurement effort because uniform reporting requirements improve the uptake of measures and help with the ability to use them to compare quality across states and across entities within states. Carolyn Allshouse of Family Voices reports that family advocates serve as key partners in the efforts to improve quality of care; she and other center authors document the important role that families play in the PQMP measurement development process.
      • Allshouse C.
      The importance of the family voice in quality measure development for children's health care.
      • Byron S.C.
      • Gardner W.
      • Kleinman L.C.
      • et al.
      Developing measures for pediatric quality: methods and experiences of the CHIPRA pediatric quality measures program grantees.
      As noted by Sachdeva et al
      • Sachdeva R.
      • McInerny T.
      • Perrin J.M.
      Quality measures and the practicing pediatrician: perspectives from the American Academy of Pediatrics.
      and by Miller,
      • Miller M.R.
      • Schwalenstocher E.
      • Weitcha M.
      Importance of the pediatric quality measurement program in advancing children's health care: a view from children's hospitals.
      pediatricians and children's hospitals are interested in clinically relevant measures as a part of the process to improve care; particularly measures that can serve multiple purposes such as quality improvement and accountability. Additionally, Silber and Forrest
      • Silber J.H.
      • Forrest C.B.
      Concept and measurement of pediatric value.
      add to the national conversation about assessing value by discussing the importance of both taking the child/family perspective into account and using appropriate methods of standardization in the analysis of data from quality measurement measures designed to help patients and families.

      Scope of PQMP Measurement Initiative

      The majority of articles in this supplement focus on specific measurement topics and illustrate the broad scope of the PQMP measurement effort, as shaped by the breadth of topics in the CHIPRA legislation and the priorities of the Centers for Medicare and Medicaid Services. Articles by Nakamura et al
      • Nakamura M.M.
      • Toomey S.L.
      • Zaslavsky A.M.
      • et al.
      An overview of pediatric hospital readmission rates as a quality of care measure.
      and Lorch et al
      • Lorch S.A.
      • Passarella M.
      • Zeigler A.
      Challenges to determining state-level variation in readmission rates of neonatal intensive care patients.
      are related to the controversial topic of measuring hospital readmissions in neonatal and general pediatric care. Several articles address development of measures related to care for chronic conditions that are both common, such as the article on attention-deficit/hyperactivity disorder outcomes by Woods et al,
      • Woods D.
      • Wolraich M.
      • Pierce K.
      • et al.
      Considerations and evidence for an ADHD outcome measure.
      and relatively low prevalence, such as measures of the quality of sickle cell care by Reeves et al
      • Reeves S.L.
      • Garcia E.
      • Kleyn M.
      • et al.
      Identifying sickle cell disease cases using administrative data.
      and the quality of use of antipsychotic drugs by Kealey et al.
      • Kealey E.
      • Scholle S.H.
      • Byron S.C.
      • et al.
      Quality concerns in antipsychotic prescribing for youth: a review of treatment guidelines.
      Cross-cutting articles by Byron and colleagues
      • Byron S.C.
      • Gardner W.
      • Kleinman L.C.
      • et al.
      Developing measures for pediatric quality: methods and experiences of the CHIPRA pediatric quality measures program grantees.
      and Mistry and colleagues
      • Mistry K.B.
      • Chesley F.
      • LLanos K.
      • et al.
      Advancing children's healthcare through the pediatric quality measures program.
      provide details about measures being developed on a wide range of additional topics across the centers.

      Challenges in Developing and Testing Pediatric Quality Measures

      A number of the topic-specific articles noted above, as well as the cross-topic articles in the supplement, detail the centers' experiences with various steps in the measure development process and challenges the centers encountered. As noted by Mistry et al
      • Mistry K.B.
      • Chesley F.
      • LLanos K.
      • et al.
      Advancing children's healthcare through the pediatric quality measures program.
      and Byron et al,
      • Byron S.C.
      • Gardner W.
      • Kleinman L.C.
      • et al.
      Developing measures for pediatric quality: methods and experiences of the CHIPRA pediatric quality measures program grantees.
      although measure topics were assigned to the centers of excellence, the assignments were often quite broad. Literature reviews can be helpful in narrowing a topic to specific measure foci, based on where the best evidence exists. Several of the articles include extensive reviews of the evidence.
      • Nakamura M.M.
      • Toomey S.L.
      • Zaslavsky A.M.
      • et al.
      An overview of pediatric hospital readmission rates as a quality of care measure.
      • Woods D.
      • Wolraich M.
      • Pierce K.
      • et al.
      Considerations and evidence for an ADHD outcome measure.
      • Kealey E.
      • Scholle S.H.
      • Byron S.C.
      • et al.
      Quality concerns in antipsychotic prescribing for youth: a review of treatment guidelines.
      After the focus of a measure has been identified, the measures must be clearly specified and subsequently (and iteratively) tested, typically with real data.
      • Byron S.C.
      • Gardner W.
      • Kleinman L.C.
      • et al.
      Developing measures for pediatric quality: methods and experiences of the CHIPRA pediatric quality measures program grantees.
      Several articles point out data-related challenges and associated implications for measure implementation. For example, Bannister and colleagues
      • Bannister L.
      • Singer J.A.
      • Lawthers A.G.
      A CHIPRA quality demonstration grantee's perspective on the CHIPRA core measures set: uses, challenges, and the future.
      point out that slight differences state-to-state in Medicaid administration or data collection can lead to differences in the implementation of measures and can reduce the comparability of measures. Gidengil and colleagues
      • Gidengil C.
      • Mangione-Smith R.
      • Bailey C.
      • et al.
      Using Medicaid and CHIP claims data to support pediatric quality measurement: lessons from three centers of excellence in measure development.
      examine issues relevant to Medicaid and CHIP claims data and recommend steps to make these systems more helpful to quality measurement. Some, but not all, challenges are remediable by measure developers. Reeves et al
      • Reeves S.L.
      • Garcia E.
      • Kleyn M.
      • et al.
      Identifying sickle cell disease cases using administrative data.
      relate an innovative approach to challenges in validating Medicaid claims data by using data from a state's newborn screening program. Looking to the future, Bailey and colleagues
      • Bailey C.
      • Tinoco A.
      • Mistry K.B.
      • et al.
      Addressing electronic clinical information in the construction of quality measures.
      discuss the advantages and disadvantages of using claims data and electronic medical records in their current state of development and address the challenges as well as the promise of future electronic clinical information for measurement.
      Bevans et al
      • Bevans K.
      • Moon J.
      • Carle A.C.
      • et al.
      Patient-reported outcomes as indicators of pediatric health care quality.
      report on the potential of Patient-Reported Outcome-Performance Measures (PRO-PMs) for making measurement more meaningful by integrating the patient perspective and engaging more families in the quality enterprise. Despite attention to these measures under the Patient Reported Outcome Measurement Information System (PROMIS), PRO-PMs have not been extensively incorporated into quality measurement sets thus far. As noted by Silber and Forrest,
      • Silber J.H.
      • Forrest C.B.
      Concept and measurement of pediatric value.
      prioritizing high-value outcomes that matter most to the patient (and family) may further support the use of PRO-PMs, and increased support may move the field toward resolution of technical issues such as the use of PRO-PMs in electronic health records.
      However, as Austin and colleagues
      • Austin J.M.
      • Miller M.R.
      • Pronovost P.J.
      Young and reckless? Greater standardization and transparency of performance is needed for pediatric performance measures.
      note in their commentary, more information about the performance of performance measures is a vital consideration for the future because it will guide the choices about which measures to use. In addition, Dougherty and colleagues
      • Dougherty D.
      • Mistry K.
      • Lindy O.
      • et al.
      A systematic evidence-informed quality measurement life-cycle approach to measure retirement in CHIPRA.
      discuss the role of periodic assessment of measures for possible retirement as part of a life-cycle approach to measurement that again highlights the need to continually evaluate use and implementation of measures over time.

      Conclusion

      The articles in this supplement are intended to be useful to a variety of readers, including individuals and organizations choosing measures to assess and monitor health care quality, individuals and systems whose care is being measured, future developers of quality measures, and child-focused clinical and health services researchers. As noted by the Institute of Medicine in 2005
      • Institute of Medicine
      Performance Measurement: Accelerating Improvement.
      and 2011,

      Kaiser Family Foundation. Focus on health reform: summary of the Affordable Care Act. Available at: http://kaiserfamilyfoundation.files.wordpress.com/2011/04/8061-021.pdf. Accessed June 17, 2014.

      and more recently by Berenson et al,

      Berenson RA, Pronovost PJ, Krumholz HM. Achieving the potential of health care performance measures. May 2013. Available at: http://www.rwjf.org/en/research-publications/find-rwjf-research/2013/05/achieving-the-potential-of-health-care-performance-measures.html. Accessed June 22, 2014.

      the high stakes often associated with quality measurement have intensified the need for transparent measure development and testing. Understanding the careful and challenging process of developing pediatric quality measures will help consumers and providers understand the inherent strengths and limitations of different measures and encourage the use of useful and comprehensive sets of measures to monitor improvements in quality and outcomes.
      The implementers of CHIPRA were specifically tasked with improving the quality and outcomes of care for children, with a special focus on the more than 42 million children enrolled in Medicaid and CHIP.

      Medicaid. FY 2010: number of children ever enrolled in Medicaid and CHIP. Available at: http://www.medicaid.gov/Medicaid-CHIP-Program-Information/By-Topics/Childrens-Health-Insurance-Program-CHIP/Downloads/2010_enrollment_datapdf.pdf. Accessed June 18, 2014.

      Improving the measurement of quality in pediatrics is an important step in the process of improving the care and well-being of US children. The articles in this supplement illustrate the considerations necessary for creating good measure sets and provide strategies for overcoming challenges encountered in the measurement development process.

      Acknowledgments

      A Pediatric Quality Measures Program grant ( U18HS20408 ) from the Agency for Healthcare Research and Quality supported Dr Forrest's work.

      References

      1. Children's Health Insurance Program Reauthorization Act of 2009. Available at: http://www.gpo.gov/fdsys/pkg/PLAW-111publ3/html/PLAW-111publ3.htm. Accessed June 17, 2014.

      2. Medicaid. 2014 core set of children's health care quality measures for Medicaid and CHIP. Available at: http://www.medicaid.gov/Medicaid-CHIP-Program-Information/By-Topics/Quality-of-Care/Downloads/ChildCoreMeasures.pdf. Accessed June 17, 2014.

        • Mistry K.B.
        • Chesley F.
        • LLanos K.
        • et al.
        Advancing children's healthcare through the pediatric quality measures program.
        Acad Pediatr. 2014; 14: s19-s26
        • Conway P.F.
        • Mostashari F.
        • Clancy C.
        The future of quality measurement for improvement and accountability.
        JAMA. 2013; 309: 2215-2216
      3. US Department of Health and Human Services. 2013 annual progress report to Congress: national strategy for quality improvement in health care. Available at: http://www.ahrq.gov/workingforquality/nqs/nqs2013annlrpt.htm. Accessed June 16, 2014.

        • Institute of Medicine
        Child and Adolescent Health and Health Care Quality: Measuring What Matters.
        National Academies Press, Washington, DC2011
      4. Kaiser Family Foundation. Focus on health reform: summary of the Affordable Care Act. Available at: http://kaiserfamilyfoundation.files.wordpress.com/2011/04/8061-021.pdf. Accessed June 17, 2014.

      5. Centers for Medicare and Medicaid Services. The CMS innovation center. Available at: http://innovation.cms.gov. Accessed June 16, 2014.

      6. Patient Centered Outcomes Research Institute (PCORI). PCORnet: The National Patient-Centered Clinical Research Network. December 28, 2013. Available at: http://www.pcori.org/funding-opportunities/pcornet-national-patient-centered-clinical-research-network/. Accessed June 16, 2014.

        • Dougherty D.
        • Conway P.
        The 3Ts roadmap to transform US health care: the “how” of high-quality care.
        JAMA. 2008; 299: 2319-2321
        • Miles P.V.
        • Moyer V.A.
        Quality improvement and maintenance of certification.
        Acad Pediatr. 2013; 13: S14-S15
        • Doetsch J.
        • Smith G.
        A CHIPRA quality demonstration grantee's perspective on the child core measure set and quality measurement.
        Acad Pediatr. 2014; 14: s4-s5
        • Bannister L.
        • Singer J.A.
        • Lawthers A.G.
        A CHIPRA quality demonstration grantee's perspective on the CHIPRA core measures set: uses, challenges, and the future.
        Acad Pediatr. 2014; 14: s6-s7
        • Allshouse C.
        The importance of the family voice in quality measure development for children's health care.
        Acad Pediatr. 2014; 14: s8-s9
        • Byron S.C.
        • Gardner W.
        • Kleinman L.C.
        • et al.
        Developing measures for pediatric quality: methods and experiences of the CHIPRA pediatric quality measures program grantees.
        Acad Pediatr. 2014; 14: s27-s32
        • Sachdeva R.
        • McInerny T.
        • Perrin J.M.
        Quality measures and the practicing pediatrician: perspectives from the American Academy of Pediatrics.
        Acad Pediatr. 2014; 14: s10-s11
        • Miller M.R.
        • Schwalenstocher E.
        • Weitcha M.
        Importance of the pediatric quality measurement program in advancing children's health care: a view from children's hospitals.
        Acad Pediatr. 2014; 14: s12-s14
        • Silber J.H.
        • Forrest C.B.
        Concept and measurement of pediatric value.
        Acad Pediatr. 2014; 14: s33-s38
        • Nakamura M.M.
        • Toomey S.L.
        • Zaslavsky A.M.
        • et al.
        An overview of pediatric hospital readmission rates as a quality of care measure.
        Acad Pediatr. 2014; 14: s39-s46
        • Lorch S.A.
        • Passarella M.
        • Zeigler A.
        Challenges to determining state-level variation in readmission rates of neonatal intensive care patients.
        Acad Pediatr. 2014; 14: s47-s53
        • Woods D.
        • Wolraich M.
        • Pierce K.
        • et al.
        Considerations and evidence for an ADHD outcome measure.
        Acad Pediatr. 2014; 14: s54-s60
        • Reeves S.L.
        • Garcia E.
        • Kleyn M.
        • et al.
        Identifying sickle cell disease cases using administrative data.
        Acad Pediatr. 2014; 14: S61-S67
        • Kealey E.
        • Scholle S.H.
        • Byron S.C.
        • et al.
        Quality concerns in antipsychotic prescribing for youth: a review of treatment guidelines.
        Acad Pediatr. 2014; 14: s68-s75
        • Gidengil C.
        • Mangione-Smith R.
        • Bailey C.
        • et al.
        Using Medicaid and CHIP claims data to support pediatric quality measurement: lessons from three centers of excellence in measure development.
        Acad Pediatr. 2014; 14: s76-s81
        • Bailey C.
        • Tinoco A.
        • Mistry K.B.
        • et al.
        Addressing electronic clinical information in the construction of quality measures.
        Acad Pediatr. 2014; 14: s82-s89
        • Bevans K.
        • Moon J.
        • Carle A.C.
        • et al.
        Patient-reported outcomes as indicators of pediatric health care quality.
        Acad Pediatr. 2014; 14: s90-s96
        • Austin J.M.
        • Miller M.R.
        • Pronovost P.J.
        Young and reckless? Greater standardization and transparency of performance is needed for pediatric performance measures.
        Acad Pediatr. 2014; 14: s15-s16
        • Dougherty D.
        • Mistry K.
        • Lindy O.
        • et al.
        A systematic evidence-informed quality measurement life-cycle approach to measure retirement in CHIPRA.
        Acad Pediatr. 2014; 14: s97-s104
        • Institute of Medicine
        Performance Measurement: Accelerating Improvement.
        National Academies Press, Washington, DC2005 (Available at:) (Accessed June 22, 2014)
      7. Berenson RA, Pronovost PJ, Krumholz HM. Achieving the potential of health care performance measures. May 2013. Available at: http://www.rwjf.org/en/research-publications/find-rwjf-research/2013/05/achieving-the-potential-of-health-care-performance-measures.html. Accessed June 22, 2014.

      8. Medicaid. FY 2010: number of children ever enrolled in Medicaid and CHIP. Available at: http://www.medicaid.gov/Medicaid-CHIP-Program-Information/By-Topics/Childrens-Health-Insurance-Program-CHIP/Downloads/2010_enrollment_datapdf.pdf. Accessed June 18, 2014.