A CHIPRA Quality Demonstration Grantee's Perspective on the CHIPRA Core Measures Set: Uses, Challenges, and the Future

      The Center for Medicare and Medicaid Services (CMS) published the initial set of Children's Health Insurance Program Reauthorization Act of 2009 (CHIPRA) core pediatric quality measures in December 2009, signaling to states its priorities in the arena of child health care quality. The challenge undertaken by CMS and the Subcommittee of the National Advisory Committee (SNAC) in creating the initial set was enormous: how to select a reasonable set of measures that would simultaneously represent multiple important dimensions of quality, include metrics of processes and outcomes, and be reasonable for states to collect? The initial set of 24 measures represented a balanced and thoughtful collection of metrics.
      How have states responded to the core set? What activities have been stimulated and what results achieved? This commentary reflects on Massachusetts' experience with the CHIPRA core measures set over the past 4 years.

      Using the CHIPRA Core Measures Set

      Since 1996, Massachusetts has used measurement to drive quality improvement by requiring contracted managed care plans to collect and report on a limited set of Healthcare Effectiveness Data and Information Set measures, always including measures relevant to the pediatric population. The advent of the CHIPRA core set meant MassHealth was able to explicitly align agency priorities, and those of its contracted managed care plans, with CMS priorities. Currently, MassHealth requires its managed care plans to develop and implement activities related to 8 of the CHIPRA core measures and to periodically report on the status of these activities.
      The publication of the CHIPRA core set presented MassHealth with new opportunities to use measurement to transform health care, opportunities well beyond the traditional purposes of monitoring and improving quality. MassHealth's new health system reform initiative, Primary Care Payment Reform (PCPR), uses measurement for accountability and draws from the CHIPRA measure set to populate the quality measure bundles defining how practices will be paid. Under PCPR, in addition to a capitation, practices will receive payments on the basis of quality and will share in any savings resulting from more efficient care delivery. Twenty-three quality metrics comprise the PCPR quality bundle, including 8 CHIPRA measures.
      However, perhaps most importantly, when MassHealth enters discussions about aligning quality metrics across payers in Massachusetts, as it did with the legislatively mandated Statewide Quality Advisory Committee (SQAC), it has an influential card in its back pocket: the CHIPRA core measures set. The existence of the core set, along with its clear commitment to measures relevant and important to the pediatric population, was a key factor in ensuring that measures of child health and health care figure prominently in the statewide quality measure set released by the SQAC.

      Statewide Quality Advisory Committee. Statewide quality measure set. Available at: Accessed October 10, 2013.

      It's Not All Roses: Challenges Remain

      Although Massachusetts reaps the benefits of a core set of pediatric quality measures, challenges persist in using the measures to drive improvement and support purchase of high-value care. For example, producing accurate performance results from available data remains problematic, as does finding metrics to address priority areas not currently included in the core measures set.
      The Massachusetts CHIPRA quality demonstration grant team encountered several snags when calculating results for the initial set of core measures. First, we found that specifications were often not detailed enough to use on claims and encounter data, and they needed to be refined to ensure accurate measurement of the process or outcome of interest. We also found that it was critical to have staff experienced in using Medicaid data sets to assist with mapping the CMS specifications to these data sources. Given that Medicaid claims and encounter databases are not uniform across states, it is likely that other states are similarly modifying specifications to meet their individual needs, resulting in performance results that may not be comparable across states.
      The core measures also rely on data sources other than claims and encounters, including medical record–derived data and patient experience surveys. Undertaking medical record reviews and administering patient surveys is resource intensive and costly, especially if the state is looking to measure performance at a practice level, as this greatly increases the number of records that must be reviewed and surveys administered to achieve a reliable result. However, to support quality improvement at the health system level, practice-level granularity is clearly critical. This allows states to understand where the best practices are and to identify the most impactful opportunities for improvement.
      In addition to challenges in calculating measure results, the current measure set does not span the full scope of clinical areas that a state or other stakeholders may be interested in. For example, we measure whether young children are screened for developmental concerns but not the rate at which children identified with developmental concerns were seen for follow-up, nor whether the referral loop between the specialist and the primary care provider was closed.

      Pediatric Quality Measures Program Offers Hope for Maintaining the Measurement Motion

      The Pediatric Quality Measures Program's Centers of Excellence (CoEs) are working on addressing these and other challenges in using the CHIPRA core measures set to drive improvements in pediatric health care and to keep children's health issues in the forefront of discussions on quality, cost, and value.
      The CoEs are developing and testing new measures on such things as follow-up after developmental screening and identification of elevated body mass index, as well as measures looking at care coordination. Having these measures will help states implementing alternative payment methodologies assess how well medical homes and neighborhoods are coordinating and communicating between and among team members. Work on improving measures in the current set is also underway, with CoEs looking at ways to make existing measures more amenable to use in quality improvement efforts. Finally, the CoEs are thinking about specifying measures for use with electronic health records. As the use of electronic health records expands, such specifications will allow for more timely and efficient data collection, allowing more resources to be dedicated to improvement-focused activities.


      Funded by the Centers for Medicare and Medicaid Services through grant funds issued pursuant to CHIPRA section 401(d).


      1. Statewide Quality Advisory Committee. Statewide quality measure set. Available at: Accessed October 10, 2013.