The University of Florida Child Health Quality (CHeQ) initiative, funded by the Agency for Health Care Research and Quality (AHRQ)/Centers for Medicare and Medicaid Services (CMS) Pediatric Quality Measurement Program, examined measures that states use to evaluate quality of oral health care for children in Medicaid and the Children's Health Insurance Program (CHIP). The Core Set of Children's Health Care Quality Measures for Medicaid and CHIP (Child Core Set) is intended to serve as a set of measures which, taken together, can be used to estimate the “overall national quality of health care for Medicaid and CHIP beneficiaries.”
The Core Sets are comprised of quality measures collected at the state level. The Dental Quality Alliance (DQA) “Dental Sealants for 6 to 9-Year-Old Children at Elevated Caries Risk (SEAL-CH)”
was added to the Child Core Set in 2015. This sealant measure was added to the Child Core Set Oral Health domain to improve dental outcomes for children covered by Medicaid and CHIP. There is clear evidence that sealants reduce dental caries. Measures on the Child Core Set and other pediatric measures that are meaningfully designed, assessed, and tested are central to quality improvement efforts. A key element of the Pediatric Quality Measurement Program CHeQ project was to evaluate the existing oral health measures as part of a broader quality measure development lifecycle process. This commentary discusses the process and outcomes related to the critical “feedback” stage of the lifecycle in which, after implementation, stakeholders such as researchers, dental practitioners, clinicians, health system leaders, patients and policymakers, evaluate the measure to determine the usefulness and validity in real-world settings. To be successful, the feedback process must engage a multistakeholder group that can bridge research, policy, and patient experiences with care to improve the measure and outline future areas of work.
The CHeQ initiative
- Shenkman E
- Tomar S
- Manning D
- et al.
Feasibility and usability of measuring receipt of sealants in 2 states.
prioritized feedback from key stakeholders. The CHeQ research, along with feedback from measure implementers and DQA workgroup efforts, re-examined the validity and usability of the SEAL-CH measure. The CHeQ research, combined with input from other stakeholders that the DQA contacted, resulted in the development of an improved sealant measure “Sealant Receipt on Permanent First Molars.” The new DQA sealant measure is defined as the percentage of enrolled children who have ever received sealants on permanent first molar teeth: 1) at least one sealant and 2) all 4 molars sealed by the 10th birthdate.
Based on recommendation of the Annual Core Set Review Workgroup final report, CMS removed the SEAL-CH measure and added, SFM-CH, in the 2021 Child Core Set.
Stakeholder Engagement Is A Catalyst For Change
Stakeholder engagement serves as a central component to measure development, implementation, and assessment. Once an organization develops and implements a measure, valuable information and feedback emerges. It is important that the measure development team critically evaluate the feedback in a manner that balances considerations from clinical practice, research, and policy perspectives. The selection of stakeholders representing all perspectives is critical to obtaining balanced feedback to achieve an optimal health care quality measure that is at the intersection of reliability, validity, usability, and feasibility.
The CHeQ initiative's research on the SEAL-CH measure used an established, long-standing collaboration with the Texas Health and Human Services Commission, which oversees the state Medicaid Program. Texas’ Medicaid Program has a value-based purchasing initiative, which began in 2014 and is focused on optimizing health outcomes and patient experience while increasing efficiency., ,
Ongoing stakeholder engagement with Medicaid managed care plans, dental maintenance organizations (DMOs), clinicians, community members, and policy makers is integral to the Texas Health and Human Services Commission efforts. In 2017, the CHeQ team invited the DQA and the American Dental Association (ADA) to have representatives join its oral health meetings. The DQA and ADA representatives were invited because they worked together, along with researchers at the University of Florida to develop the SEAL-CH Measure.
- Herndon JB
- Tomar SL
- Catalanotto FA
- et al.
Measuring quality of dental care: caries prevention services for children.
Following its development, the DQA and ADA continued to monitor the use of the measure and identify opportunities for improvement.
At the same time that the CHeQ team began holding its oral health meetings, the DQA formed a workgroup comprised of dental practitioners, Medicaid DMOs, researchers, Texas Medicaid staff, and AHRQ and CMS representatives. The workgroup's purpose was to discuss concerns about the SEAL-CH measure, its intended use, measure validity, feasibility in reporting, and strategies for measure improvement. In addition to the workgroup, CMS and AHRQ held a series of conference calls and webinars with the CHeQ team and other stakeholders to discuss gaps in the SEAL measure and possible alternatives to address those gaps. These efforts reinforced the critical role of continuously seeking feedback.
Using Texas and Florida Medicaid claims data, the CHeQ team assessed the reliability, validity, usability, and feasibility of the SEAL measure. The published results
are briefly summarized in this paragraph. The SEAL-CH measure incorporated the concept of elevated caries risk in its specifications, thus requiring a method to identify elevated risk. However, the measure does not specify which approach or approaches should be used to identify elevated risk leading to variability in how children are identified for inclusion in the measure. In examining Texas and Florida Medicaid, we found that one state, Florida, primarily used claims data combined with a 3-year look-back period; the other state, Texas, used risk codes. Our data analysis revealed that the different ways of measuring risk in the 2 states identify different children, raising questions about the reliability of using different methods to assess caries risk within one measure.
Through CHeQ engagement, Dental Directors and Quality Improvement Directors from 2 Medicaid DMOs—DentaQuest and MCNA Dental—identified further concerns with the SEAL-CH measure. DentaQuest and MCNA Dental also worked closely with the DQA and with state Medicaid agencies to convey the concerns they identified, which included: 1) the reliability of caries risk assessment information given the previously described variability in measuring risk, 2) variations in dental coding practices from state to state, 3) the validity of the reported rate given that it only examined sealant placement within the reporting year, and 4) the feasibility of identifying, and inability to subsequently exclude, children from the measure due to clinical contraindication for sealants (already sealed, not yet erupted, extracted, carious, or restored teeth). The DMOs also raised concerns about the overall use of the measure given concerns raised about reliability and validity. Further, the DMOs raised concerns about using the measure in the context of value-based purchasing efforts given its previously described limitations. The DQA indicated the intended use of the measure was to track population trends in the receipt of sealants. However, regardless of the intended use, reliability and validity is foundational for any measure.
In February 2019, the DQA asked the CHeQ team to provide written feedback on proposed changes to the SEAL-CH measure. In June 2019, the DQA issued a final report that contained the specifications for the new SFM-CH measure that incorporated the CHeQ oral health group and other stakeholder recommendations. The new measure addressed the reliability and variability concerns with the previous measure by eliminating the risk criteria in accordance with the most current ADA and AAPD guidelines for sealant placement. The new measure also addressed validity of the reported rate and the feasibility concerns of identifying and removing exclusions by expanding the reporting period to include a 3-year look-back period for sealant application prior to the child's tenth birthday. The report noted that the revised measure had limitations. These limitations include an inability to identify children with active decay on first permanent molars, which would not be eligible for sealants; and children who obtained sealants prior to Medicaid enrollment. As described in the following section, dental care measures may benefit from current developments and advances in linking data longitudinally and across multiple sources including dental clinical and claims data.
Consideration For The Future Of Pediatric Dental Care Measurement
As the quality measure lifecycle process requires that the new SFM-CH undergo evaluation and feedback to improve the measure and determine if it is both feasible and reliable, assessment of the new measure to inform ongoing measure refinement should continue to incorporate stakeholder-engaged approaches, for which there is already a strong foundation. As the field advances, important challenges remain such as the concerns around administrative health data not being sufficient for development of valid measures. Electronic medical and dental records would better link physical and oral health clinical data with claims data. Further, enhanced data linkage is important to understand care that children may have received prior to their Medicaid enrollment or during gaps in enrollment. Such linkage of data could enhance development of measures with greater validity, usability, and feasibility. Buy-in and feedback from stakeholders (eg, state Medicaid, dental and health plans, providers, professional organizations, and consumers) are integral to catalyze and sustain changes and drive innovations. Involving the right stakeholders is key to achieve a balance between clinical practice, research, and policy needs. This balance includes the assessment of cost and burden of measure calculation using a claims-based approach relative to a hybrid approach that incorporates dental records.
A stakeholder-engaged approach informed our work with the SEAL-CH measure. This approach bridged research and policy, starting with identification of the topic from DMOs through the design and conduct of the study. We conducted the research in the broader context of working with DQA on the Sealant Measure Workgroup and including DQA and ADA representatives on the CHeQ oral health team. Participation in the DQA Sealant Work Group contributed to the development of the new dental sealant measure. The development of a new, improved measure is an important step. Perhaps what is more critical is that stakeholders must continue to refine this measure and develop other oral health care measures for children which are robust and meaningful for policy makers, research scientists, dental practitioners, patients, and families.
We would like to thank Dr Robyn Sagatov for her thoughtful feedback and support finalizing the manuscript.
Financial statement: This study was conducted under a grant from the Agency for Healthcare Research and Quality (grant # U18 HS025298 ).
Disclaimer: The views expressed in this article are those of the authors, and no official endorsement by the Agency for Healthcare Research and Quality, Centers for Medicare and Medicaid Services, or the Department of Health and Human Services is intended or should be inferred.
This article is published as part of a supplement sponsored by the US Department of Health and Human Services, the Centers for Medicare and Medicaid Services, and the Agency for Healthcare Research and Quality.
The authors have no conflicts of interest to disclose.
© 2021 Published by Elsevier Inc. on behalf of Academic Pediatric Association