Advertisement
Articles–State Profiles, Duration of Coverage, Availability of Services, Quality Measures, Measuring Family Experiences of Care, State Quality Measure Needs, Reporting Quality| Volume 11, ISSUE 3, SUPPLEMENT , S68-S76, May 01, 2011

Medicaid and CHIP Children’s Healthcare Quality Measures: What States Use and What They Want

      Abstract

      Objective

      The objective of this research was to explore state Medicaid and Children’s Health Insurance Program (CHIP) use of children’s healthcare quality measures and the need for additional support as the Children’s Health Insurance Program Reauthorization Act (CHIPRA) legislation is being implemented.

      Methods

      This summary analysis draws from a December 2008 survey of state CHIP programs fielded by the National Academy for State Health Policy (NASHP) and a February 2009 survey of Medicaid and CHIP programs conducted by Health Management Associates (HMA).

      Results

      Nearly all (90%) Medicaid and CHIP directors rate children’s health care quality as a high priority (7 or above on a 10-point scale). Almost all state Medicaid programs collect performance measures from plans or providers and almost all CHIP programs collect pediatric performance measures from managed care organizations, although significantly fewer collect such data from fee-for-service providers. Most state programs also collect encounter data, perform enrollee surveys, and utilize medical record reviews. Fewer track duration of enrollment in coverage or analyze measures for disparities by race and ethnicity. Almost all state programs feel that their quality improvement efforts have led to positive changes. Less than one third believe current measures are adequate, almost two thirds seek better measures of care coordination, and roughly half agreed better outcome measures are needed. Many state programs supplement existing standard measures with their own state-developed measures.

      Conclusion

      The surveys’ findings of high levels of state children’s health care quality measurement and improvement activity and interest suggest that new federal CHIPRA provisions are coming at an opportune time. To achieve significant state participation in measuring and reporting on children’s access to quality care, national programs need to be guided not only by national priorities but by state goals, capacity, and practice.

      Keywords

      In 1997, the state Children’s Health Insurance Program (CHIP) was enacted to expand health insurance for low-income children, and states have achieved substantial success in achieving the legislation’s primary goal. Over 7 million children were enrolled in CHIP in 2008,

      Centers for Medicare & Medicaid Services. FY 2008 number of children ever enrolled year–SCHIP by program type. Available at: http://www.cms.hhs.gov/NationalCHIPPolicy/downloads/FY2008StateTotalTable012309FINAL.pdf. Accessed January 12, 2010.

      and that number is projected to double by the year 2013.

      Congressional Budget Office. Spending and enrollment detail for CBO’s March 2009 baseline: Children’s Health Insurance Program (CHIP). Available at: http://www.cbo.gov/budget/factsheets/2009b/chip.pdf. Accessed January 12, 2010.

      Although progress also has been made in measuring and improving quality of health care,
      • National Academy for State Health Policy
      SCHIP at 10: Progress and Results.
      this focus was less explicit in the 1997 federal legislation, and attention to quality of care has been secondary in many states’ efforts.
      In 2002, the Centers for Medicare & Medicaid Services established 4 widely used pediatric primary and preventive health care measures—the number of children and adolescents who saw a primary care provider during the annual reporting period, reported by 4 age groups; number of well-child visits in the first 15 months of life; annual well-child visits for ages 3 through 6; and use of appropriate medications for children with persistent asthma, reported by 3 age groups—drawn from the National Committee for Quality Assurance Healthcare Effectiveness Data and Information Set (HEDIS), for voluntary state reporting in annual CHIP reports. Although these measures provided a useful starting point for assessing quality of care for CHIP enrollees, state capacities for measurement, technical issues, and other factors have limited the completeness of state reporting.
      • Partridge L.
      Review of Access and Quality of Care in SCHIP Using Standardized National Performance Measures.
      • Duchon L.
      • Smith V.
      Quality Performance Measurement in Medicaid and SCHIP: Results of a 2006 National Survey of State Officials.
      Additionally, these measures are limited in scope and their consequent ability to support more robust quality assessment and improvement initiatives at the state or national level.
      The Children’s Health Insurance Program Reauthorization Act (CHIPRA) represents landmark legislation for children’s health care quality, requiring 1) that core pediatric measures be identified and then improved for voluntary use initially by both Medicaid and CHIP programs and later by all public and private health care delivery programs; 2) that multiple stakeholders, including state Medicaid and CHIP programs, be engaged in the identification and improvement of measures; and 3) that the measures be evidence-based and able to identify disparities by race, ethnicity, socioeconomic status, and special health care needs.

      Children’s Health Insurance Program Reauthorization Act of 2009, title IV. Available at: http://frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname=111_cong_public_laws&docid=f:publ003.111.pdf%20. Accessed March 26, 2010.

      To meet these requirements, the Centers for Medicare & Medicaid Services partnered with the Agency for Healthcare Research and Quality (AHRQ) to capitalize on AHRQ technical expertise in health care quality measurement. The AHRQ National Advisory Council on Healthcare Research and Quality created the Subcommittee on Children’s Healthcare Quality Measures for Medicaid and CHIP Programs (SNAC), to recommend an initial core set of children’s health care quality measures.
      • Mangione-Smith R.
      • Schiff J.
      • Dougherty D.
      Identifying children's health care quality measures for medicaid and CHIP: an evidence-informed, publicly transparent expert process.
      The SNAC included leaders from Medicaid and CHIP programs, in addition to other experts and stakeholders. To complement the expertise of the Medicaid and CHIP representatives on the Subcommittee, AHRQ and Centers for Medicare & Medicaid Services commissioned analyses of 2 recent surveys of Medicaid and CHIP programs performed by private sector organizations, the National Academy for State Health Policy (NASHP) and Health Management Associates (HMA), both of which included questions about children’s health care quality measurement.

      Hess C, Farrell K, deLone S, et al. Charting CHIP IV: A Report on State Children's Health Insurance Programs Prior to Major Federal Policy Changes in 2009 and 2010. Washington, DC: National Academy for State Health Policy. In press.

      • Smith V.
      • Edwards J.
      • Reagan E.
      • Roberts D.
      Medicaid and CHIP Strategies for Improving Child Health.

      Methods

      Responding to guidance from AHRQ on specific issues relevant to CHIPRA and the Subcommittee deliberations, we conducted analyses of the aforementioned recent surveys by NASHP and HMA.

      Overview of Medicaid and CHIP Programs

      Medicaid is a federal and state-funded program, administered by states, to provide health care coverage to low-income individuals, including children. CHIP builds on Medicaid’s coverage of children. States can use federal CHIP funds to expand coverage of children under Medicaid, to establish programs separate from and subject to different federal rules than Medicaid, or to adopt both approaches. CHIP programs involving an expansion of Medicaid coverage are referred to as “M-CHIP” and separate programs as “S-CHIP.” Forty-four states operate S-CHIP programs, excluding 4 S-CHIP programs that cover only unborn children. Thirty-three states and the District of Columbia operate M-CHIP programs. Policies governing M-CHIP programs are the same as those governing Medicaid, the only difference involving the availability of the enhanced federal matching rate and source of federal funds used to cover enrolled children.

      Overview of Surveys

      Both surveys, sent electronically to all 50 states and the District of Columbia, posed multiple-choice questions, with space for additional comment where appropriate. Funded by the David and Lucile Packard Foundation, the NASHP survey covered multiple aspects of state CHIP policies and operations, including quality and performance measurement, in order to provide states, federal policy makers, researchers, and other stakeholders with information about the policy and program choices made by states in implementing CHIP. Separate survey instruments, each containing over 140 questions, including 9 probing information directly relevant to this article, were developed for each program type and reviewed by 3 CHIP directors prior to finalization. In December 2008, states operating either or both an S-CHIP or M-CHIP program were asked to complete both surveys. Supported by the Commonwealth Fund, the HMA survey, containing 15 questions of which 8 elicited information directly relevant to this article, was sent in February 2009 to all state Medicaid directors and to directors of all S-CHIP programs to identify state children’s health care quality strategies and needs in Medicaid and CHIP.

      Measures for Topics of Particular Relevance to CHIPRA and SNAC Deliberations

      Ratings of Quality as a Priority

      The HMA survey asked about the priority of quality relative to other Medicaid and CHIP concerns.

      State Medicaid and CHIP Data Collection and Performance Measurement

      Both surveys sought information regarding state collection and use of nationally standardized performance measures, state-defined performance measures, and enrollee surveys. Some NASHP, although no HMA, survey questions required states to distinguish between quality measurement involving providers reimbursed on a fee-for-service basis versus health plans paid to provide a comprehensive package of benefits to program enrollees. (The term comprehensive health plan rather than managed care organization was used to distinguish state experience with managed care organizations providing a comprehensive set of benefits versus those providing a limited set of benefits. In this article, the term managed care organization refers to plans providing a comprehensive benefit package.)
      The NASHP survey also asked state programs about the collection and use of encounter data; performance of medical record reviews; collection of data relating to the duration of coverage once children are enrolled; the adequacy of the primary care and specialty provider capacity for their programs; and use of data collected to evaluate disparities in access to care or treatment based on race or ethnicity. The HMA survey asked surveyed state officials about the priority of children’s health care quality for Medicaid and CHIP.

      Duration of Enrollment and Coverage

      CHIPRA requires that the initial core set of children’s health care quality measures include the “duration of children’s health insurance coverage over a 12-month period.” The NASHP survey asked state CHIP programs about 3 types of data related to this concept: 1) duration of coverage once children are enrolled, 2) retention of children at renewal, and 3) the extent to which children “churn” off and back onto coverage.

      Provider Availability

      NASHP asked both S-CHIP and M-CHIP programs for their perspectives on whether “the primary and/or specialty provider capacity for their program is adequate.”

      Racial and Ethnic Disparities

      CHIPRA specifically calls for inclusion of measures in the core set that can be used to analyze racial, ethnic, and socioeconomic disparities in child health and health care for children. The NASHP survey asked respondents whether they use any of the data collected to evaluate disparities in access to care or treatment based on race or ethnicity.

      Challenges and Needs

      The HMA survey sought information on the challenges states face in improving children’s health care quality in their programs, as well as what would most help them achieve their goals. Questions probed the adequacy of measures currently available, new measures states would find valuable and ways in which current measures could be improved, and technical assistance needs.

      Results

      Thirty-five S-CHIP (88%) and 27 M-CHIP (79%) programs from 46 states and the District of Columbia responded to the NASHP survey. Forty-two Medicaid programs (82%) and 11 S-CHIP programs (28%) from 44 states responded to the HMA survey. Results from the NASHP and HMA surveys are presented for the following areas relevant to the objective of informing development of the new core set of children’s health care quality measures: state data collection and performance measurement, duration of enrollment and retention, provider availability, racial and ethnic disparities, and state challenges and needs in measuring children’s health care quality. Because of the relatively low response rate of S-CHIP programs in the HMA survey, we have not included data from the HMA survey specific to state S-CHIP programs except as expressly noted. Thus, data presented for S-CHIP and M-CHIP programs derive from the NASHP survey. Data presented for state Medicaid programs derive from the HMA survey.

      State Medicaid and CHIP Data Collection and Performance Measurement

      Ratings of Quality as a Priority

      Medicaid and CHIP officials rate children’s health care quality as a top children’s health priority; 90% of directors from both programs responding to the HMA survey rated the priority of children’s health care quality in their states as 7 or higher on a 10-point scale. Almost 60% rated quality as a 9 or 10 in priority.

      Current Collection of Health Care Quality Performance Measurement Data

      Consistent with the finding of the priority assigned to children’s health care quality, both the NASHP and HMA surveys found that most state Medicaid and CHIP programs collect quality performance measurement data. As shown in Table 1, 94% of S-CHIP programs and 85% of M-CHIP programs collect performance measures; 98% of Medicaid programs do so. Across both S-CHIP and M-CHIP programs, 97% of CHIP programs using managed care organizations collect performance measures from these plans; only 59% of CHIP programs using fee-for-service providers collect performance measures from these providers.
      Table 1State Collection of Performance Measures and Other Data in Medicaid and CHIP Programs
      Data source is the Fourth Comprehensive Survey of Children’s Health Insurance Programs, conducted by the National Academy for State Health Policy in 2008, and the Survey of Medicaid Programs on Quality Measures and Improvement Incentives in Medicaid and SCHIP, conducted by Health Management Associates in 2009. Ellipses indicate that the information reported was not requested in the survey. CHIP = Children’s Health Insurance Program.
      Program TypeSource of DataNumber of States or Programs RespondingPrograms Collecting Performance Measures % (No.)Programs Performing Enrollee Surveys % (No.)Programs Collecting Encounter Data % (No.)Programs Performing Medical Record Reviews % (No.)
      TotalHEDIS
      The denominator used to determine the percentage of states or programs collecting each type of performance measure is the total number of states or programs responding to the survey, except that in determining the percentage of states or programs using HEDIS, state-defined and NQF-endorsed performance measures, the denominator is the total number of states or programs collecting performance measures.
      HEDIS + State Defined
      The denominator used to determine the percentage of states or programs collecting each type of performance measure is the total number of states or programs responding to the survey, except that in determining the percentage of states or programs using HEDIS, state-defined and NQF-endorsed performance measures, the denominator is the total number of states or programs collecting performance measures.
      State Defined Only
      The denominator used to determine the percentage of states or programs collecting each type of performance measure is the total number of states or programs responding to the survey, except that in determining the percentage of states or programs using HEDIS, state-defined and NQF-endorsed performance measures, the denominator is the total number of states or programs collecting performance measures.
      NQF-Endorsed
      The denominator used to determine the percentage of states or programs collecting each type of performance measure is the total number of states or programs responding to the survey, except that in determining the percentage of states or programs using HEDIS, state-defined and NQF-endorsed performance measures, the denominator is the total number of states or programs collecting performance measures.
      TotalCAHPS
      CAHPS = Consumer Assessment of Healthcare Providers and Systems.
      State Medicaid programsHMA
      HMA = Health Management Associates.
      survey
      4298 (41)90 (37)76 (31)10 (4)17 (7)71 (30)
      States (with either or both CHIP programs)NASHP
      NASHP = National Academy for State Health Policy.
      survey
      4693 (43)98 (42)53 (23)7 (3)91 (42)79 (33)85 (39)63 (29)
      M-CHIP programs
      M-CHIP = CHIP programs that involve an expansion of Medicaid coverage.
      NASHP survey2785 (23)96 (22)56 (13)4 (1)81 (22)86 (19)81 (22)63 (17)
      S-CHIP programs
      S-CHIP = a separate CHIP program.
      NASHP survey3594 (33)94 (31)45 (15)6 (2)89 (31)68 (21)83 (29)60 (21)
      All CHIP programs using MCOs
      MCO = managed care organization.
      NASHP survey3697 (35)91 (32)49 (17)6 (2)94 (34)82 (28)92 (33)67 (24)
      All CHIP programs using FFS
      FFS = fee for service.
      providers
      NASHP survey3959 (23)74 (17)30 (7)26 (6)46 (18)72 (13)49 (19)26 (10)
      Data source is the Fourth Comprehensive Survey of Children’s Health Insurance Programs, conducted by the National Academy for State Health Policy in 2008, and the Survey of Medicaid Programs on Quality Measures and Improvement Incentives in Medicaid and SCHIP, conducted by Health Management Associates in 2009. Ellipses indicate that the information reported was not requested in the survey. CHIP = Children’s Health Insurance Program.
      The denominator used to determine the percentage of states or programs collecting each type of performance measure is the total number of states or programs responding to the survey, except that in determining the percentage of states or programs using HEDIS, state-defined and NQF-endorsed performance measures, the denominator is the total number of states or programs collecting performance measures.
      CAHPS = Consumer Assessment of Healthcare Providers and Systems.
      § HMA = Health Management Associates.
      NASHP = National Academy for State Health Policy.
      M-CHIP = CHIP programs that involve an expansion of Medicaid coverage.
      # S-CHIP = a separate CHIP program.
      ∗∗ MCO = managed care organization.
      †† FFS = fee for service.
      Shown in Table 2, of the programs that collect performance measures, 82% of the S-CHIP and 83% of the M-CHIP programs use performance measures for quality studies; 67% of Medicaid programs use quality and performance data to report on health plan performance; 52% of Medicaid programs publicly report on health plan or individual provider performance; and 43% of Medicaid programs award financial incentives for exceeding specified quality benchmarks. Smaller percentages impose financial penalties for failure to meet performance targets or base auto enrollment in plans on quality performance.
      Table 2Use of Performance and Quality Data in Medicaid and CHIP Programs
      Data Source is the Fourth Comprehensive Survey of Children's Health Insurance Programs, conducted by the National Academy for State Health Policy in 2008, and the Survey of Medicaid Programs on Quality Measures and Improvement Incentives in Medicaid and SCHIP, conducted by Health Management Associates in 2009. CHIP = Children's Health Insurance Program.
      Program TypeSource of DataNumber of Responding ProgramsUse of Performance Measures by Medicaid Programs, Percentage of Responding Programs
      Report on Health Plan PerformancePublicly Report on Health Plan PerformanceFinancial Incentives for Exceeding Quality BenchmarksFinancial Penalties for Failing to Meet Performance TargetsBase Auto Enrollment in Plans on Quality Performance
      MedicaidHMA
      HMA = Health Management Associates.
      426752432117
      Use of Quality Data by CHIP Programs for Quality Studies, Percentage of Responding Programs
      Use of Performance MeasuresUse of Encounter Data
      M-CHIP
      M-CHIP = Children's Health Insurance Program that involves a Medicaid expansion.
      NASHP
      NASHP = National Academy for State Health Policy.
      278377
      S-CHIP
      S-CHIP = a separate CHIP program.
      NASHP358269
      Data Source is the Fourth Comprehensive Survey of Children's Health Insurance Programs, conducted by the National Academy for State Health Policy in 2008, and the Survey of Medicaid Programs on Quality Measures and Improvement Incentives in Medicaid and SCHIP, conducted by Health Management Associates in 2009. CHIP = Children's Health Insurance Program.
      HMA = Health Management Associates.
      M-CHIP = Children's Health Insurance Program that involves a Medicaid expansion.
      § NASHP = National Academy for State Health Policy.
      S-CHIP = a separate CHIP program.
      Both HEDIS and state-developed measures tailored to meet state-specific quality goals are collected. Shown in Table 1, 90% of the 41 state Medicaid programs that collect performance measures use HEDIS measures, with 76% supplementing HEDIS with state-defined measures and 10% using only state-defined measures. All but one of the Medicaid programs that reported using HEDIS measures use both HEDIS access and HEDIS effectiveness of care measures. Seventeen percent use measures endorsed by the National Quality Forum in addition to other measures. As for CHIP, 94% of the 33 S-CHIP and 96% of the 23 M-CHIP programs that collect performance measures use HEDIS measures, with 45% of S-CHIP and 56% of M-CHIP programs using both HEDIS and state-defined measures. Only 2 S-CHIP programs and 1 M-CHIP program use only state-defined measures. Proportionally fewer programs use HEDIS, and more use only state-defined measures with respect to fee-for-service providers than managed care organizations.

      Data Sources

      Encounter data—information regarding services provided to individual enrollees—is used by states for a variety of purposes, including monitoring plan and provider performance, quality assurance, and reporting on performance measures.
      • Department of Health and Human Services
      Medicaid Managed Care Encounter Data: Collection and Use.
      Shown in Table 1, 92% of CHIP programs that deliver care through managed care organizations collect encounter data from these plans. Roughly half of programs that provide care on a fee-for-service basis collect encounter data from fee-for-service providers. Also shown in Table 1, 60% of S-CHIP and 63% of M-CHIP programs perform, or have an independent contractor perform, medical record reviews of their fee-for-service providers, managed care organizations, or both. Use by states of medical record reviews is more common with respect to care provided through managed care than through fee-for-service providers.
      Finally, many CHIP programs report that they, a contractor, or their health plans perform enrollee surveys, although, as with other types of measurement, enrollee surveys are more commonly conducted with respect to children enrolled in managed care organizations than those receiving services from fee-for-service providers. Shown in Table 1, across both S-CHIP and M-CHIP programs, 94% of CHIP programs that provide services through managed care organizations and 46% of programs providing services on a fee-for-service basis use enrollee surveys. Of the CHIP programs reporting use of enrollee surveys, 79% use the Consumer Assessment of Healthcare Providers and Systems (CAHPS) surveys. A similar percentage (71%) of Medicaid programs use CAHPS.

      Duration of Enrollment and Retention

      As shown in Table 3, 7 S-CHIP programs (20%) and 1 M-CHIP program (4%) track length of enrollment, although they do so based on different time intervals. A somewhat greater number—10 S-CHIP (29%) and 3 M-CHIP programs (11%)—track average length of enrollment.
      Table 3State Collection of Enrollment and Retention Data in Separate and Medicaid Expansion CHIP Programs
      Data source is the Fourth Comprehensive Survey of Children’s Health Insurance Programs, conducted by the National Academy for State Health Policy in 2008. CHIP = Children’s Health Insurance Program.
      Program TypeNumber of Responding ProgramsPrograms Collecting Duration of Enrollment Measures % (No.)Programs Collecting Retention of Enrollment % (No.)
      Length of EnrollmentAverage Length of EnrollmentPrograms Tracking Rate of Retention at Eligibility RenewalPrograms Tracking Reasons Children Lose Eligibility at RenewalPrograms Tracking Churning
      States (with either or both CHIP programs)4615 (7)24 (11)28 (13)35 (16)7 (3)
      M-CHIP
      M-CHIP = a CHIP with a Medicaid expansion.
      programs
      274 (1)11 (3)11 (3)11 (3)4 (1)
      S-CHIP
      S-CHIP = a separate CHIP program.
      programs
      3520 (7)29 (10)34 (12)40 (14)9 (3)
      Data source is the Fourth Comprehensive Survey of Children’s Health Insurance Programs, conducted by the National Academy for State Health Policy in 2008. CHIP = Children’s Health Insurance Program.
      M-CHIP = a CHIP with a Medicaid expansion.
      S-CHIP = a separate CHIP program.
      Twelve S-CHIP (34%) and 3 M-CHIP programs (11%) track the rate of retention of coverage at the point at which a child’s eligibility must be renewed in one or both programs. Fourteen S-CHIP (40%) and 3 M-CHIP programs (11%) track the reasons children lose eligibility at renewal.
      Only 1 M-CHIP (4%) and 3 S-CHIP programs (9%) track data on “churning”—defined in the NASHP survey as the number or percentage of children who are disenrolled from the program and then reapply and are found eligible for coverage within a period of time specified by the state. Those that do track churning use different time periods: 3, 6, or 12 months.

      Provider Availability

      Shown in Table 4, 74% of M-CHIP and 54% of S-CHIP programs believe that their provider capacity is inadequate in some way. Shortages of dental and mental health providers were most frequently cited. Eighty-four percent of the S-CHIP and 85% of M-CHIP programs experiencing some type of provider shortage cited a shortage of general pediatric dentists; 68% of S-CHIP and 75% of M-CHIP programs cited a shortage of pediatric dental specialists. Fifty-three percent of S-CHIP and 55% of M-CHIP programs reported a shortage of pediatric psychiatrists, and 26% of S-CHIP and 35% of M-CHIP programs reported a shortage of other mental health providers. Fewer programs pointed to a shortage of pediatricians, other primary care providers, or other types of specialists. When asked if they experienced a shortage of providers in any other ways, over one third of S-CHIP and M-CHIP programs reported experiencing geographic-based shortages.
      Table 4State Perspectives on Provider Capacity Shortages in CHIP
      Data source is the Fourth Comprehensive Survey of Children’s Health Insurance Programs, conducted by the National Academy for State Health Policy in 2008. CHIP = Children’s Health Insurance Program.
      S-CHIP
      S-CHIP = a separate CHIP program.
      Programs
      M-CHIP
      M-CHIP = a CHIP that involves a Medicaid expansion.
      Programs
      No.%No.%
      Number of programs that responded to survey35882779
      Provider capacity adequate1646726
      Provider capacity not adequate19542074
       Pediatricians316630
       Other primary care providers316630
       Pediatric dentists (general/preventive care)16841785
       Pediatric dentists (specialists)13681575
       Pediatric psychiatrists10531155
       Other pediatric mental health providers526735
       Other types of specialists15315
       Geographic-based provider shortages737735
      Data source is the Fourth Comprehensive Survey of Children’s Health Insurance Programs, conducted by the National Academy for State Health Policy in 2008. CHIP = Children’s Health Insurance Program.
      S-CHIP = a separate CHIP program.
      M-CHIP = a CHIP that involves a Medicaid expansion.

      Racial and Ethnic Disparities

      Twelve states reported using collected data to evaluate disparities in access to care or treatment based on race or ethnicity in their S-CHIP program, M-CHIP program, or both, representing 7 S-CHIP (20%) and 5 M-CHIP programs (18%). The types of analyses performed by these programs vary. Of the programs that provided information in response to an open-ended question regarding the types of analyses performed, 4 reported analyzing data on performance measures for differences by race, ethnicity, and language spoken. One state, in both its S-CHIP and M-CHIP program, provides guidance on organizational practices that contribute to reducing health disparities by using the National Cultural and Linguistic Appropriate Services Standards.

      US Department of Health and Human Services, OPHS, Office of Minority Health. National Standards for Culturally and Linguistically Appropriate Services in Health Care. Rockville, Md: IQ Solutions, Inc; 2001. Available at: http://minorityhealth.hhs.gov/assets/pdf/checked/finalreport.pdf. Accessed February 7, 2010.

      One S-CHIP program uses collected data to develop “a cultural competency plan,” and another uses the data to ensure that services are “culturally appropriate,” although these terms are not defined. Two programs analyze utilization or quality improvement project data for racial and ethnic disparities.

      Challenges and Needs

      States clearly were not satisfied with measures available to them to meet their programs’ needs. They also had some clear ideas in common about additional measures that would be useful, issues of standardization, as well as technical assistance needs.

      Adequacy of Current Measures

      Only 28% of Medicaid and S-CHIP programs responding to the HMA survey believe that current performance measures are adequate. Of measures currently in use, states identified the HEDIS measures of immunization rates, well-child visits at 15 months, and adolescent well-care visits as being most useful. Despite the relative frequency with which CAHPS is used, several respondents to both surveys indicated that CAHPS does not adequately meet their needs. A few offered that enrollee satisfaction data collected at the plan level, regardless of the survey instrument, is not as useful as that collected at the provider level. Some have looked to other enrollee surveys, including the Promoting Healthy Development Survey

      US Department of Health and Human Services. Child and Adolescent Health Measurement Initiative (CAHMI): Promoting Healthy Development Survey (PHDS). Available at: http://www.ahrq.gov/chtoolbx/measure6.htm. Accessed March 25, 2010.

      and the Young Adult Healthcare Survey,

      US Department of Health and Human Services. Child and Adolescent Health Measurement Initiative (CAHMI): Young Adult Healthcare Survey (YAHCS). Available at: http://www.ahrq.gov/chtoolbx/measure7.htm. Accessed March 25, 2010.

      or have developed their own.

      Measures Needed

      When presented with a list of areas for which better measures might be needed, 61% of HMA survey respondents agreed that better measures of care coordination would be useful; over 40% agreed that better measures of mental health screening (49%), mental health treatment (43%), and access to dental care (41%) would be helpful; roughly one fifth indicated interest in better measures of adolescent access to family planning counseling (21%) and patient safety (19%). In addition to the choices offered in the HMA survey, states identified the need for additional measures in the following areas: underuse of needed care, longitudinal measures that allow tracking from diagnosis and referral through treatment and health outcomes, measures relevant to special populations, patient centeredness of care, health literacy, and population health measures. Fifty-two percent of survey respondents agreed new and better outcome measures are needed, and 38% would prefer measures that place less data collection burden on plans and providers.

      Need for More Standardization

      Almost one third of the HMA respondents identified a need for greater standardization of measurement and definitions. Many measures, designed for use in a managed care delivery system, are not easily adapted for programs relying on fee-for-service providers, and some officials mentioned the need for measures appropriate in a fee-for-service context, which would permit comparisons between fee-for-service and managed care; others identified a need for standardized definitions of the populations included in benchmark data (e.g., Temporary Assistance for Needy Families and Supplemental Security Income recipients) and a way to adapt measures for children who are not continuously covered in the program.

      Technical Assistance Needs

      Such standardization would help to facilitate one of states’ highest priorities for technical assistance: the creation of a national database, permitting the calculation of quality improvement benchmarks and meaningful comparisons among states. Seventy-five percent of programs responding to the HMA survey identified the creation of such a database as a high priority. Other types of technical assistance state programs would value include best practices information from other states (62%); information on disparities in a national database (51%); training in how to specify quality measures, analyze data, and present results (47%); and direct support of health information technology capacity and provider training (47%).

      Limitations

      Our attempt to synthesize data from 2 different surveys has some limitations. Although both surveys sought information on states’ experience with children’s health care quality measurement and improvement from overlapping groups of state programs, survey respondents from each state were not necessarily the same. The time period and specific issues addressed, and question phrasing, in each survey also differed. Both surveys were developed prior to the enactment of CHIPRA and not with the objective of informing the development of a core set of children’s health care quality measures. Finally, both the NASHP and HMA surveys were designed as descriptive point-in-time studies of self-reported state policies and procedures, and although both were developed with input from intended respondents, none of the survey instruments was tested for validity or reliability. Nonetheless, the surveys reveal important information about the extent and ways in which states currently are measuring and attempting to improve access to quality care through their programs, the type of data they collect, and how it is used. Further, because the Paperwork Reduction Act of 1981

      Paperwork Reduction Act, 44 USC §3501 (1981).

      limits the extent to which federal agencies can conduct standardized surveys of program officials and because of the limited time available to develop recommendations for the initial core set of pediatric measures, reliance on privately-funded data already collected was needed.

      Discussion

      Both the NASHP and the HMA surveys were fielded prior to the implementation of the CHIPRA provisions on pediatric quality measurement, and it is noteworthy that both reflect a high level of state activity and interest in this area. State officials responding to the HMA survey overwhelmingly ranked children’s health care quality as a high priority. Both surveys found that the vast majority of Medicaid and CHIP programs collect performance measures for children’s health care despite the absence of any federal mandates to do so, and states actively use the performance measures and data collected.
      The provisions in CHIPRA to develop a core set of pediatric quality measures thus are being planted in fertile ground. Depending on how the provisions are cultivated, including the degree of resonance with state needs, they could set the stage for significant, meaningful progress in state reporting on the quality of care provided to children enrolled in Medicaid and CHIP, and ultimately, on the quality of care for all children.
      The NASHP and HMA surveys indicate that most states are seeking additional meaningful measures of children’s health care quality. Over three quarters of the HMA survey respondents did not find current measures to be adequate. Although HEDIS measures are commonly employed, more than 8 of 10 Medicaid programs in the HMA survey and more than half of both S-CHIP and M-CHIP programs in the NASHP survey either supplement HEDIS measures with, or only use, state-developed performance measures. From the surveys, we also know that states are interested in good measures for patients’ experiences with care, as well as measures for care coordination and access to appropriate mental health and dental health services. The level of state activity in developing measures and the strong degree of state interest in specific areas suggest that future work to refine and expand on the initial core set might benefit from a closer examination of state experience and views.
      The tools and measures states employ and the ways in which they use information collected vary widely. Our analysis also confirms that there is more state experience with performance measures for health plans than for fee-for-service providers. Developing measures that can be used to compare access to quality care across programs and states, as well as across payment systems, is of interest to a number of states.
      Duration of enrollment, not often considered part of quality, is included in CHIPRA specifications for the initial core set of pediatric quality measures. The NASHP survey found only a small proportion of states measuring duration of enrollment or retention in some way. However, this limited state experience may be due more to lack of measures than lack of interest. No existing measures of duration of enrollment have been endorsed by a nationally recognized entity such as the National Committee for Quality Assurance or the National Quality Forum, and although the 2 potential measures recommended by Kenney and Pelletier
      • Kenney G.M.
      • Pelletier J.E.
      Monitoring duration of coverage in Medicaid and CHIP to assess program performance and quality.
      for inclusion in the initial core set may be suitable for future testing, they did not meet selection criteria for the initial core set.

      Agency for Healthcare Research and Quality. Background report for the request for public comment on initial, recommended core set of children's healthcare quality measures for voluntary use by Medicaid and CHIP programs. Rockville, MD: Agency for Healthcare Research and Quality; 2009. Available at: http://www.ahrq.gov/chipra/corebackgrnd.htm. Accessed January 5, 2010.

      US Department of Health and Human Services, Office of the Secretary. Medicaid and CHIP programs; initial core set of children's healthcare quality measures for voluntary use by Medicaid and CHIP programs. Washington, DC: US Government Printing Office; 2009. Available at: http://www.gpo.gov/fdsys/pkg/FR-2009-12-29/html/E9-30802.htm. Accessed March 26, 2010.

      From the authors’ extensive ongoing interactions with Medicaid and CHIP officials, it seems that many more states would like to measure performance on this dimension, both for its utility in assessing enrollment efforts and because continuous enrollment is a prerequisite for valid measurement of quality of care.
      • Summer L.
      • Mann C.
      Instability of Public Health Insurance Coverage for Children and their Families: Causes, Consequences, and Remedies.
      Availability of services also is included in CHIPRA specifications for the initial core set of pediatric quality measures, and the NASHP survey reflects considerable state concern in this area. Roughly two thirds of states reported inadequate provider capacity in either or both of their S-CHIP or M-CHIP programs. Finally, our findings on states use of race and ethnicity data are disappointing but not surprising. As noted in the background report accompanying publication of the proposed measures for inclusion in the initial core set, few of those measures are used to report data that distinguish care quality by race, ethnicity, or other factors.

      Agency for Healthcare Research and Quality. Background report for the request for public comment on initial, recommended core set of children's healthcare quality measures for voluntary use by Medicaid and CHIP programs. Rockville, MD: Agency for Healthcare Research and Quality; 2009. Available at: http://www.ahrq.gov/chipra/corebackgrnd.htm. Accessed January 5, 2010.

      US Department of Health and Human Services, Office of the Secretary. Medicaid and CHIP programs; initial core set of children's healthcare quality measures for voluntary use by Medicaid and CHIP programs. Washington, DC: US Government Printing Office; 2009. Available at: http://www.gpo.gov/fdsys/pkg/FR-2009-12-29/html/E9-30802.htm. Accessed March 26, 2010.

      The paucity of such measures at the national level is consistent with practice at the state level, as the NASHP survey reveals that the majority of states do not use data to evaluate disparities in access to care or treatment based on race or ethnicity in their CHIP programs.

      Conclusion

      To achieve significant state participation in measuring and reporting on children’s access to quality care, national programs need to be guided not only by national policy makers’ priorities, but also by state goals, capacity, and practice. If measures do not meet states’ needs, or the gap between measures and state priorities, practice, and capacity is too large, many states, particularly in poor economic times, will not be willing or able to collect and report the data. Therefore, in developing measures that states can and will use, it is critical to understand the extent and ways in which states currently are measuring and attempting to improve access to quality of care through their programs, the type of data they collect, and how the data are used.
      In the limited time provided under CHIPRA, the process for developing the initial core set of children’s health care quality measures included some important opportunities for state input. But these limited and short-term interactions need to be expanded to allow for sustained engagement with all, or at least a broad representative group of, state programs over time—not just to inform the selection of the initial core set but to enable meaningful, ongoing development and refinement of the core set and to maximize the value and extent of its use. The information presented in this article is a start, but much more can be done to engage states in developing and implementing children’s health care quality measurement and improvement programs.

      Acknowledgments

      Funding for this report was provided through purchase orders HHSP233200900802P and HHSP233200900781P by the US Department of Health and Human Services Agency for Healthcare Research and Quality , with funds provided by the US Department of Health and Human Services Centers for Medicare & Medicaid Services . The David and Lucile Packard Foundation, grant 2009–34617, supported the development, administration, and analysis of the NASHP survey. Development, administration, and analysis of the HMA survey were supported by the Commonwealth Fund . We also thank Elizabeth Osius and Neesha Rao for their assistance in analyzing the NASHP survey data and Jennifer Edwards for her assistance with the HMA survey data.

      References

      1. Centers for Medicare & Medicaid Services. FY 2008 number of children ever enrolled year–SCHIP by program type. Available at: http://www.cms.hhs.gov/NationalCHIPPolicy/downloads/FY2008StateTotalTable012309FINAL.pdf. Accessed January 12, 2010.

      2. Congressional Budget Office. Spending and enrollment detail for CBO’s March 2009 baseline: Children’s Health Insurance Program (CHIP). Available at: http://www.cbo.gov/budget/factsheets/2009b/chip.pdf. Accessed January 12, 2010.

        • National Academy for State Health Policy
        SCHIP at 10: Progress and Results.
        National Academy for State Health Policy, Washington, DC2007
        • Partridge L.
        Review of Access and Quality of Care in SCHIP Using Standardized National Performance Measures.
        National Health Policy Forum, Washington, DC2007
        • Duchon L.
        • Smith V.
        Quality Performance Measurement in Medicaid and SCHIP: Results of a 2006 National Survey of State Officials.
        National Association of Children’s Hospitals, Washington, DC2006
      3. Children’s Health Insurance Program Reauthorization Act of 2009, title IV. Available at: http://frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname=111_cong_public_laws&docid=f:publ003.111.pdf%20. Accessed March 26, 2010.

        • Mangione-Smith R.
        • Schiff J.
        • Dougherty D.
        Identifying children's health care quality measures for medicaid and CHIP: an evidence-informed, publicly transparent expert process.
        Acad Pediatr. 2011; 11: S11-S21
      4. Hess C, Farrell K, deLone S, et al. Charting CHIP IV: A Report on State Children's Health Insurance Programs Prior to Major Federal Policy Changes in 2009 and 2010. Washington, DC: National Academy for State Health Policy. In press.

        • Smith V.
        • Edwards J.
        • Reagan E.
        • Roberts D.
        Medicaid and CHIP Strategies for Improving Child Health.
        The Commonwealth Fund, New York, NY2009
        • Department of Health and Human Services
        Medicaid Managed Care Encounter Data: Collection and Use.
        Department of Health and Human Services, Office of Inspector General, Washington, DC2009 (OEI-07-06-00540)
      5. US Department of Health and Human Services, OPHS, Office of Minority Health. National Standards for Culturally and Linguistically Appropriate Services in Health Care. Rockville, Md: IQ Solutions, Inc; 2001. Available at: http://minorityhealth.hhs.gov/assets/pdf/checked/finalreport.pdf. Accessed February 7, 2010.

      6. US Department of Health and Human Services. Child and Adolescent Health Measurement Initiative (CAHMI): Promoting Healthy Development Survey (PHDS). Available at: http://www.ahrq.gov/chtoolbx/measure6.htm. Accessed March 25, 2010.

      7. US Department of Health and Human Services. Child and Adolescent Health Measurement Initiative (CAHMI): Young Adult Healthcare Survey (YAHCS). Available at: http://www.ahrq.gov/chtoolbx/measure7.htm. Accessed March 25, 2010.

      8. Paperwork Reduction Act, 44 USC §3501 (1981).

        • Kenney G.M.
        • Pelletier J.E.
        Monitoring duration of coverage in Medicaid and CHIP to assess program performance and quality.
        Acad Pediatr. 2011; 11: S34-S41
      9. Agency for Healthcare Research and Quality. Background report for the request for public comment on initial, recommended core set of children's healthcare quality measures for voluntary use by Medicaid and CHIP programs. Rockville, MD: Agency for Healthcare Research and Quality; 2009. Available at: http://www.ahrq.gov/chipra/corebackgrnd.htm. Accessed January 5, 2010.

      10. US Department of Health and Human Services, Office of the Secretary. Medicaid and CHIP programs; initial core set of children's healthcare quality measures for voluntary use by Medicaid and CHIP programs. Washington, DC: US Government Printing Office; 2009. Available at: http://www.gpo.gov/fdsys/pkg/FR-2009-12-29/html/E9-30802.htm. Accessed March 26, 2010.

        • Summer L.
        • Mann C.
        Instability of Public Health Insurance Coverage for Children and their Families: Causes, Consequences, and Remedies.
        Georgetown University Health Policy Institute, Washington, DC2006