Advertisement

Challenges to Measuring Variation in Readmission Rates of Neonatal Intensive Care Patients

      Abstract

      Objective

      To examine the viability of a hospital readmission quality metric for infants requiring neonatal intensive care.

      Methods

      Two cohorts were constructed. First, a cohort was constructed from infants born in California from 1995 to 2009 at 23 to 34 weeks' gestation, using birth certificates linked to maternal and infant inpatient records (N = 343,625). Second, the Medicaid Analytic eXtract (MAX) identified Medicaid-enrolled infants admitted to the neonatal intensive care unit (NICU) during their birth hospitalization in 18 states during 2006 to 2008 (N = 254,722). Hospital and state-level unadjusted readmission rates and rates adjusted for gestational age, birth weight, insurance status, gender, and common complications of preterm birth were calculated.

      Results

      Within California, there were wide variations in hospital-level readmission rates that were not completely explained through risk adjustment. Similar unadjusted variation was seen between states using MAX data, but risk adjustment and calculation of hospital-level rates were not possible because of missing gestational age, birth weight, and birth hospital data.

      Conclusions

      The California cohort shows significant variation in hospital-level readmission rates after risk adjustment, supporting the premise that readmission rates of prematurely born infants may reflect care quality. However, state data do not include term and early term infants requiring neonatal intensive care. MAX allows for multistate comparisons of all infants requiring NICU care. However, there were extensive missing data in the few states with sufficient information on managed care patients to calculate state-level measures. Constructing a valid readmission measure for NICU care across diverse states and regions requires improved data collection, including potential linkage between MAX data and vital statistics records.

      Keywords

      Preventing hospital readmissions is an area of emphasis for both insurers and public health professionals,
      • Tsai T.C.
      • Joynt K.E.
      • Orav E.J.
      • et al.
      Variation in surgical-readmission rates and quality of hospital care.
      • Berry J.G.
      • Toomey S.L.
      • Zaslavsky A.M.
      • et al.
      Pediatric readmission prevalence and variability across hospitals.
      especially in adult medicine, as the Center for Medicare and Medicaid Services has begun a readmission reduction program focused on variations in hospital readmission rates.

      Center for Medicare & Medicaid Services. Readmissions Reduction Program. August 2, 2013. Available at: http://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/AcuteInpatientPPS/Readmissions-Reduction-Program.html. Accessed January 2, 2014.

      Additionally, the Affordable Care Act includes provisions for financial incentives for improvements in care quality, with a particular focus on readmissions,

      Patient Protection and Affordable Care Act. 111th Congress of the United States of America at the Second Session. January 5, 2010, Washington, DC. Available at: http://www.gpo.gov/fdsys/pkg/BILLS-111hr3590enr/pdf/BILLS-111hr3590enr.pdf. Accessed May 29, 2014.

      and similar incentives are being explored for children enrolled in Medicaid or Children's Health Insurance Program (CHIP) through the Children's Health Insurance Program Reauthorization Act of 2009 (CHIPRA) Quality Demonstration Grant Program.
      • Fairbrother G.
      • Simpson L.A.
      Measuring and reporting quality of health care for children: CHIPRA and beyond.
      In pediatric medicine, there are groups of high-risk patients for whom hospital readmissions occur frequently. One such group is children discharged from the neonatal intensive care unit (NICU). Premature infants have an approximately 3-fold increase in risk of hospital readmission after discharge compared to term infants, with higher rates in infants of younger gestational age.
      • Ray K.N.
      • Lorch S.A.
      Hospitalization of early preterm, late preterm, and term infants during the first year of life by gestational age.
      These hospital readmissions contribute to the higher health care costs and utilization seen in prematurely born infants.
      • Wade K.C.
      • Lorch S.A.
      • Bakewell-Sachs S.
      • et al.
      Pediatric care for preterm infants after NICU discharge: high number of office visits and prescription medications.
      • Underwood M.A.
      • Danielsen B.
      • Gilbert W.M.
      Cost, causes and rates of rehospitalization of preterm infants.
      A limited number of studies show variation in readmission rates in 1 Canadian province
      • Martens P.J.
      • Derksen S.
      • Gupta S.
      Predictors of hospital readmission of Manitoba newborns within six weeks postbirth discharge: a population-based study.
      and selected groups of hospitals.
      • Morris B.H.
      • Gard C.C.
      • Kennedy K.
      Rehospitalization of extremely low birth weight (ELBW) infants: are there racial/ethnic disparities?.
      • Escobar G.J.
      • Greene J.D.
      • Hulac P.
      • et al.
      Rehospitalisation after birth hospitalisation: patterns among infants of all gestations.
      Differences in rates, though, may result from differences in illness severity
      • Lorch S.A.
      • Baiocchi M.
      • Silber J.H.
      • et al.
      The role of outpatient facilities in explaining variations in risk-adjusted readmission rates between hospitals.
      • Ambalavanan N.
      • Carlo W.A.
      • McDonald S.A.
      • et al.
      Identification of extremely premature infants at high risk of rehospitalization.
      or other patient characteristics
      • Lorch S.A.
      • Baiocchi M.
      • Silber J.H.
      • et al.
      The role of outpatient facilities in explaining variations in risk-adjusted readmission rates between hospitals.
      • Ambalavanan N.
      • Carlo W.A.
      • McDonald S.A.
      • et al.
      Identification of extremely premature infants at high risk of rehospitalization.
      • Ray K.
      • Escobar G.J.
      • Lorch S.A.
      Premature infants born to adolescent mothers: health care utilization after initial discharge academic pediatrics.
      across hospitals. There are no studies that show an association between changes in processes of care, especially around hospital discharge, and patient outcomes.
      • Goyal N.
      • Zubizarreta J.R.
      • Small D.S.
      • et al.
      Length of stay and readmissions among late preterm infants: an instrumental variable approach.
      Also, there are no studies of readmissions in this population among a wide number of states.
      NICU patients are a difficult group to study because 1) they may be more likely to be missing unique identifiers such as social security numbers; 2) their relatively high transfer rates require a data set with admissions linked by patient; 3) clinical data are important for adequate adjustment for case mix
      • Ray K.N.
      • Lorch S.A.
      Hospitalization of early preterm, late preterm, and term infants during the first year of life by gestational age.
      ; 4) prematurely born infants are a heterogeneous group that vary in gestational age and the prevalence of common chronic complications of preterm birth such as bronchopulmonary dysplasia (BPD), intraventricular hemorrhage (IVH), and retinopathy of prematurity (ROP), that affect the risk of hospital readmission
      • Ray K.N.
      • Lorch S.A.
      Hospitalization of early preterm, late preterm, and term infants during the first year of life by gestational age.
      • Lorch S.A.
      • Baiocchi M.
      • Silber J.H.
      • et al.
      The role of outpatient facilities in explaining variations in risk-adjusted readmission rates between hospitals.
      ; and 5) it is not clear what time frames after discharge should be examined. As a result, there are several challenges to using readmissions as a quality measure: the difficulty of procuring data from a wide range of states and hospitals for adequate comparison; the ability to construct a complete cohort of children receiving care at a specific neonatal intensive care unit; and access to clinical data for a risk-adjustment model that meets face validity of the end users.
      In order to assess quality and incentivize improvement, government institutions must monitor patient outcomes, identify poorly performing or high-performing institutions, and determine financial implications with value-based pricing. Ensuring adequate fidelity of the information presented is critical to these goals. Databases used by policy makers have different strengths and weaknesses (Table 1). The goal of this study is to examine unadjusted and risk-adjusted rates of hospital readmission at 5 time points for infants likely to receive treatment in the NICU based on either gestational age or procedure codes indicating treatment outside of well-baby care, using 2 potential data sets: 1 with complete clinical data (state data from California) and 1 without these clinical data (Medicaid/CHIP patients in MAX), and examine how these data sets address the above challenges.
      Table 1Data Sources for Population-Health Assessment of Health Care Quality
      Data SourceAdvantagesDisadvantagesExamples
      Prospective cohorts
      • High data accuracy
      • Complete data for evaluation
      • Limited to specific hospitals or health care providers
      • Limited time frame
      • Cost
      National Institute of Child Health and Human Development Neonatal Research Network (NICHD NRN)
      Electronic medical record databases
      • Complete data for region
      • Less onerous data collection
      • Reliance on accurate data entry by providers
      • Requires creation of fields for specific data of interest; otherwise relies on notation by providers in text fields
      • Requires access to and proficiency with technology
      • Variation in data dictionaries and structure across data sets
      • Cost
      Kaiser Permanente health information system
      Linked administrative data (birth certificates + claims)
      • Population-based data for specific region (states)
      • Cost to construct data sets
      • Clinical data limited to fields commonly collected by birth certificates and in claims data
      • Reliance on accurate coding of claims data and accurate completion of birth certificates, both of which require validation
      California data set: birth certificates linked to maternal and infant inpatient records
      Administrative claims data (billing data from states, insurance data)
      • Large population across geographic regions (national)
      • Limited to no clinical data
      • Reliance on accurate coding by providers, which requires validation
      • Insurance data sets are only from a subset of the population treated by states or hospitals
      Medicaid Analytic eXtract

      Methods

       Challenge 1: Data From a Wide Range of States and Hospitals

       California Linked Data Set

      Two separate cohorts of infants were constructed for this project. The first cohort consisted of infants born in California at a gestational age between 23 and 34 weeks between 1995 and 2009. The department of health linked these infants' birth certificates to death certificates using name and date of birth, and then de-identified the records. Then, over 98% of these records were linked to maternal and newborn hospital records using prior methods.
      • Herrchen B.
      • Gould J.B.
      • Nesbitt T.S.
      Vital statistics linked birth/infant death and hospital discharge record linkage for epidemiological studies.
      • Phibbs C.S.
      • Baker L.C.
      • Caughey A.B.
      • et al.
      Level and volume of neonatal intensive care and mortality in very-low-birth-weight infants.
      Over 80% of the unmatched live birth or fetal death certificate records were missing the delivery hospital, suggesting a birth at home or a birthing center. The unmatched records had similar gestational age and racial/ethnic distributions to the matched records. Because this data set contains records for all hospitals in California, we can measure readmissions at any California hospital, not simply readmissions to the discharging hospital. To ensure that there were enough patients per hospital to make reliable estimates of the readmission rate,
      • Silber J.H.
      • Rosenbaum P.R.
      • Brachet T.J.
      • et al.
      The Hospital Compare mortality model and the volume–outcome relationship.
      we limited the analyses to those hospitals that discharged over 50 eligible patients per year (N = 154). This cohort serves as a gold standard with variables necessary for risk adjustment from prior studies even if limited to infants delivered within a specific range of gestational ages.

       Medicaid Analytic eXtract

      The quality of the California data set is atypical because of the financial and time costs needed for its construction. Thus, the second cohort utilized the Medicaid Analytic eXtract (MAX), a data set derived from the Medicaid Statistical Information System, which is collected at the individual level by state Medicaid programs and standardized into MAX for interstate comparison by CMS. This data set is already in use by the federal government and states to assess Medicaid/CHIP program performance.

      Sebelius K; Department of Health and Human Services. Children’s Health Insurance Program Reauthorization Act. 2011 annual report on the quality of care for children in Medicaid and CHIP. September 2011. Available at: http://www.medicaid.gov/Medicaid-CHIP-Program-Information/By-Topics/Quality-of-Care/Downloads/2011_StateReporttoCongress.pdf. Accessed May 29, 2014.

      The first challenge to using MAX is that managed care claims data are often absent, and when they are included, the number of patients and encounter claims varies tremendously by state. Thus, we selected states that 1) had less than 10% of their Medicaid patients enrolled in a comprehensive managed care plan for the years of analysis; 2) had their managed care system classified as a Primary Care Case Management delivery system, whose claims are included in MAX; or 3) had external validation of the completeness and accuracy of their managed care encounter records in the MAX data set.

      Dodd AH, Nysenbaum J, Zlatinov AMPR. Assessing the usability of the MAX 2007 inpatient and prescription encounter data for enrollees in comprehensive managed care. Mathematica Policy Research: Medicaid Policy Brief. Brief 5. April 2012.

      Byrd VLH, Dodd AH, Malsberger R, et al. Assessing the usability of the MAX 2008 encounter data for enrollees in comprehensive managed care. Mathematica Policy Research: Medicaid Policy Brief. Brief 7. July 2012.

      Second, not all states report data from patients enrolled in the state CHIP. Although claims from states with an M-CHIP Medicaid expansion program are available in MAX, claims from states with a separate S-CHIP program, typically overseen by a state managed care insurer, are not. To mitigate this issue, we selected states with S-CHIP enrollment of less than 15% of all Medicaid/CHIP children, except for Idaho (increased to 40% in 2008) and Montana (20% to 25% between 2006 and 2008), whose data were included for geographic diversity. After applying these criteria, we had data from 18 states: Arizona, Idaho, Illinois, Indiana, Kansas, Kentucky, Louisiana, Missouri, Montana, New Hampshire, New Jersey, New Mexico, New York, North Carolina, Virginia, Vermont, Wisconsin, and Wyoming. Readmission rates from these states were compared to readmission rates from the remaining states and the District of Columbia. As in the California cohort, the MAX data set includes all hospitals in a given state and thus captures readmissions to all hospitals.

       Challenge 2: Constructing a Complete Cohort of Children Receiving Neonatal Intensive Care

       California Linked Data Set

      Because state data contains only gestational age and birth weight to determine whether a child required admission to a NICU, this study will use similar criteria to other work on the readmission risk of graduates of neonatal intensive care: a gestational age of 23 to 34 weeks. This definition misses term and early term infants who required NICU care. Our California cohort identified 343,625 infants, or approximately 5% of all births in the state.

       Medicaid Analytic eXtract

      Unlike other data sets, the MAX dataset contains Current Procedural Terminology (CPT) codes to allow the identification of all infants who required neonatal intensive care during their birth hospitalization regardless of gestational age or birth weight. Thus, patients were eligible for inclusion if they had at least 1 CPT code for hospital care other than standard well-baby care during their birth hospitalization: 99293–99296, or 99298–99300. These codes identified 254,722 patients from the 2,753,902 infants born in the 18 states between 2006 and 2008, for a total of 9.7% of the births.

       Outcome Measures

      CMS uses a 30-day time frame for their readmission measures.

      Center for Medicare & Medicaid Services. Readmissions Reduction Program. August 2, 2013. Available at: http://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/AcuteInpatientPPS/Readmissions-Reduction-Program.html. Accessed January 2, 2014.

      However, there are no studies to support this time frame over shorter frames
      • Escobar G.J.
      • Greene J.D.
      • Hulac P.
      • et al.
      Rehospitalisation after birth hospitalisation: patterns among infants of all gestations.
      • Escobar G.J.
      • Joffe S.
      • Gardner M.N.
      • et al.
      Rehospitalization in the first two weeks after discharge from the neonatal intensive care unit.
      and longer frames used in prior work.
      • Ray K.N.
      • Lorch S.A.
      Hospitalization of early preterm, late preterm, and term infants during the first year of life by gestational age.
      • Lorch S.A.
      • Baiocchi M.
      • Silber J.H.
      • et al.
      The role of outpatient facilities in explaining variations in risk-adjusted readmission rates between hospitals.
      Shorter time periods may better reflect the care delivered by the inpatient hospital course, whereas longer time periods may reflect either the transition of care to outpatient providers or the overall illness severity of these infants. Thus, we will examine rehospitalizations for any reason within 7, 14, 30, 90, and 365 days after discharge from the birth hospitalization. These time periods are assessed cumulatively, such that readmissions occurring within prior time periods are included. Because of missing inpatient records noted below, hospitalizations after discharge were identified in MAX using inpatient CPT codes of 99221–99223 and 99231–99233 or a hospital admission more than 1 day after discharge from the NICU in the state-based data.

       Challenge 3: Adequate Clinical Data for Risk Adjustment

       California Linked Data Set

      We include characteristics of the infant that may increase the risk of hospital readmission after discharge from the NICU on the basis of prior work
      • Ray K.N.
      • Lorch S.A.
      Hospitalization of early preterm, late preterm, and term infants during the first year of life by gestational age.
      • Lorch S.A.
      • Baiocchi M.
      • Silber J.H.
      • et al.
      The role of outpatient facilities in explaining variations in risk-adjusted readmission rates between hospitals.
      • Lorch S.A.
      • Baiocchi M.
      • Ahlberg C.E.
      • et al.
      The differential impact of delivery hospital on the outcomes of premature infants.
      : gestational age, birth weight, gender, and insurance status. Gestational age and birth weight are specifically captured in birth certificate records (Table 1). We also assessed how risk-adjusted hospital rates changed when we included common complications of preterm birth associated with readmissions in prior work, and captured using ICD-9CM codes in hospital administrative records: BPD, IVH, ROP, and necrotizing enterocolitis (NEC). Including these factors is controversial when assessing a facility's quality, though, because poor inpatient quality of care may result in higher complication rates. Gestational age, birth weight, sociodemographic information, and complications of premature birth were available in over 98% of records in the California state data, and have been used in prior work from this data set.
      • Phibbs C.S.
      • Baker L.C.
      • Caughey A.B.
      • et al.
      Level and volume of neonatal intensive care and mortality in very-low-birth-weight infants.
      • Lorch S.A.
      • Baiocchi M.
      • Ahlberg C.E.
      • et al.
      The differential impact of delivery hospital on the outcomes of premature infants.

       Medicaid Analytic eXtract

      MAX data, like many administrative data sets, do not collect gestational age and birth weight, so we used ICD-9 CM codes or CPT codes to categorize patients into 2-week gestational age categories or 3 birth weight categories. Some infants were missing some portion of birth hospital record in the inpatient file in MAX and thus had no or limited ICD-9 data to construct the gestational age, birth weight, or complication variables. This issue occurred because records from 1 or more hospital stays during the birth hospitalization were not linked to the MAX record. The majority of these cases occurred when the infant was hospitalized outside the home state of the mother. The amount of missing data was examined by state to ensure that the missingness was not centered on 1 or several large states in the data set.

       Data Analysis

      Readmission rates are shown as box-and-whisker plots. Variation between the states with the lowest and highest readmission rates were reported as both absolute difference in rates and standardized difference in rates, calculated by the following formula: [(highest rate − lowest rate) × 100]/(standard deviation of all rates). For risk-adjusted rates, which we were able to calculate only using the California cohort, separate logistic regression models shown in Online Appendix 1 estimated the risk of readmission within one of the specified time frames using the above risk adjustment variables and fixed hospital effects. Risk-adjusted readmission rates were calculated for each hospital using methods from the Center for Medicare and Medicaid services.

      Center for Medicare & Medicaid Services. Readmissions Reduction Program. August 2, 2013. Available at: http://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/AcuteInpatientPPS/Readmissions-Reduction-Program.html. Accessed January 2, 2014.

      Results

       Hospital-Level Variation in the State of California

       Unadjusted Variation

      Among infants with a gestational age between 23 and 34 weeks, there was substantial variation in the unadjusted readmission rates among California hospitals regardless of time period examined (Fig. 1) with a standardized difference that ranged from 578% to 683%.
      Figure thumbnail gr1
      Figure 1California linked data set. Unadjusted rates of hospital readmissions 7, 14, 30, 90, and 365 days after NICU discharge for hospitals with >50 eligible discharges/year. Edges of each box represent 25th and 75th percentiles; inner line denotes median. Bars above and below box represent upper and lower adjacent values, respectively, or 1.5 times interquartile range above and below box. Outliers beyond these bars are shown as dots.

       Adjusted Variation

      The large variation between hospitals persisted after adjusting for gestational age and sociodemographic factors (Fig. 2; Online Appendix 1), with standardized differences again ranging between 660% and 724%. Adding common complications of preterm birth to the risk adjustment model made little difference to the readmission rates, with an average decline of 0.2% (standard deviation 2.3%) for readmissions 7, 14, and 30 days after discharge, and a 0.3% average decline for readmissions further from discharge.
      Figure thumbnail gr2
      Figure 2California linked data set. Hospital readmission rates, adjusted for gestational age, gender, insurance status, maternal age, and race/ethnicity. Admissions occurred within 7, 14, 30, 90, and 365 days of discharge from birth hospitalization for hospitals with >50 eligible discharges/year. Edges of each box represent 25th and 75th percentiles; inner line denotes median. Bars above and below box represent upper and lower adjacent values, respectively, or 1.5 times interquartile range above and below box. Outliers beyond these bars are shown as dots. The hospital with the lowest adjusted readmission rates was used as the reference hospital.

       State-Level Variation in the MAX Data Set

      Variation in unadjusted readmission rates between 18 identified states is shown in Figure 3. There are large variations in these rates across all time frames examined. The standardized differences between the states with the lowest and highest unadjusted readmission rates ranged from 316% to 352% from 7 to 365 days after discharge. The 18 identified states had statistically different readmission rates at 90 and 365 days after discharge compared to the 33 states not included in this study.
      Figure thumbnail gr3
      Figure 3MAX data set. Unadjusted rates of readmissions across 18 states in MAX and 33 states not included. Edges of each box represent 25th and 75th percentiles; inner line denotes median. Bars above and below box represent upper and lower adjacent values, respectively, or 1.5 times interquartile range above and below box. Outliers beyond these bars are shown as dots.
      Effective risk-adjustment using the MAX data was impossible as a result of the extensive amount of missing data in the data set (Table 2). Gestational age was missing in 46% to 81% of patients and birth weight was missing in 18% to 61% of patients, and the data available for birth weight did not allow for more than 3 categories (<1500 g, 1500 to 2500 g, >2500 g). Missing hospital data ranged from 2.9% in New York to 27.0% in Illinois, reflecting the different cross-border trends of patients within these states. The greatest impact seen in states with large obstetric or pediatric hospitals immediately across state lines, such as Illinois (St. Louis, MO), New Jersey (Philadelphia, PA; New York City, NY), and Kentucky (Cincinnati, OH).
      Table 2State Counts of Missing Birth Weight, Gestational Age, and Inpatient Hospital Record Within MAX Data Set
      StateMissing Birth WeightMissing Gestational AgeMissing Inpatient Record
      n%n%n%
      AZ883242.741458070.5612325.96
      ID159239.47234958.2457914.36
      IL2021054.672531868.48999227.03
      IN682638.771345576.43277915.79
      KS547261.39685176.87145916.37
      KY276127.03619960.69219621.50
      LA308518.591114767.1613998.43
      MO667943.141199877.5012898.33
      MT50029.8577146.031197.10
      NC840133.451317452.469043.60
      NH54639.5777255.94614.42
      NJ682750.561093981.01255018.88
      NM294938.56555772.66116115.18
      NY3206459.633932173.1215492.88
      VA326234.06607263.40139314.55
      VT32335.9751957.8015417.15
      WI265129.09522357.328138.92
      WY88456.52102065.22634.03
      MAX indicates Medicaid Analytic eXtract.

      Discussion

      There are many potential data sources to examine readmission rates of neonatal intensive care units. These data sets have different advantages and disadvantages. California state data containing clinical information needed for risk adjustment (answering challenge 3) show substantial unadjusted and risk-adjusted hospital-level variation in readmission rates. Only a small portion of this variation could be explained by differences in gestational age and sociodemographic data, and thus, at least in this patient population, readmission rates may assess care quality.
      Although these variations in California are potentially more intervenable by state policy makers than variations across larger geographic areas, they are only for a single state and, like other state data sets without CPT codes, exclude term and early term infants who required neonatal intensive care.
      • Lorch S.A.
      • Myers S.
      • Carr B.
      The regionalization of pediatric health care: a state of the art review.
      However, assessing NICU readmissions on a state or national level using MAX data suffers from substantial challenges. Even though the degree of variation in unadjusted readmission rates between states in the MAX data set is similar to those seen in California hospitals, there are limited clinical data to allow for risk adjustment (challenge 3). Gestational age is missing in 40% to 80% of the infants, making multiple imputation and other techniques difficult to implement and reducing the face validity of any risk-adjusted result. Finally, the majority of states do not include information from managed care providers, which biases the calculation of readmission rates in those states. Thus, neither the California state data nor the MAX data set can answer challenge 1 (data from a wide range of states and hospitals).
      This work highlights what can be done with a complete data set, as in the California study, but also the challenge to implementing such a metric for state stakeholders. Adequate risk adjustment for NICU readmissions requires either gestational age or birth weight as a measure of prematurity. Gestational age is considered ideal because birth weight may be lower in small-for-gestational age infants, or higher for infants of diabetic mothers. In these cases, the risk of complications and death is better associated with the infant's gestational age.
      • Escobar G.J.
      • Greene J.D.
      • Hulac P.
      • et al.
      Rehospitalisation after birth hospitalisation: patterns among infants of all gestations.
      However, as our data show, gestational age is extremely difficult to obtain in any data set that does not specifically collect such information. Although multiple imputation is one way of dealing with the missing data, it is onerous to implement and insufficient when states had a high degree of missing data for this important variable, up to 60% in the MAX data set. Birth weight was more commonly included in MAX, but does not allow stratification of the population beyond the 3 categories of <1500 g, 1500 to 2500 g, and >2500 g. Unless there is improvement in the use of ICD-9 codes to identify gestational age and birth weight in preterm infants (challenge 3), and improved managed care data quality in states (challenge 1), states may have to include birth certificate data in the implementation of NICU readmissions, similar to other CHIPRA measures for Cesarean rate for nulliparous singleton vertex and live births weighing less than 2500 g.

      Centers for Medicare and CHIP Services; Center for Medicare and Medicaid Services. Core set of children’s health care quality measures for Medicaid and CHIP (Child Core Set): technical specifications and resource manual for federal fiscal year 2013 reporting. May 2013. Available at: http://www.medicaid.gov/Medicaid-CHIP-Program-Information/By-Topics/Quality-of-Care/Downloads/Medicaid-and-CHIP-Child-Core-Set-Manual.pdf. Accessed May 29, 2014.

      Alternatively, states could consider alternative data sets as described in Table 1.
      The full significance of the hospital and state variations is also unclear. Further research is needed to determine the potential preventability of hospital readmissions after discharge from the NICU, especially considering prior work suggesting that less than 5% of all pediatric readmissions were preventable.
      • Hain P.D.
      • Gay J.C.
      • Berutti T.W.
      • et al.
      Preventability of early readmissions at a children’s hospital.
      Many established quality metrics, including those identified through the CHIPRA program, strive for a 100% or 0% performance rate.

      Centers for Medicare and CHIP Services; Center for Medicare and Medicaid Services. Core set of children’s health care quality measures for Medicaid and CHIP (Child Core Set): technical specifications and resource manual for federal fiscal year 2013 reporting. May 2013. Available at: http://www.medicaid.gov/Medicaid-CHIP-Program-Information/By-Topics/Quality-of-Care/Downloads/Medicaid-and-CHIP-Child-Core-Set-Manual.pdf. Accessed May 29, 2014.

      Other quality measures with a baseline number of events that should occur, such as the rate of operative Cesarean sections, demonstrate the difficulty for defining the preventability of these events, the correct number of events per hospital, and defining which hospitals are truly outliers.
      • Srinivas S.K.
      • Fager C.
      • Lorch S.A.
      Evaluating risk-adjusted cesarean section rates as a measure of obstetric quality.
      • Bailit J.L.
      • Love T.E.
      • Dawson N.V.
      Quality of obstetric care and risk-adjusted primary cesarean delivery rates.
      • Gould J.B.
      • Danielsen B.
      • Korst L.M.
      • et al.
      Cesarean delivery rates and neonatal morbidity in a low-risk population.
      Even the hospitals with the lowest risk-adjusted rates of readmissions had rates between 2.3 and 3%, and it is unclear whether this rate is too low. Additionally, although some of the variation could be related to the quality of care provided either during the hospitalization or in the discharge process,
      • Escobar G.J.
      • Greene J.D.
      • Hulac P.
      • et al.
      Rehospitalisation after birth hospitalisation: patterns among infants of all gestations.
      • Profit J.
      • McCormick M.C.
      • Escobar G.J.
      • et al.
      Neonatal intensive care unit census influences discharge of moderately preterm infants.
      other variation could be related to practice or the ability of high-risk children to receive outpatient services or a medical home that may reduce the likelihood of a hospitalization.
      • Morris B.H.
      • Gard C.C.
      • Kennedy K.
      Rehospitalization of extremely low birth weight (ELBW) infants: are there racial/ethnic disparities?.
      • Lorch S.A.
      • Baiocchi M.
      • Silber J.H.
      • et al.
      The role of outpatient facilities in explaining variations in risk-adjusted readmission rates between hospitals.
      • Lamarche-Vadel A.
      • Blondel B.
      • Truffer P.
      • et al.
      Re-hospitalization in infants younger than 29 weeks’ gestation in the EPIPAGE cohort.
      However, the wide range of readmission rates suggest some association with care delivery without the hospital and outpatient networks caring for these infants.
      This work shows variation in hospital-level readmission rates for preterm graduates of neonatal intensive care even after adjusting for gestational age, sociodemographic factors, and common complications of preterm birth. However, much more work is needed to understand the underlying factors associated with these variations, and to identify the level of readmissions for which a state or hospital should strive. There are substantial, potentially insurmountable barriers to implementing such a measure in data sets commonly available to public health agencies, because these data sets do not currently contain the clinical data needed for adequate risk adjustment or lack claims information on a large number of patients because of their type of insurance or where they received care. The use of MAX or other administrative data sets to examine the health care of infants who require neonatal intensive care at birth may require information available in electronic health records, birth certificates, or in all-payer data sets available in some states.

      National Conference of State Legislatures. All-Payer Claims Databases—Health Cost Containment October 2013. Available at: http://www.ncsl.org/research/health/collecting-health-data-all-payer-claims-database.aspx. Accessed January 2, 2014.

      These challenges require discussion before implementing this metric on a large scale.

      Acknowledgments

      Funded by the Agency for Healthcare Research and Quality , U18 HS020508 , PI: Jeffrey H. Silber and Agency for Healthcare Research and Quality , R01 HS018661-01 , PI: Scott A. Lorch.

      Supplementary Data

      References

        • Tsai T.C.
        • Joynt K.E.
        • Orav E.J.
        • et al.
        Variation in surgical-readmission rates and quality of hospital care.
        N Engl J Med. 2013; 369: 1134-1142
        • Berry J.G.
        • Toomey S.L.
        • Zaslavsky A.M.
        • et al.
        Pediatric readmission prevalence and variability across hospitals.
        JAMA. 2013; 309: 372-380
      1. Center for Medicare & Medicaid Services. Readmissions Reduction Program. August 2, 2013. Available at: http://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/AcuteInpatientPPS/Readmissions-Reduction-Program.html. Accessed January 2, 2014.

      2. Patient Protection and Affordable Care Act. 111th Congress of the United States of America at the Second Session. January 5, 2010, Washington, DC. Available at: http://www.gpo.gov/fdsys/pkg/BILLS-111hr3590enr/pdf/BILLS-111hr3590enr.pdf. Accessed May 29, 2014.

        • Fairbrother G.
        • Simpson L.A.
        Measuring and reporting quality of health care for children: CHIPRA and beyond.
        Acad Pediatr. 2011; 11: S77-S84
        • Ray K.N.
        • Lorch S.A.
        Hospitalization of early preterm, late preterm, and term infants during the first year of life by gestational age.
        Hosp Pediatr. 2013; 3: 194-203
        • Wade K.C.
        • Lorch S.A.
        • Bakewell-Sachs S.
        • et al.
        Pediatric care for preterm infants after NICU discharge: high number of office visits and prescription medications.
        J Perinatol. 2008; 28: 696-701
        • Underwood M.A.
        • Danielsen B.
        • Gilbert W.M.
        Cost, causes and rates of rehospitalization of preterm infants.
        J Perinatol. 2007; 27: 614-619
        • Martens P.J.
        • Derksen S.
        • Gupta S.
        Predictors of hospital readmission of Manitoba newborns within six weeks postbirth discharge: a population-based study.
        Pediatrics. 2004; 114: 708-713
        • Morris B.H.
        • Gard C.C.
        • Kennedy K.
        Rehospitalization of extremely low birth weight (ELBW) infants: are there racial/ethnic disparities?.
        J Perinatol. 2005; 25: 656-663
        • Escobar G.J.
        • Greene J.D.
        • Hulac P.
        • et al.
        Rehospitalisation after birth hospitalisation: patterns among infants of all gestations.
        Arch Dis Child. 2005; 90: 125-131
        • Lorch S.A.
        • Baiocchi M.
        • Silber J.H.
        • et al.
        The role of outpatient facilities in explaining variations in risk-adjusted readmission rates between hospitals.
        Health Serv Res. 2010; 45: 24-41
        • Ambalavanan N.
        • Carlo W.A.
        • McDonald S.A.
        • et al.
        Identification of extremely premature infants at high risk of rehospitalization.
        Pediatrics. 2011; 128: e1216-e1225
        • Ray K.
        • Escobar G.J.
        • Lorch S.A.
        Premature infants born to adolescent mothers: health care utilization after initial discharge academic pediatrics.
        Acad Pediatr. 2010; 10: 302-308
        • Goyal N.
        • Zubizarreta J.R.
        • Small D.S.
        • et al.
        Length of stay and readmissions among late preterm infants: an instrumental variable approach.
        Hosp Pediatr. 2013; 3: 7-15
        • Herrchen B.
        • Gould J.B.
        • Nesbitt T.S.
        Vital statistics linked birth/infant death and hospital discharge record linkage for epidemiological studies.
        Comput Biomed Res. 1997; 30: 290-305
        • Phibbs C.S.
        • Baker L.C.
        • Caughey A.B.
        • et al.
        Level and volume of neonatal intensive care and mortality in very-low-birth-weight infants.
        N Engl J Med. 2007; 356: 2165-2175
        • Silber J.H.
        • Rosenbaum P.R.
        • Brachet T.J.
        • et al.
        The Hospital Compare mortality model and the volume–outcome relationship.
        Health Serv Res. 2010; 45: 1148-1167
      3. Sebelius K; Department of Health and Human Services. Children’s Health Insurance Program Reauthorization Act. 2011 annual report on the quality of care for children in Medicaid and CHIP. September 2011. Available at: http://www.medicaid.gov/Medicaid-CHIP-Program-Information/By-Topics/Quality-of-Care/Downloads/2011_StateReporttoCongress.pdf. Accessed May 29, 2014.

      4. Dodd AH, Nysenbaum J, Zlatinov AMPR. Assessing the usability of the MAX 2007 inpatient and prescription encounter data for enrollees in comprehensive managed care. Mathematica Policy Research: Medicaid Policy Brief. Brief 5. April 2012.

      5. Byrd VLH, Dodd AH, Malsberger R, et al. Assessing the usability of the MAX 2008 encounter data for enrollees in comprehensive managed care. Mathematica Policy Research: Medicaid Policy Brief. Brief 7. July 2012.

        • Escobar G.J.
        • Joffe S.
        • Gardner M.N.
        • et al.
        Rehospitalization in the first two weeks after discharge from the neonatal intensive care unit.
        Pediatrics. 1999; 104: e2
        • Lorch S.A.
        • Baiocchi M.
        • Ahlberg C.E.
        • et al.
        The differential impact of delivery hospital on the outcomes of premature infants.
        Pediatrics. 2012; 130: 270-278
        • Lorch S.A.
        • Myers S.
        • Carr B.
        The regionalization of pediatric health care: a state of the art review.
        Pediatrics. 2010; 126: 1182-1190
      6. Centers for Medicare and CHIP Services; Center for Medicare and Medicaid Services. Core set of children’s health care quality measures for Medicaid and CHIP (Child Core Set): technical specifications and resource manual for federal fiscal year 2013 reporting. May 2013. Available at: http://www.medicaid.gov/Medicaid-CHIP-Program-Information/By-Topics/Quality-of-Care/Downloads/Medicaid-and-CHIP-Child-Core-Set-Manual.pdf. Accessed May 29, 2014.

        • Hain P.D.
        • Gay J.C.
        • Berutti T.W.
        • et al.
        Preventability of early readmissions at a children’s hospital.
        Pediatrics. 2013; 131: e171-e181
        • Srinivas S.K.
        • Fager C.
        • Lorch S.A.
        Evaluating risk-adjusted cesarean section rates as a measure of obstetric quality.
        Obstet Gynecol. 2010; 115: 1007-1013
        • Bailit J.L.
        • Love T.E.
        • Dawson N.V.
        Quality of obstetric care and risk-adjusted primary cesarean delivery rates.
        Am J Obstet Gynecol. 2006; 194: 402-407
        • Gould J.B.
        • Danielsen B.
        • Korst L.M.
        • et al.
        Cesarean delivery rates and neonatal morbidity in a low-risk population.
        Obstet Gynecol. 2004; 104: 11-19
        • Profit J.
        • McCormick M.C.
        • Escobar G.J.
        • et al.
        Neonatal intensive care unit census influences discharge of moderately preterm infants.
        Pediatrics. 2007; 119: 314-319
        • Lamarche-Vadel A.
        • Blondel B.
        • Truffer P.
        • et al.
        Re-hospitalization in infants younger than 29 weeks’ gestation in the EPIPAGE cohort.
        Acta Paediatr. 2004; 93: 1340-1345
      7. National Conference of State Legislatures. All-Payer Claims Databases—Health Cost Containment October 2013. Available at: http://www.ncsl.org/research/health/collecting-health-data-all-payer-claims-database.aspx. Accessed January 2, 2014.