If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Department of Pediatrics, The Children's Hospital of Philadelphia and Perelman School of Medicine at The University of Pennsylvania, Philadelphia, PaCenter for Outcomes Research, The Children's Hospital of Philadelphia, Philadelphia, PaLeonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, Pa
To examine the viability of a hospital readmission quality metric for infants requiring neonatal intensive care.
Methods
Two cohorts were constructed. First, a cohort was constructed from infants born in California from 1995 to 2009 at 23 to 34 weeks' gestation, using birth certificates linked to maternal and infant inpatient records (N = 343,625). Second, the Medicaid Analytic eXtract (MAX) identified Medicaid-enrolled infants admitted to the neonatal intensive care unit (NICU) during their birth hospitalization in 18 states during 2006 to 2008 (N = 254,722). Hospital and state-level unadjusted readmission rates and rates adjusted for gestational age, birth weight, insurance status, gender, and common complications of preterm birth were calculated.
Results
Within California, there were wide variations in hospital-level readmission rates that were not completely explained through risk adjustment. Similar unadjusted variation was seen between states using MAX data, but risk adjustment and calculation of hospital-level rates were not possible because of missing gestational age, birth weight, and birth hospital data.
Conclusions
The California cohort shows significant variation in hospital-level readmission rates after risk adjustment, supporting the premise that readmission rates of prematurely born infants may reflect care quality. However, state data do not include term and early term infants requiring neonatal intensive care. MAX allows for multistate comparisons of all infants requiring NICU care. However, there were extensive missing data in the few states with sufficient information on managed care patients to calculate state-level measures. Constructing a valid readmission measure for NICU care across diverse states and regions requires improved data collection, including potential linkage between MAX data and vital statistics records.
especially in adult medicine, as the Center for Medicare and Medicaid Services has begun a readmission reduction program focused on variations in hospital readmission rates.
Additionally, the Affordable Care Act includes provisions for financial incentives for improvements in care quality, with a particular focus on readmissions,
and similar incentives are being explored for children enrolled in Medicaid or Children's Health Insurance Program (CHIP) through the Children's Health Insurance Program Reauthorization Act of 2009 (CHIPRA) Quality Demonstration Grant Program.
In pediatric medicine, there are groups of high-risk patients for whom hospital readmissions occur frequently. One such group is children discharged from the neonatal intensive care unit (NICU). Premature infants have an approximately 3-fold increase in risk of hospital readmission after discharge compared to term infants, with higher rates in infants of younger gestational age.
across hospitals. There are no studies that show an association between changes in processes of care, especially around hospital discharge, and patient outcomes.
Also, there are no studies of readmissions in this population among a wide number of states.
NICU patients are a difficult group to study because 1) they may be more likely to be missing unique identifiers such as social security numbers; 2) their relatively high transfer rates require a data set with admissions linked by patient; 3) clinical data are important for adequate adjustment for case mix
; 4) prematurely born infants are a heterogeneous group that vary in gestational age and the prevalence of common chronic complications of preterm birth such as bronchopulmonary dysplasia (BPD), intraventricular hemorrhage (IVH), and retinopathy of prematurity (ROP), that affect the risk of hospital readmission
; and 5) it is not clear what time frames after discharge should be examined. As a result, there are several challenges to using readmissions as a quality measure: the difficulty of procuring data from a wide range of states and hospitals for adequate comparison; the ability to construct a complete cohort of children receiving care at a specific neonatal intensive care unit; and access to clinical data for a risk-adjustment model that meets face validity of the end users.
In order to assess quality and incentivize improvement, government institutions must monitor patient outcomes, identify poorly performing or high-performing institutions, and determine financial implications with value-based pricing. Ensuring adequate fidelity of the information presented is critical to these goals. Databases used by policy makers have different strengths and weaknesses (Table 1). The goal of this study is to examine unadjusted and risk-adjusted rates of hospital readmission at 5 time points for infants likely to receive treatment in the NICU based on either gestational age or procedure codes indicating treatment outside of well-baby care, using 2 potential data sets: 1 with complete clinical data (state data from California) and 1 without these clinical data (Medicaid/CHIP patients in MAX), and examine how these data sets address the above challenges.
Table 1Data Sources for Population-Health Assessment of Health Care Quality
Data Source
Advantages
Disadvantages
Examples
Prospective cohorts
•
High data accuracy
•
Complete data for evaluation
•
Limited to specific hospitals or health care providers
•
Limited time frame
•
Cost
National Institute of Child Health and Human Development Neonatal Research Network (NICHD NRN)
Electronic medical record databases
•
Complete data for region
•
Less onerous data collection
•
Reliance on accurate data entry by providers
•
Requires creation of fields for specific data of interest; otherwise relies on notation by providers in text fields
•
Requires access to and proficiency with technology
•
Variation in data dictionaries and structure across data sets
•
Cost
Kaiser Permanente health information system
Linked administrative data (birth certificates + claims)
•
Population-based data for specific region (states)
•
Cost to construct data sets
•
Clinical data limited to fields commonly collected by birth certificates and in claims data
•
Reliance on accurate coding of claims data and accurate completion of birth certificates, both of which require validation
California data set: birth certificates linked to maternal and infant inpatient records
Administrative claims data (billing data from states, insurance data)
•
Large population across geographic regions (national)
•
Limited to no clinical data
•
Reliance on accurate coding by providers, which requires validation
•
Insurance data sets are only from a subset of the population treated by states or hospitals
Challenge 1: Data From a Wide Range of States and Hospitals
California Linked Data Set
Two separate cohorts of infants were constructed for this project. The first cohort consisted of infants born in California at a gestational age between 23 and 34 weeks between 1995 and 2009. The department of health linked these infants' birth certificates to death certificates using name and date of birth, and then de-identified the records. Then, over 98% of these records were linked to maternal and newborn hospital records using prior methods.
Over 80% of the unmatched live birth or fetal death certificate records were missing the delivery hospital, suggesting a birth at home or a birthing center. The unmatched records had similar gestational age and racial/ethnic distributions to the matched records. Because this data set contains records for all hospitals in California, we can measure readmissions at any California hospital, not simply readmissions to the discharging hospital. To ensure that there were enough patients per hospital to make reliable estimates of the readmission rate,
we limited the analyses to those hospitals that discharged over 50 eligible patients per year (N = 154). This cohort serves as a gold standard with variables necessary for risk adjustment from prior studies even if limited to infants delivered within a specific range of gestational ages.
Medicaid Analytic eXtract
The quality of the California data set is atypical because of the financial and time costs needed for its construction. Thus, the second cohort utilized the Medicaid Analytic eXtract (MAX), a data set derived from the Medicaid Statistical Information System, which is collected at the individual level by state Medicaid programs and standardized into MAX for interstate comparison by CMS. This data set is already in use by the federal government and states to assess Medicaid/CHIP program performance.
The first challenge to using MAX is that managed care claims data are often absent, and when they are included, the number of patients and encounter claims varies tremendously by state. Thus, we selected states that 1) had less than 10% of their Medicaid patients enrolled in a comprehensive managed care plan for the years of analysis; 2) had their managed care system classified as a Primary Care Case Management delivery system, whose claims are included in MAX; or 3) had external validation of the completeness and accuracy of their managed care encounter records in the MAX data set.
Dodd AH, Nysenbaum J, Zlatinov AMPR. Assessing the usability of the MAX 2007 inpatient and prescription encounter data for enrollees in comprehensive managed care. Mathematica Policy Research: Medicaid Policy Brief. Brief 5. April 2012.
Byrd VLH, Dodd AH, Malsberger R, et al. Assessing the usability of the MAX 2008 encounter data for enrollees in comprehensive managed care. Mathematica Policy Research: Medicaid Policy Brief. Brief 7. July 2012.
Second, not all states report data from patients enrolled in the state CHIP. Although claims from states with an M-CHIP Medicaid expansion program are available in MAX, claims from states with a separate S-CHIP program, typically overseen by a state managed care insurer, are not. To mitigate this issue, we selected states with S-CHIP enrollment of less than 15% of all Medicaid/CHIP children, except for Idaho (increased to 40% in 2008) and Montana (20% to 25% between 2006 and 2008), whose data were included for geographic diversity. After applying these criteria, we had data from 18 states: Arizona, Idaho, Illinois, Indiana, Kansas, Kentucky, Louisiana, Missouri, Montana, New Hampshire, New Jersey, New Mexico, New York, North Carolina, Virginia, Vermont, Wisconsin, and Wyoming. Readmission rates from these states were compared to readmission rates from the remaining states and the District of Columbia. As in the California cohort, the MAX data set includes all hospitals in a given state and thus captures readmissions to all hospitals.
Challenge 2: Constructing a Complete Cohort of Children Receiving Neonatal Intensive Care
California Linked Data Set
Because state data contains only gestational age and birth weight to determine whether a child required admission to a NICU, this study will use similar criteria to other work on the readmission risk of graduates of neonatal intensive care: a gestational age of 23 to 34 weeks. This definition misses term and early term infants who required NICU care. Our California cohort identified 343,625 infants, or approximately 5% of all births in the state.
Medicaid Analytic eXtract
Unlike other data sets, the MAX dataset contains Current Procedural Terminology (CPT) codes to allow the identification of all infants who required neonatal intensive care during their birth hospitalization regardless of gestational age or birth weight. Thus, patients were eligible for inclusion if they had at least 1 CPT code for hospital care other than standard well-baby care during their birth hospitalization: 99293–99296, or 99298–99300. These codes identified 254,722 patients from the 2,753,902 infants born in the 18 states between 2006 and 2008, for a total of 9.7% of the births.
Outcome Measures
CMS uses a 30-day time frame for their readmission measures.
Shorter time periods may better reflect the care delivered by the inpatient hospital course, whereas longer time periods may reflect either the transition of care to outpatient providers or the overall illness severity of these infants. Thus, we will examine rehospitalizations for any reason within 7, 14, 30, 90, and 365 days after discharge from the birth hospitalization. These time periods are assessed cumulatively, such that readmissions occurring within prior time periods are included. Because of missing inpatient records noted below, hospitalizations after discharge were identified in MAX using inpatient CPT codes of 99221–99223 and 99231–99233 or a hospital admission more than 1 day after discharge from the NICU in the state-based data.
Challenge 3: Adequate Clinical Data for Risk Adjustment
California Linked Data Set
We include characteristics of the infant that may increase the risk of hospital readmission after discharge from the NICU on the basis of prior work
: gestational age, birth weight, gender, and insurance status. Gestational age and birth weight are specifically captured in birth certificate records (Table 1). We also assessed how risk-adjusted hospital rates changed when we included common complications of preterm birth associated with readmissions in prior work, and captured using ICD-9CM codes in hospital administrative records: BPD, IVH, ROP, and necrotizing enterocolitis (NEC). Including these factors is controversial when assessing a facility's quality, though, because poor inpatient quality of care may result in higher complication rates. Gestational age, birth weight, sociodemographic information, and complications of premature birth were available in over 98% of records in the California state data, and have been used in prior work from this data set.
MAX data, like many administrative data sets, do not collect gestational age and birth weight, so we used ICD-9 CM codes or CPT codes to categorize patients into 2-week gestational age categories or 3 birth weight categories. Some infants were missing some portion of birth hospital record in the inpatient file in MAX and thus had no or limited ICD-9 data to construct the gestational age, birth weight, or complication variables. This issue occurred because records from 1 or more hospital stays during the birth hospitalization were not linked to the MAX record. The majority of these cases occurred when the infant was hospitalized outside the home state of the mother. The amount of missing data was examined by state to ensure that the missingness was not centered on 1 or several large states in the data set.
Data Analysis
Readmission rates are shown as box-and-whisker plots. Variation between the states with the lowest and highest readmission rates were reported as both absolute difference in rates and standardized difference in rates, calculated by the following formula: [(highest rate − lowest rate) × 100]/(standard deviation of all rates). For risk-adjusted rates, which we were able to calculate only using the California cohort, separate logistic regression models shown in Online Appendix 1 estimated the risk of readmission within one of the specified time frames using the above risk adjustment variables and fixed hospital effects. Risk-adjusted readmission rates were calculated for each hospital using methods from the Center for Medicare and Medicaid services.
Hospital-Level Variation in the State of California
Unadjusted Variation
Among infants with a gestational age between 23 and 34 weeks, there was substantial variation in the unadjusted readmission rates among California hospitals regardless of time period examined (Fig. 1) with a standardized difference that ranged from 578% to 683%.
Figure 1California linked data set. Unadjusted rates of hospital readmissions 7, 14, 30, 90, and 365 days after NICU discharge for hospitals with >50 eligible discharges/year. Edges of each box represent 25th and 75th percentiles; inner line denotes median. Bars above and below box represent upper and lower adjacent values, respectively, or 1.5 times interquartile range above and below box. Outliers beyond these bars are shown as dots.
The large variation between hospitals persisted after adjusting for gestational age and sociodemographic factors (Fig. 2; Online Appendix 1), with standardized differences again ranging between 660% and 724%. Adding common complications of preterm birth to the risk adjustment model made little difference to the readmission rates, with an average decline of 0.2% (standard deviation 2.3%) for readmissions 7, 14, and 30 days after discharge, and a 0.3% average decline for readmissions further from discharge.
Figure 2California linked data set. Hospital readmission rates, adjusted for gestational age, gender, insurance status, maternal age, and race/ethnicity. Admissions occurred within 7, 14, 30, 90, and 365 days of discharge from birth hospitalization for hospitals with >50 eligible discharges/year. Edges of each box represent 25th and 75th percentiles; inner line denotes median. Bars above and below box represent upper and lower adjacent values, respectively, or 1.5 times interquartile range above and below box. Outliers beyond these bars are shown as dots. The hospital with the lowest adjusted readmission rates was used as the reference hospital.
Variation in unadjusted readmission rates between 18 identified states is shown in Figure 3. There are large variations in these rates across all time frames examined. The standardized differences between the states with the lowest and highest unadjusted readmission rates ranged from 316% to 352% from 7 to 365 days after discharge. The 18 identified states had statistically different readmission rates at 90 and 365 days after discharge compared to the 33 states not included in this study.
Figure 3MAX data set. Unadjusted rates of readmissions across 18 states in MAX and 33 states not included. Edges of each box represent 25th and 75th percentiles; inner line denotes median. Bars above and below box represent upper and lower adjacent values, respectively, or 1.5 times interquartile range above and below box. Outliers beyond these bars are shown as dots.
Effective risk-adjustment using the MAX data was impossible as a result of the extensive amount of missing data in the data set (Table 2). Gestational age was missing in 46% to 81% of patients and birth weight was missing in 18% to 61% of patients, and the data available for birth weight did not allow for more than 3 categories (<1500 g, 1500 to 2500 g, >2500 g). Missing hospital data ranged from 2.9% in New York to 27.0% in Illinois, reflecting the different cross-border trends of patients within these states. The greatest impact seen in states with large obstetric or pediatric hospitals immediately across state lines, such as Illinois (St. Louis, MO), New Jersey (Philadelphia, PA; New York City, NY), and Kentucky (Cincinnati, OH).
Table 2State Counts of Missing Birth Weight, Gestational Age, and Inpatient Hospital Record Within MAX Data Set
There are many potential data sources to examine readmission rates of neonatal intensive care units. These data sets have different advantages and disadvantages. California state data containing clinical information needed for risk adjustment (answering challenge 3) show substantial unadjusted and risk-adjusted hospital-level variation in readmission rates. Only a small portion of this variation could be explained by differences in gestational age and sociodemographic data, and thus, at least in this patient population, readmission rates may assess care quality.
Although these variations in California are potentially more intervenable by state policy makers than variations across larger geographic areas, they are only for a single state and, like other state data sets without CPT codes, exclude term and early term infants who required neonatal intensive care.
However, assessing NICU readmissions on a state or national level using MAX data suffers from substantial challenges. Even though the degree of variation in unadjusted readmission rates between states in the MAX data set is similar to those seen in California hospitals, there are limited clinical data to allow for risk adjustment (challenge 3). Gestational age is missing in 40% to 80% of the infants, making multiple imputation and other techniques difficult to implement and reducing the face validity of any risk-adjusted result. Finally, the majority of states do not include information from managed care providers, which biases the calculation of readmission rates in those states. Thus, neither the California state data nor the MAX data set can answer challenge 1 (data from a wide range of states and hospitals).
This work highlights what can be done with a complete data set, as in the California study, but also the challenge to implementing such a metric for state stakeholders. Adequate risk adjustment for NICU readmissions requires either gestational age or birth weight as a measure of prematurity. Gestational age is considered ideal because birth weight may be lower in small-for-gestational age infants, or higher for infants of diabetic mothers. In these cases, the risk of complications and death is better associated with the infant's gestational age.
However, as our data show, gestational age is extremely difficult to obtain in any data set that does not specifically collect such information. Although multiple imputation is one way of dealing with the missing data, it is onerous to implement and insufficient when states had a high degree of missing data for this important variable, up to 60% in the MAX data set. Birth weight was more commonly included in MAX, but does not allow stratification of the population beyond the 3 categories of <1500 g, 1500 to 2500 g, and >2500 g. Unless there is improvement in the use of ICD-9 codes to identify gestational age and birth weight in preterm infants (challenge 3), and improved managed care data quality in states (challenge 1), states may have to include birth certificate data in the implementation of NICU readmissions, similar to other CHIPRA measures for Cesarean rate for nulliparous singleton vertex and live births weighing less than 2500 g.
Alternatively, states could consider alternative data sets as described in Table 1.
The full significance of the hospital and state variations is also unclear. Further research is needed to determine the potential preventability of hospital readmissions after discharge from the NICU, especially considering prior work suggesting that less than 5% of all pediatric readmissions were preventable.
Other quality measures with a baseline number of events that should occur, such as the rate of operative Cesarean sections, demonstrate the difficulty for defining the preventability of these events, the correct number of events per hospital, and defining which hospitals are truly outliers.
Even the hospitals with the lowest risk-adjusted rates of readmissions had rates between 2.3 and 3%, and it is unclear whether this rate is too low. Additionally, although some of the variation could be related to the quality of care provided either during the hospitalization or in the discharge process,
other variation could be related to practice or the ability of high-risk children to receive outpatient services or a medical home that may reduce the likelihood of a hospitalization.
However, the wide range of readmission rates suggest some association with care delivery without the hospital and outpatient networks caring for these infants.
This work shows variation in hospital-level readmission rates for preterm graduates of neonatal intensive care even after adjusting for gestational age, sociodemographic factors, and common complications of preterm birth. However, much more work is needed to understand the underlying factors associated with these variations, and to identify the level of readmissions for which a state or hospital should strive. There are substantial, potentially insurmountable barriers to implementing such a measure in data sets commonly available to public health agencies, because these data sets do not currently contain the clinical data needed for adequate risk adjustment or lack claims information on a large number of patients because of their type of insurance or where they received care. The use of MAX or other administrative data sets to examine the health care of infants who require neonatal intensive care at birth may require information available in electronic health records, birth certificates, or in all-payer data sets available in some states.
These challenges require discussion before implementing this metric on a large scale.
Acknowledgments
Funded by the Agency for Healthcare Research and Quality , U18 HS020508 , PI: Jeffrey H. Silber and Agency for Healthcare Research and Quality , R01 HS018661-01 , PI: Scott A. Lorch.
Dodd AH, Nysenbaum J, Zlatinov AMPR. Assessing the usability of the MAX 2007 inpatient and prescription encounter data for enrollees in comprehensive managed care. Mathematica Policy Research: Medicaid Policy Brief. Brief 5. April 2012.
Byrd VLH, Dodd AH, Malsberger R, et al. Assessing the usability of the MAX 2008 encounter data for enrollees in comprehensive managed care. Mathematica Policy Research: Medicaid Policy Brief. Brief 7. July 2012.
The authors declare that they have no conflict of interest.
Publication of this article was supported by the US Department of Health and Human Services and the Agency for Healthcare Research and Quality.
The views expressed in this article are those of the authors and do not necessarily represent the views of the US Department of Health and Human Services or the Agency for Healthcare Research and Quality.
00231-9&id=gr1.jpg){kind=link}
00231-9&id=gr2.jpg){kind=link}
00231-9&id=gr3.jpg){kind=link}