Advertisement

Identifying Sickle Cell Disease Cases Using Administrative Claims

      Abstract

      Objective

      To develop and test the accuracy of administrative claims method for identifying children with sickle cell disease (SCD) to enable quality of care assessments among children enrolled in Medicaid.

      Methods

      All administrative claims with an SCD diagnosis were obtained from Michigan Medicaid from 2008 to 2011 for children ≤18 years, representing 1828 individuals. All Medicaid claims were obtained for these children and classified into categories on the basis of SCD care; these classifications were used to develop 37 alternative case definitions for identifying children with SCD. Children with ≥1 SCD claim in 2010 or 2011 were identified as confirmed SCD or not SCD using the gold standard of Michigan newborn screening administrative records. Measures of performance were calculated for each case definition for eligible children in 2010. Further validation of the case definitions was performed among eligible children in 2011.

      Results

      In 2010, a total of 938 children met eligibility criteria and were linked to newborn screening records; 605 (59%) were confirmed SCD, and 333 (32%) were not SCD. Measures of performance varied among the 37 case definitions, and the 4 best case definitions on the basis of the sensitivity, specificity, and area under the receiver operating characteristic (ROC) curve were validated among 924 children meeting eligibility criteria in 2011. The case definition of at least 3 SCD claims in any position identified children with SCD with the most accuracy, with an area under the ROC curve of 0.91 (95% confidence interval 0.89, 0.93).

      Conclusions

      This definition can be used to facilitate a more accurate identification of children with SCD in future studies. Further investigation is necessary to determine whether this method translates to other populations besides Michigan Medicaid-insured children.

      Keywords

      Sickle cell disease (SCD) is a chronic disease affecting mainly minority populations and is characterized by significant morbidity and mortality. SCD is estimated to currently affect 90,000 to 100,000 Americans (approximately 1 in 500 African American births), although variation exists among prevalence estimates.
      • Hassell K.L.
      Population estimates of sickle cell disease in the US.
      • Berg A.O.
      Sickle cell disease: screening, diagnosis, management, and counseling in newborns and infants. The Agency for Health Care Policy and Research.
      • Lorey F.W.
      • Arnopp J.
      • Cunningham G.C.
      Distribution of hemoglobinopathy variants by ethnicity in a multiethnic state.
      • Michlitsch J.
      • Azimi M.
      • Hoppe C.
      • et al.
      Newborn screening for hemoglobinopathies in California.
      SCD has multiple clinically significant forms, further complicating estimates of the true burden of disease. There are 6 sickle cell genotypes; however, the most common variants are sickle cell anemia (hemoglobin SS), hemoglobin (Hb) SC, HbS/β0-thalassemia, and HbS/β+-thalassemia.
      • Stuart M.J.
      • Nagel R.L.
      Sickle-cell disease.
      Children with SCD are at risk for chronic symptoms that can seriously impact quality of life, including pain episodes, severe anemia, and pulmonary complications.
      • Gladwin M.T.
      • Vichinsky E.
      Pulmonary complications of sickle cell disease.

      National Heart Lung and Blood Institute. The management of sickle cell disease. 2002. Available at: http://www.nhlbi.nih.gov/health/prof/blood/sickle/sc_mngt.pdf. Accessed October 27, 2013.

      SCD can have devastating consequences among children if uncontrolled and can lead to potentially life-threatening complications. Children with SCD are 7 to 30 times more likely to be hospitalized, 2 to 6 times more likely to visit the emergency department, 300 times more likely to have a stroke, and 100 times more likely to develop pneumococcal infection; further, they have over 8 times the health care expenditures than their counterparts without SCD.
      • Shankar S.M.
      • Arbogast P.G.
      • Mitchel E.
      • et al.
      Medical care utilization and mortality in sickle cell disease: a population-based study.
      • Bilenker J.H.
      • Weller W.E.
      • Shaffer T.J.
      • et al.
      The costs of children with sickle cell anemia: preparing for managed care.
      • Raphael J.L.
      • Dietrich C.L.
      • Whitmire D.
      • et al.
      Healthcare utilization and expenditures for low income children with sickle cell disease.
      • Ohene-Frempong K.
      • Weiner S.J.
      • Sleeper L.A.
      • et al.
      Cerebrovascular accidents in sickle cell disease: rates and risk factors.
      Given these risks, it is essential that children with SCD have effective follow-up immediately after birth and that preventive services are obtained throughout childhood.

      National Heart Lung and Blood Institute. The management of sickle cell disease. 2002. Available at: http://www.nhlbi.nih.gov/health/prof/blood/sickle/sc_mngt.pdf. Accessed October 27, 2013.

      • Vichinsky E.
      • Hurst D.
      • Earles A.
      • et al.
      Newborn screening for sickle cell disease: effect on mortality.
      • Gaston M.H.
      • Verter J.I.
      • Woods G.
      • et al.
      Prophylaxis with oral penicillin in children with sickle cell anemia. A randomized trial.
      • Adams R.J.
      • McKie V.C.
      • Carl E.M.
      • et al.
      Long-term stroke risk in children with sickle cell disease screened with transcranial Doppler.
      At birth, all children are screened for SCD through state newborn screening (NBS) programs; these results could potentially enable identification of cases for ongoing quality of care assessments. However, state Medicaid programs may not have the technical capacity or policies established to authorize links between NBS results and administrative claims to support quality of care assessments. Absent the capability to establish these linkages, a claims-based definition is necessary to identify SCD cases. Although quality of care assessments that use administrative claims data have been previously developed for conditions such as asthma and diabetes,
      • Christakis D.A.
      • Feudtner C.
      • Pihoker C.
      • Connell F.A.
      Continuity and quality of care for children with diabetes who are covered by Medicaid.
      • Lieu T.A.
      • Lozano P.
      • Finkelstein J.A.
      • et al.
      Racial/ethnic variation in asthma status and management practices among children in managed Medicaid.
      • Cotter J.J.
      • Smith W.R.
      • Rossiter L.F.
      • et al.
      Combining state administrative databases and provider records to assess the quality of care for children enrolled in Medicaid.
      • Finkelstein J.A.
      • Lozano P.
      • Farber H.J.
      • et al.
      Underuse of controller medications among Medicaid-insured children with asthma.
      • Dombkowski K.J.
      • Wasilevich E.A.
      • Lyon-Callo S.K.
      Pediatric asthma surveillance using Medicaid claims.
      a mechanism to identify SCD cases using claims has not been validated. If successful, a claims-based method offers important opportunities to evaluate population-based quality of care among children with SCD without requiring linkages to external data sources such as those maintained by state NBS programs. With that in mind, our objective was to develop and test the feasibility and accuracy of an administrative claims method for identifying children with SCD to enable quality of care assessments among children enrolled in Medicaid.

      Methods

      We developed and tested alternative methods for identifying children with SCD using Medicaid administrative claims data. We used a 5-step process that included: 1) acquisition of all Medicaid administrative claims for any child with at least 1 SCD claim; 2) classification of claims into meaningful groups relevant to SCD care; 3) development of alternative case definitions using these variables; 4) identification of the testing population to validate the accuracy of the alternative case definitions; and 5) test of the accuracy of the alternative case definitions to identify children with SCD.

      Acquisition of Claims

      In partnership with the Michigan Department of Community Health, we obtained all Medicaid administrative claims with a SCD ICD-9 diagnosis code for children 18 years or younger. All children were included, regardless of Medicaid enrollment status, during the period 2008 to 2011. Consistent with other studies and Agency for Healthcare Research and Quality (AHRQ) Healthcare Cost and Utilization Project Single-Level Clinical Classification Software (HCUP CCS), we included claims with ICD-9 diagnosis codes for HbSS (282.60, 282.61, 282.62), HbSC (282.63, 282.64), HbSD (282.68, 282.69), and HbS β-thalassemia (282.41 and 282.42); we did not include sickle cell trait (282.5) or other hemoglobinopathies.

      Healthcare Cost and Utilization Project. Clinical Classifications Software (CCS) for ICD-9-CM. Available at: http://www.hcup-us.ahrq.gov/toolssoftware/ccs/ccs.jsp. Accessed September 26, 2013.

      • Yusuf H.R.
      • Atrash H.K.
      • Grosse S.D.
      • et al.
      Emergency department visits made by patients with sickle cell disease: a descriptive study, 1999–2007.
      • Leschke J.
      • Panepinto J.A.
      • Nimmer M.
      • et al.
      Outpatient follow-up and rehospitalizations for sickle cell disease patients.
      • Ovbiagele B.
      • Adams R.J.
      Trends in comorbid sickle cell disease among stroke patients.
      A total of 66,274 SCD claims containing 304,289 revenue and/or procedure codes representing 1828 unique individuals were identified from 2008 to 2011. All Michigan Medicaid administrative claims data were acquired for these 1828 individuals for each year, including detailed enrollment characteristics (containing demographics and program eligibility information), provider information, and health services (including inpatient, outpatient, emergency department, and pharmacy). These data were linked to provide a comprehensive overview of paid services rendered to any child with an SCD claim. The claims tables included codes from all major health care coding schemes used to track patients and obtain reimbursement, including ICD-9-CM diagnosis codes, diagnosis-related groups, uniform billing (UB-92) codes, ICD-9-CM surgical codes, current procedural terminology (CPT) codes, Healthcare Common Procedure Coding System, and national drug codes. A de-duplication process was implemented, and claims were grouped together into events according to dates of service.

      Classification of Claims

      As a precursor to creating alternative case definitions, we classified individuals on the basis of claims history relevant to SCD care using the extracted SCD claims. Our classification approaches were derived from methodologies published by AHRQ, National Committee for Quality Assurance Healthcare Effectiveness Data and Information Set, and Centers for Medicare and Medicaid Services.

      Healthcare Cost and Utilization Project. Clinical Classifications Software (CCS) for ICD-9-CM. Available at: http://www.hcup-us.ahrq.gov/toolssoftware/ccs/ccs.jsp. Accessed September 26, 2013.

      National Committee for Quality Assurance. Healthcare Effectiveness Data and Information Set (HEDIS). Available at: http://www.ncqa.org/HEDISQualityMeasurement/HEDISMeasures/HEDIS2012.aspx. Accessed September 25, 2013.

      Centers for Medicare and Medicaid Services. Available at: http://www.cms.gov. Accessed September 26, 2013.

      We started the process with a complete extraction of all SCD claims to maximize the likelihood that even diagnosis and procedure codes that were infrequently used would be given consideration in our approach. Interim results were shared among the team of investigators that included substantial expertise in Medicaid claims data analyses, chronic disease epidemiology, newborn screening, and statistical programming. Team members reviewed candidate case definitions based solely on one coding system (eg, an outpatient definition using CPT codes) and evaluated the incremental advantages or disadvantages of including other coding schemas (eg, an outpatient definition using both CPT and revenue codes). These considerations were aided by tabular frequency counts of the number of unique individuals and event counts for each code as well as visual representations (eg, Venn diagrams). These methods were jointly reviewed by team members to evaluate the degree of overlap between code groups and the unique contribution of each code group in capturing distinct individuals with SCD. We used an iterative approach to determine the degree to which each child had specific groups of claims representing meaningful categories of SCD care. We subsequently classified each child's claims from several perspectives, ranging from simple counts of SCD claims to claim counts on the basis of combinations of different SCD services.

      Development of Alternative Case Definitions

      From our analysis of SCD claims groupings, we identified 9 mutually exclusive claims categories: 7 health services categories (inpatient, outpatient, home health care, emergency department, blood transfusion), and 2 medication categories (antibiotic prophylaxis and hydroxyurea). In addition, 2 composite groups were formed: evaluation/consultation claims and an overall count of SCD claims (irrespective of type of service). Table 1 provides these categories and lists several additional categories that were considered but not included in the final case definitions. These categories served as the basis for the development of alternative case definitions to identify children with SCD from administrative claims. Table 2 illustrates the resultant 37 case definitions considered; the definitions reflect alternatives aimed at balancing inclusion of cases to maximize sensitivity with the addition of increasingly restrictive criteria to gain specificity. Alternative definitions were also considered on the basis of whether the diagnosis code for SCD was reported as the primary diagnosis or any mention of SCD for emergency department and inpatient claims.
      Table 1Claims Classifications Included in Case Defintions
      CategoryDefinition
      SCD claim countHealthcare Cost and Utilization Clinical Classification Software (HCUP CCS) #61 for ICD-9-CM codes (trait excluded)
      Evaluation/consultationHCUP CCS #227 for procedure codes
      Outpatient
      Codes for clinic care and preventive medicine were dropped due to low additional yield (1.1%) among the SCD cases.
      99201–99205, 99211–99215, 99241–99245
      ED
      CPT codes for ED were dropped due to low additional yield (6.9%) among the SCD cases, difficulty of processing claims lacking admission dates, discharge dates, and primary diagnosis.
      HCUP ED utilization flag
      Inpatient hospitalization
      CPT codes for Inpatient Hospitalization were dropped due to low additional yield (8.7%) among the SCD cases, difficulty of processing claims lacking admission dates, discharge dates, and primary diagnosis.
      HCUP cost-center clusters: RBU, SCU, NUR where facility type is outpatient or inpatient hospital
      Home health careHCUP CCS #236 for procedure codes
      Blood transfusionHCUP CCS #222 and Centers for Medicare and Medicaid Services (CMS) cost-center cluster blood processing/transfusion
      Antibiotic prophylaxisAminopenicillins, β-lactamase inhibitors, macrolides, miscellaneous antibiotics, natural penicillins
      HydroxyureaHydroxyurea
      SCD = sickle cell disease; ED = emergency department; RBU = routine bed units; SCU = special care units; NUR = nursery.
      Other categories created but not used in case definition: Observation Stays, SCD Screening and Confirmatory Testing, Chemistry/Hematology Lab Claims, and Transracial Doppler.
      Codes for clinic care and preventive medicine were dropped due to low additional yield (1.1%) among the SCD cases.
      CPT codes for ED were dropped due to low additional yield (6.9%) among the SCD cases, difficulty of processing claims lacking admission dates, discharge dates, and primary diagnosis.
      CPT codes for Inpatient Hospitalization were dropped due to low additional yield (8.7%) among the SCD cases, difficulty of processing claims lacking admission dates, discharge dates, and primary diagnosis.
      Table 2Description of Case Definitions Developed for Identification of Children With SCD
      All definitions list minimum number of required claims.
      Definition NumberCriteria
      All SCD Claims
       12 claims, any position
       23 claims, any position
       34 claims, any position
       45 claims, any position
      Interview Evaluation/Consultation SCD Claims
       51 claim, any position
       62 claims, any position
       73 claims, any position
       84 claims, any position
       95 claims, any position
      Outpatient SCD Claims
       101 claim, any position
       112 claims, any position
       123 claims, any position
       134 claims, any position
      Emergency Department SCD Claims
       141 claim, any position
       152 claims, any position
       163 claims, any position
       174 claims, any position
       181 claim, primary position
       192 claims, primary position
       203 claims, primary position
       214 claims, primary position
      Inpatient Hospitalization SCD Claims
       221 claim, any position
       232 claims, any position
       243 claims, any position
       254 claims, any position
       261 claim, primary position
       272 claims, primary position
       283 claims, primary position
       294 claims, primary position
      Combination of SCD Claims
       30At least 1 inpatient claim, primary position OR at least 1 ED claim, primary position OR at least 4 outpatient claims, any position
       31At least 1 inpatient claim, any position OR at least 1 ED claim, any position
       32At least 1 inpatient claim, any position OR at least 1 ED claim, any position OR at least 4 SCD outpatient claims, any position
       33At least 1 inpatient claim, any position OR at least 1 ED claim, any position OR at least 3 SCD outpatient claims, any position
       34At least 1 inpatient claim, any position OR at least 1 ED claim, any position OR at least 2 SCD outpatient claims, any position
       35At least 1 inpatient claim, any position OR at least 1 ED claim, any position OR at least 1 outpatient claim, any position
       36At least 1 inpatient claim, any position OR at least 1 ED claim, any position OR at least 1 outpatient claim, any position OR at least 1 home health care claim, any position OR at least 5 blood transfusions with a SCD diagnosis OR had at least 300 days of antibiotics (age 0-5 yr)
       37At least 1 inpatient claim, any position OR 2 claims for either outpatient or ED, any position
      SCD = sickle cell disease; ED = emergency department.
      All definitions list minimum number of required claims.

      Identification of Testing Population

      We selected a subset of individuals initially identified in our SCD claims database to test the accuracy of the alternative case definitions. Our testing subset included children in 2010 to 2011 who were 1 to 18 years who had at least 1 Medicaid SCD claim in either year and had a Michigan NBS result available. NBS results were available for all children born in Michigan from 1987 to 2010; children born outside the state of Michigan were excluded. In addition, we required continuous enrollment in Michigan Medicaid with no other forms of health insurance during the year of the SCD claim. Eligible children were linked to Michigan birth certificates using child's name, birth date, and sex. Unlinked records were manually reviewed to attempt to locate a birth record. Birth records were subsequently linked to newborn screening (NBS) records maintained by the Michigan Department of Community Health using common variables.
      • Korzeniewski S.J.
      • Grigorescu V.
      • Copeland G.
      • et al.
      Methodological innovations in data gathering: newborn screening linkage with live births records, Michigan, 1/2007–3/2008.
      Records that were not automatically linked were reviewed manually to identify additional matches. Using NBS records, children were classified as confirmed SCD (HbSS, HbS/β-thalassemia, HbSC, and other variants), no SCD, or unknown status. Infants in Michigan with an abnormal hemoglobin result on their NBS are referred to a hematologist for confirmatory testing and medical management, if needed. A child is classified as having confirmed SCD after receipt of disease confirmation by the NBS follow-up program.

      Michigan Department of Community Health. Michigan newborn screening annual report, 2009. Available at: http://www.michigan.gov/documents/mdch/NBSAnnualReport2009_final_338514_7.pdf. Accessed October 27, 2013.

      Children with normal hemoglobin results or abnormal hemoglobin results on their NBS but not SCD (eg, sickle cell trait, HbH disease) were classified as not SCD. Children without a documented NBS result in Michigan were classified as unknown status and excluded from further analysis.

      Testing of Alternative Case Definitions

      We tested alternative case definitions in 2 phases. The initial phase of testing occurred among children meeting eligibility criteria in 2010 who had Medicaid claims that reported an SCD diagnosis code. These children were linked to NBS results and included individuals with confirmed SCD as well as others who had been confirmed as not SCD cases. Measures of performance were calculated for each of the 37 candidate case definitions, including sensitivity, specificity, positive predictive value, and the area under the receiver operating characteristic (ROC) curve. We used the confirmed SCD diagnosis from Michigan NBS administrative records as the gold standard. On the basis of the results of these measures of performance, we identified 4 case definitions as the strongest candidates for additional investigation. In the second phase of testing, we conducted a subsequent validation of these 4 case definitions using a set of candidate SCD cases identified independently in 2011. Once again, we calculated performance measures to identify which case definitions provided the most accurate identification of children with SCD. In addition, we explored the changes in the predictive values of this final case definition by adding indicators of SCD-related medications to the definition: had hydroxyurea and/or received 300+ days of antibiotics from ages 0 to 5 years.

      Results

      Initial case definition testing was conducted among the subset of children continuously enrolled in Michigan Medicaid in 2010 who met the eligibility criteria (n = 1033). After linkage to NBS records, 605 (59%) were confirmed SCD, 333 (32%) were confirmed not SCD, and 95 (9%) were unknown (and subsequently excluded). Among children with confirmed SCD (n = 605), 49% were boys, 87% were black, and 61% were HbSS. The average (standard deviation) age was 9.7 (5.5) years (Table 3).
      Table 3Characteristics of Children With an SCD Claim in 2010 or 2011
      At least 1 Medicaid claim for SCD. SCD was determined by newborn screening.
      Characteristic20102011
      SCD (n = 605)No SCD (n = 333)SCD (n = 609)No SCD (n = 315)
      Sex
       Male299 (49%)180 (54%)301 (49%)160 (51%)
       Female306 (51%)153 (46%)308 (51%)155 (49%)
      Race
       Black527 (87%)216 (65%)521 (86%)194 (61%)
       White7 (1%)71 (21%)8 (1%)65 (21%)
       Other70 (12%)16 (5%)78 (13%)25 (8%)
       Hispanic1 (<1%)30 (9%)2 (<1%)31 (10%)
      Sickle cell subtype
       Hemoglobin SS372 (61%)357 (59%)
       Hemoglobin SC176 (29%)199 (33%)
       Hemoglobin sickle β-thalassemia56 (9%)52 (9%)
       Hemoglobin SE1 (<1%)1 (<1%)
      Age, y, mean (standard deviation)9.68 (5.47)7.11 (4.92)9.78 (5.52)6.77 (5.23)
      SCD = sickle cell disease.
      At least 1 Medicaid claim for SCD. SCD was determined by newborn screening.
      Measures of performance varied widely among the 37 case definitions (Table 4). Sensitivity ranged from 6.3% (definition 29) to 99.3% (definition 36), specificity from 52% (definition 36) to 99.7% (definitions 20, 21, 24, 27, 28), positive predictive value from 79% (definition 36) to 99.3% (definitions 26, 27), and area under the ROC curve from 0.53 (definition 29) to 0.92 (definition 6). The top 4 case definitions based on the measures of performance were definitions 1, 2, 6, and 37 (Figure).
      Table 4Measures of Performance for Case Definitions
      Children continuously enrolled in Michigan Medicaid in 2010 with either confirmed SCD or no SCD as identified by newborn screening records.
      Definition No.Area Under ROC Curve (95% CI)Sensitivity (%)Specificity (%)PPV (%)
      10.90 (0.88, 0.92)96.583.891.5
      20.91 (0.89, 0.93)90.791.395.0
      30.87 (0.85, 0.88)85.193.796.1
      40.87 (0.85, 0.89)79.095.296.8
      50.77 (0.74, 0.79)99.254.179.7
      60.92 (0.90, 0.94)93.789.594.2
      70.91 (0.89, 0.93)86.395.297.0
      80.87 (0.85, 0.89)77.595.596.9
      90.83 (0.81, 0.85)69.197.698.1
      100.80 (0.77, 0.82)92.167.083.5
      110.85 (0.83, 0.87)75.294.996.4
      120.77 (0.75, 0.79)55.797.998.0
      130.69 (0.67, 0.71)40.098.898.4
      140.77 (0.74, 0.79)69.883.588.5
      150.70 (0.68, 0.73)42.698.297.7
      160.63 (0.62, 0.65)27.899.198.2
      170.59 (0.57, 0.60)18.099.498.2
      180.75 (0.73, 0.77)51.798.598.4
      190.64 (0.62, 0.66)28.399.498.8
      200.58 (0.57, 0.60)16.999.799.0
      210.54 (0.53, 0.55)8.499.798.1
      220.74 (0.72, 0.77)52.696.196.1
      230.64 (0.63, 0.66)29.899.198.4
      240.59 (0.57, 0.60)17.599.799.1
      250.55 (0.54, 0.56)9.6
      No false-positive cases.
      No false-positive cases.
      260.72 (0.70, 0.74)44.399.499.3
      270.61 (0.60, 0.63)22.899.799.3
      280.56 (0.55, 0.57)12.699.798.7
      290.53 (0.52, 0.54)6.3
      No false-positive cases.
      No false-positive cases.
      300.83 (0.81, 0.85)67.997.698.1
      310.77 (0.75, 0.80)71.782.988.4
      320.81 (0.79, 0.84)80.082.389.1
      330.83 (0.80, 0.85)84.581.489.2
      340.86 (0.83, 0.88)92.179.389.0
      350.76 (0.73, 0.79)99.252.679.2
      360.76 (0.73, 0.78)99.352.079.0
      370.91 (0.89, 0.92)90.290.494.5
      ROC = receiver operating characteristic; CI = confidence interval; PPV = positive predictive value; SCD = sickle cell disease.
      Children continuously enrolled in Michigan Medicaid in 2010 with either confirmed SCD or no SCD as identified by newborn screening records.
      No false-positive cases.
      Figure thumbnail gr1
      FigureReceiver operating characteristic (ROC) curve of alternative case definitions to identify children with sickle cell disease using administrative claims, 2010. Sensitivity and specificity determined by using Michigan newborn screening results for children continuously enrolled in Michigan Medicaid in 2010.
      Further validation of these 4 case definitions was conducted among 997 children meeting eligibility criteria in 2011. In this second phase of validation, a total of 609 children (61%) were confirmed SCD, 315 (32%) were confirmed as not SCD, and 73 (7%) were unknown. Testing of these case definitions provided similar results to 2010, with case definition 2 (3 SCD claims in any position) emerging as the most accurate identification of children with SCD when compared to the gold standard of NBS (Table 5). We also considered the potential gain in accuracy that might be achieved by including the SCD-related medication categories (hydroxyurea and antibiotics prophylaxis) identified in Table 1. The addition of SCD-related medications to case definition 2 showed no appreciable improvement in accuracy and was not considered further.
      Table 5Measures of Performance for Top 4 Performing Case Definitions
      Children continuously enrolled in Michigan Medicaid in 2011 with either confirmed SCD or no SCD as identified by newborn screening records (n = 924).
      Definition NumberDescriptionArea Under the ROC Curve (95% Confidence Interval)Sensitivity (%)Specificity (%)PPV (%)
      1At least 2 SCD claims, any position0.88 (0.86, 0.90)94.981.090.6
      2At least 3 SCD claims, any position0.91 (0.89, 0.93)89.792.495.8
      6At least 2 Evaluation/Consultation SCD claims, any position0.90 (0.88, 0.92)92.687.993.7
      37At least 1 inpatient SCD claim, any position OR at least 2 SCD claims for either outpatient or ED, any position0.90 (0.88, 0.92)90.290.494.5
      ROC = receiver operating characteristic; CI = confidence interval; PPV = positive predictive value; SCD = sickle cell disease; ED = emergency department.
      Children continuously enrolled in Michigan Medicaid in 2011 with either confirmed SCD or no SCD as identified by newborn screening records (n = 924).

      Discussion

      In this study, we successfully developed and tested an administrative claims–based method to accurately identify children with SCD. To our knowledge, no such method has yet been systematically tested for the identification of SCD cases at the population level. We found that a count of 3 paid SCD claims in a year—irrespective of type of service—is the most accurate administrative claims–based case definition to identify children with SCD. We evaluated numerous alternative case definitions that explored a wide array of services and coding combinations, and none was better than the selected method. The final group of case definitions that we evaluated all had a relatively high degree of accuracy and area under the ROC curve. However, the SCD case definition ultimately selected through our analysis had both higher specificity and positive predictive value; it is also straightforward to implement. By using the selected case definition, SCD cases can be identified with a high degree of accuracy by finding individuals with 3 or more SCD claims annually. Notably, the selected case definition does not require the use of pharmacy claims, which can be substantially more labor intensive to analyze, and for which we observed no additional improvements in accuracy. The simplicity and accuracy of this approach suggests a high degree of feasibility for health plan quality of care assessments based solely on administrative claims data.
      Our findings are novel for the identification of SCD cases. Our findings build on similar approaches that have been used to validate claims-based case definitions for other chronic and acute conditions. Incident cases of breast cancer have previously been identified using a cancer registry as a gold standard; multiple logistic regression models that included claims-based predictors were investigated to estimate the probability of each model capturing a case.
      • Freeman J.L.
      • Zhang D.
      • Freeman D.H.
      • Goodwin J.S.
      An approach to identifying incident breast cancer cases using Medicare claims data.
      Similarly for pneumonia cases, claims-based definitions have been compared to medical charts in several different settings to identify the most appropriate algorithms for identification of cases.
      • Yu O.
      • Nelson J.C.
      • Bounds L.
      • Jackson L.A.
      Classification algorithms to improve the accuracy of identifying patients hospitalized with community-acquired pneumonia using administrative data.
      • Aronsky D.
      • Haug P.J.
      • Lagor C.
      • Dean N.C.
      Accuracy of administrative data for identifying patients with pneumonia.
      • Drahos J.
      • Vanwormer J.J.
      • Greenlee R.T.
      • et al.
      Accuracy of ICD-9-CM codes in identifying infections of pneumonia and herpes simplex virus in administrative data.
      Although our study is similar to these in the development of claims-based case definitions and their comparison to an assumed gold standard, the gold standard we used may provide a more accurate and valid mechanism to identify true cases. In this study, we used a genetic testing gold standard to validate our cases. In contrast, the gold standard source used in other studies has often involved medical record review or cases included in a disease registry; both of these methods are subject to incomplete case capture.
      • Freeman J.L.
      • Zhang D.
      • Freeman D.H.
      • Goodwin J.S.
      An approach to identifying incident breast cancer cases using Medicare claims data.
      • Yu O.
      • Nelson J.C.
      • Bounds L.
      • Jackson L.A.
      Classification algorithms to improve the accuracy of identifying patients hospitalized with community-acquired pneumonia using administrative data.
      • Aronsky D.
      • Haug P.J.
      • Lagor C.
      • Dean N.C.
      Accuracy of administrative data for identifying patients with pneumonia.
      Although a claims-driven investigation of the accuracy of alternative SCD case definitions has not previously been performed, a prior study did compare the accuracy of 1 claims-based SCD case definition to cases identified through NBS.
      • Halasa N.B.
      • Shankar S.M.
      • Talbot T.R.
      • et al.
      Incidence of invasive pneumococcal disease among individuals with sickle cell disease before and after the introduction of the pneumococcal conjugate vaccine.
      Additional studies have used different administrative claims methods to identify children SCD in the study population. Several studies have used various combinations of administrative claims; one was based on combinations of hospitalizations and outpatient visits,
      • Halasa N.B.
      • Shankar S.M.
      • Talbot T.R.
      • et al.
      Incidence of invasive pneumococcal disease among individuals with sickle cell disease before and after the introduction of the pneumococcal conjugate vaccine.
      • Amendah D.D.
      • Mvundura M.
      • Kavanagh P.L.
      • et al.
      Sickle cell disease–related pediatric medical expenditures in the US.
      • Sox C.M.
      • Cooper W.O.
      • Koepsell T.D.
      • et al.
      Provision of pneumococcal prophylaxis for publicly insured children with sickle cell disease.
      while another required only a single SCD-related claim.
      • Raphael J.L.
      • Dietrich C.L.
      • Whitmire D.
      • et al.
      Healthcare utilization and expenditures for low income children with sickle cell disease.
      • Ellison A.M.
      • Bauchner H.
      Socioeconomic status and length of hospital stay in children with vaso-occlusive crises of sickle cell disease.
      Such methods may introduce bias; our findings indicate that SCD case definitions such as these may lack sensitivity and miss cases, while other case definitions may not be sufficiently specific and consequently include children without SCD.
      This study has several limitations. The case definitions considered in our study were examined using administrative claims for one state's (Michigan) Medicaid program. Although differences exist between states' administrative claims systems, the coding methods used as the basis for this study are widely used and are common to most claims extracts, such as the Medicaid Analytic eXtract (MAX) files. We believe that Michigan is a favorable environment to test these methods for SCD because the incidence of SCD in Michigan is comparable to the national average rate. In Michigan, the incidence of HbSS was 0.24 per 1000 births (vs 0.26 nationally), 0.16 per 1000 births for HbSC (vs 0.14 nationally), and 0.04 per 1000 births for Hb β-thalassemia (vs 0.03 nationally).

      National Newborn Screening and Genetics Resource Center in Cooperation with the Maternal and Child Health Bureau Genetic Services Branch and the Association of Public Health Laboratories. National newborn screening 2006 incidence report. Available at: http://genes-r-us.uthscsa.edu/sites/genes-r-us/files/resources/genetics/2006datareport.pdf. Accessed January 28, 2013.

      Our study was limited to children insured by Medicaid with no other forms of health insurance. Although 70% of children born in Michigan with SCD (1987 to 2008) have a Medicaid ID, it is possible that health care utilization among children not fully insured by Medicaid may differ from those with private insurance. Importantly, the methods explored in this study are predicated on the completeness and accuracy of administrative claims paid for SCD-related health services; children who do not have SCD-related health care encounters or claims that are not accurately coded may reduce the specificity of this method. Furthermore, this method does not reflect potential differences in the accuracy of administrative claims to identify genetic variants of SCD. Finally, although the case definition developed in our study was shown to be accurate among children in Michigan Medicaid, the effectiveness for identification of cases of SCD over the age of 18 remains untested.
      The claims-based method for identifying children with SCD serves as an important foundation for studying multiple aspects of health among the population of children with SCD. The simplicity and accuracy of the method for SCD case identification developed in this study has multiple opportunities for application within the Pediatric Quality Measures Program (PQMP) as well as state Medicaid programs. The PQMP is aimed at expanding existing pediatric quality of care measures currently available for use by public and private health care purchasers through Children's Health Insurance Program Reauthorization Act of 2009 (CHIPRA). The methods explored in our study will enable several important quality of care measures aimed at key aspects of care for children with SCD, including appropriate use of antibiotics and annual Transcranial Doppler screening. Absent NBS data, we found that Medicaid administrative claims can be used to accurately identify SCD cases. It is important to note that the claims-based method developed in this study may be of value even in states where Medicaid claims for SCD cases can be identified by linkages with NBS data. Administrative claims–based case definitions can be used to identify SCD cases without a NBS result in state databases as a result of circumstances such as a child being born in a different state. In these cases, claims can accurately identify resident SCD cases that are enrolled in Medicaid, irrespective of the geographic location of their birth. Consequently, we believe that this approach will not only support SCD quality of care assessments but can also be instrumental in assessing the comparative effectiveness of SCD treatments regimens and in supporting ongoing surveillance of state SCD populations.

      Acknowledgment

      Supported in part by Agency for Healthcare Research and Quality (AHRQ) Cooperative Agreement U18HS020516.

      References

        • Hassell K.L.
        Population estimates of sickle cell disease in the US.
        Am J Prev Med. 2010; 38: S512-S521
        • Berg A.O.
        Sickle cell disease: screening, diagnosis, management, and counseling in newborns and infants. The Agency for Health Care Policy and Research.
        J Am Board Fam Pract. 1994; 7: 134-140
        • Lorey F.W.
        • Arnopp J.
        • Cunningham G.C.
        Distribution of hemoglobinopathy variants by ethnicity in a multiethnic state.
        Genet Epidemiol. 1996; 13: 501-512
        • Michlitsch J.
        • Azimi M.
        • Hoppe C.
        • et al.
        Newborn screening for hemoglobinopathies in California.
        Pediatr Blood Cancer. 2009; 52: 486-490
        • Stuart M.J.
        • Nagel R.L.
        Sickle-cell disease.
        Lancet. 2004; 364: 1343-1360
        • Gladwin M.T.
        • Vichinsky E.
        Pulmonary complications of sickle cell disease.
        N Engl J Med. 2008; 359: 2254-2265
      1. National Heart Lung and Blood Institute. The management of sickle cell disease. 2002. Available at: http://www.nhlbi.nih.gov/health/prof/blood/sickle/sc_mngt.pdf. Accessed October 27, 2013.

        • Shankar S.M.
        • Arbogast P.G.
        • Mitchel E.
        • et al.
        Medical care utilization and mortality in sickle cell disease: a population-based study.
        Am J Hematol. 2005; 80: 262-270
        • Bilenker J.H.
        • Weller W.E.
        • Shaffer T.J.
        • et al.
        The costs of children with sickle cell anemia: preparing for managed care.
        J Pediatr Hematol Oncol. 1998; 20: 528-533
        • Raphael J.L.
        • Dietrich C.L.
        • Whitmire D.
        • et al.
        Healthcare utilization and expenditures for low income children with sickle cell disease.
        Pediatr Blood Cancer. 2009; 52: 263-267
        • Ohene-Frempong K.
        • Weiner S.J.
        • Sleeper L.A.
        • et al.
        Cerebrovascular accidents in sickle cell disease: rates and risk factors.
        Blood. 1998; 91: 288-294
        • Vichinsky E.
        • Hurst D.
        • Earles A.
        • et al.
        Newborn screening for sickle cell disease: effect on mortality.
        Pediatrics. 1988; 81: 749-755
        • Gaston M.H.
        • Verter J.I.
        • Woods G.
        • et al.
        Prophylaxis with oral penicillin in children with sickle cell anemia. A randomized trial.
        N Engl J Med. 1986; 314: 1593-1599
        • Adams R.J.
        • McKie V.C.
        • Carl E.M.
        • et al.
        Long-term stroke risk in children with sickle cell disease screened with transcranial Doppler.
        Ann Neurol. 1997; 42: 699-704
        • Christakis D.A.
        • Feudtner C.
        • Pihoker C.
        • Connell F.A.
        Continuity and quality of care for children with diabetes who are covered by Medicaid.
        Ambul Pediatr. 2001; 1: 99-103
        • Lieu T.A.
        • Lozano P.
        • Finkelstein J.A.
        • et al.
        Racial/ethnic variation in asthma status and management practices among children in managed Medicaid.
        Pediatrics. 2002; 109: 857-865
        • Cotter J.J.
        • Smith W.R.
        • Rossiter L.F.
        • et al.
        Combining state administrative databases and provider records to assess the quality of care for children enrolled in Medicaid.
        Am J Med Qual. 1999; 14: 98-104
        • Finkelstein J.A.
        • Lozano P.
        • Farber H.J.
        • et al.
        Underuse of controller medications among Medicaid-insured children with asthma.
        Arch Pediatr Adolesc Med. 2002; 156: 562-567
        • Dombkowski K.J.
        • Wasilevich E.A.
        • Lyon-Callo S.K.
        Pediatric asthma surveillance using Medicaid claims.
        Public Health Rep. 2005; 120: 515-524
      2. Healthcare Cost and Utilization Project. Clinical Classifications Software (CCS) for ICD-9-CM. Available at: http://www.hcup-us.ahrq.gov/toolssoftware/ccs/ccs.jsp. Accessed September 26, 2013.

        • Yusuf H.R.
        • Atrash H.K.
        • Grosse S.D.
        • et al.
        Emergency department visits made by patients with sickle cell disease: a descriptive study, 1999–2007.
        Am J Prev Med. 2010; 38: S536-S541
        • Leschke J.
        • Panepinto J.A.
        • Nimmer M.
        • et al.
        Outpatient follow-up and rehospitalizations for sickle cell disease patients.
        Pediatr Blood Cancer. 2012; 58: 406-409
        • Ovbiagele B.
        • Adams R.J.
        Trends in comorbid sickle cell disease among stroke patients.
        J Neurol Sci. 2012; 313: 86-91
      3. National Committee for Quality Assurance. Healthcare Effectiveness Data and Information Set (HEDIS). Available at: http://www.ncqa.org/HEDISQualityMeasurement/HEDISMeasures/HEDIS2012.aspx. Accessed September 25, 2013.

      4. Centers for Medicare and Medicaid Services. Available at: http://www.cms.gov. Accessed September 26, 2013.

        • Korzeniewski S.J.
        • Grigorescu V.
        • Copeland G.
        • et al.
        Methodological innovations in data gathering: newborn screening linkage with live births records, Michigan, 1/2007–3/2008.
        Matern Child Health J. 2010; 14: 360-364
      5. Michigan Department of Community Health. Michigan newborn screening annual report, 2009. Available at: http://www.michigan.gov/documents/mdch/NBSAnnualReport2009_final_338514_7.pdf. Accessed October 27, 2013.

        • Freeman J.L.
        • Zhang D.
        • Freeman D.H.
        • Goodwin J.S.
        An approach to identifying incident breast cancer cases using Medicare claims data.
        J Clin Epidemiol. 2000; 53: 605-614
        • Yu O.
        • Nelson J.C.
        • Bounds L.
        • Jackson L.A.
        Classification algorithms to improve the accuracy of identifying patients hospitalized with community-acquired pneumonia using administrative data.
        Epidemiol Infect. 2011; 139: 1296-1306
        • Aronsky D.
        • Haug P.J.
        • Lagor C.
        • Dean N.C.
        Accuracy of administrative data for identifying patients with pneumonia.
        Am J Med Qual. 2005; 20: 319-328
        • Drahos J.
        • Vanwormer J.J.
        • Greenlee R.T.
        • et al.
        Accuracy of ICD-9-CM codes in identifying infections of pneumonia and herpes simplex virus in administrative data.
        Ann Epidemiol. 2013; 23: 291-293
        • Halasa N.B.
        • Shankar S.M.
        • Talbot T.R.
        • et al.
        Incidence of invasive pneumococcal disease among individuals with sickle cell disease before and after the introduction of the pneumococcal conjugate vaccine.
        Clin Infect Dis. 2007; 44: 1428-1433
        • Amendah D.D.
        • Mvundura M.
        • Kavanagh P.L.
        • et al.
        Sickle cell disease–related pediatric medical expenditures in the US.
        Am J Prev Med. 2010; 38: S550-S556
        • Sox C.M.
        • Cooper W.O.
        • Koepsell T.D.
        • et al.
        Provision of pneumococcal prophylaxis for publicly insured children with sickle cell disease.
        JAMA. 2003; 290: 1057-1061
        • Ellison A.M.
        • Bauchner H.
        Socioeconomic status and length of hospital stay in children with vaso-occlusive crises of sickle cell disease.
        J Natl Med Assoc. 2007; 99: 192-196
      6. Centers for Medicare and Medicaid Services. Available at: http://www.cms.gov/Research-Statistics-Data-and-Systems/Computer-Data-and-Systems/MedicaidDataSourcesGenInfo/MAXGeneralInformation.html. Accessed October 22, 2013.

      7. National Newborn Screening and Genetics Resource Center in Cooperation with the Maternal and Child Health Bureau Genetic Services Branch and the Association of Public Health Laboratories. National newborn screening 2006 incidence report. Available at: http://genes-r-us.uthscsa.edu/sites/genes-r-us/files/resources/genetics/2006datareport.pdf. Accessed January 28, 2013.