If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
To develop and test the accuracy of administrative claims method for identifying children with sickle cell disease (SCD) to enable quality of care assessments among children enrolled in Medicaid.
Methods
All administrative claims with an SCD diagnosis were obtained from Michigan Medicaid from 2008 to 2011 for children ≤18 years, representing 1828 individuals. All Medicaid claims were obtained for these children and classified into categories on the basis of SCD care; these classifications were used to develop 37 alternative case definitions for identifying children with SCD. Children with ≥1 SCD claim in 2010 or 2011 were identified as confirmed SCD or not SCD using the gold standard of Michigan newborn screening administrative records. Measures of performance were calculated for each case definition for eligible children in 2010. Further validation of the case definitions was performed among eligible children in 2011.
Results
In 2010, a total of 938 children met eligibility criteria and were linked to newborn screening records; 605 (59%) were confirmed SCD, and 333 (32%) were not SCD. Measures of performance varied among the 37 case definitions, and the 4 best case definitions on the basis of the sensitivity, specificity, and area under the receiver operating characteristic (ROC) curve were validated among 924 children meeting eligibility criteria in 2011. The case definition of at least 3 SCD claims in any position identified children with SCD with the most accuracy, with an area under the ROC curve of 0.91 (95% confidence interval 0.89, 0.93).
Conclusions
This definition can be used to facilitate a more accurate identification of children with SCD in future studies. Further investigation is necessary to determine whether this method translates to other populations besides Michigan Medicaid-insured children.
Sickle cell disease (SCD) is a chronic disease affecting mainly minority populations and is characterized by significant morbidity and mortality. SCD is estimated to currently affect 90,000 to 100,000 Americans (approximately 1 in 500 African American births), although variation exists among prevalence estimates.
SCD has multiple clinically significant forms, further complicating estimates of the true burden of disease. There are 6 sickle cell genotypes; however, the most common variants are sickle cell anemia (hemoglobin SS), hemoglobin (Hb) SC, HbS/β0-thalassemia, and HbS/β+-thalassemia.
Children with SCD are at risk for chronic symptoms that can seriously impact quality of life, including pain episodes, severe anemia, and pulmonary complications.
SCD can have devastating consequences among children if uncontrolled and can lead to potentially life-threatening complications. Children with SCD are 7 to 30 times more likely to be hospitalized, 2 to 6 times more likely to visit the emergency department, 300 times more likely to have a stroke, and 100 times more likely to develop pneumococcal infection; further, they have over 8 times the health care expenditures than their counterparts without SCD.
Given these risks, it is essential that children with SCD have effective follow-up immediately after birth and that preventive services are obtained throughout childhood.
At birth, all children are screened for SCD through state newborn screening (NBS) programs; these results could potentially enable identification of cases for ongoing quality of care assessments. However, state Medicaid programs may not have the technical capacity or policies established to authorize links between NBS results and administrative claims to support quality of care assessments. Absent the capability to establish these linkages, a claims-based definition is necessary to identify SCD cases. Although quality of care assessments that use administrative claims data have been previously developed for conditions such as asthma and diabetes,
a mechanism to identify SCD cases using claims has not been validated. If successful, a claims-based method offers important opportunities to evaluate population-based quality of care among children with SCD without requiring linkages to external data sources such as those maintained by state NBS programs. With that in mind, our objective was to develop and test the feasibility and accuracy of an administrative claims method for identifying children with SCD to enable quality of care assessments among children enrolled in Medicaid.
Methods
We developed and tested alternative methods for identifying children with SCD using Medicaid administrative claims data. We used a 5-step process that included: 1) acquisition of all Medicaid administrative claims for any child with at least 1 SCD claim; 2) classification of claims into meaningful groups relevant to SCD care; 3) development of alternative case definitions using these variables; 4) identification of the testing population to validate the accuracy of the alternative case definitions; and 5) test of the accuracy of the alternative case definitions to identify children with SCD.
Acquisition of Claims
In partnership with the Michigan Department of Community Health, we obtained all Medicaid administrative claims with a SCD ICD-9 diagnosis code for children 18 years or younger. All children were included, regardless of Medicaid enrollment status, during the period 2008 to 2011. Consistent with other studies and Agency for Healthcare Research and Quality (AHRQ) Healthcare Cost and Utilization Project Single-Level Clinical Classification Software (HCUP CCS), we included claims with ICD-9 diagnosis codes for HbSS (282.60, 282.61, 282.62), HbSC (282.63, 282.64), HbSD (282.68, 282.69), and HbS β-thalassemia (282.41 and 282.42); we did not include sickle cell trait (282.5) or other hemoglobinopathies.
A total of 66,274 SCD claims containing 304,289 revenue and/or procedure codes representing 1828 unique individuals were identified from 2008 to 2011. All Michigan Medicaid administrative claims data were acquired for these 1828 individuals for each year, including detailed enrollment characteristics (containing demographics and program eligibility information), provider information, and health services (including inpatient, outpatient, emergency department, and pharmacy). These data were linked to provide a comprehensive overview of paid services rendered to any child with an SCD claim. The claims tables included codes from all major health care coding schemes used to track patients and obtain reimbursement, including ICD-9-CM diagnosis codes, diagnosis-related groups, uniform billing (UB-92) codes, ICD-9-CM surgical codes, current procedural terminology (CPT) codes, Healthcare Common Procedure Coding System, and national drug codes. A de-duplication process was implemented, and claims were grouped together into events according to dates of service.
Classification of Claims
As a precursor to creating alternative case definitions, we classified individuals on the basis of claims history relevant to SCD care using the extracted SCD claims. Our classification approaches were derived from methodologies published by AHRQ, National Committee for Quality Assurance Healthcare Effectiveness Data and Information Set, and Centers for Medicare and Medicaid Services.
We started the process with a complete extraction of all SCD claims to maximize the likelihood that even diagnosis and procedure codes that were infrequently used would be given consideration in our approach. Interim results were shared among the team of investigators that included substantial expertise in Medicaid claims data analyses, chronic disease epidemiology, newborn screening, and statistical programming. Team members reviewed candidate case definitions based solely on one coding system (eg, an outpatient definition using CPT codes) and evaluated the incremental advantages or disadvantages of including other coding schemas (eg, an outpatient definition using both CPT and revenue codes). These considerations were aided by tabular frequency counts of the number of unique individuals and event counts for each code as well as visual representations (eg, Venn diagrams). These methods were jointly reviewed by team members to evaluate the degree of overlap between code groups and the unique contribution of each code group in capturing distinct individuals with SCD. We used an iterative approach to determine the degree to which each child had specific groups of claims representing meaningful categories of SCD care. We subsequently classified each child's claims from several perspectives, ranging from simple counts of SCD claims to claim counts on the basis of combinations of different SCD services.
Development of Alternative Case Definitions
From our analysis of SCD claims groupings, we identified 9 mutually exclusive claims categories: 7 health services categories (inpatient, outpatient, home health care, emergency department, blood transfusion), and 2 medication categories (antibiotic prophylaxis and hydroxyurea). In addition, 2 composite groups were formed: evaluation/consultation claims and an overall count of SCD claims (irrespective of type of service). Table 1 provides these categories and lists several additional categories that were considered but not included in the final case definitions. These categories served as the basis for the development of alternative case definitions to identify children with SCD from administrative claims. Table 2 illustrates the resultant 37 case definitions considered; the definitions reflect alternatives aimed at balancing inclusion of cases to maximize sensitivity with the addition of increasingly restrictive criteria to gain specificity. Alternative definitions were also considered on the basis of whether the diagnosis code for SCD was reported as the primary diagnosis or any mention of SCD for emergency department and inpatient claims.
Table 1Claims Classifications Included in Case Defintions
Category
Definition
SCD claim count
Healthcare Cost and Utilization Clinical Classification Software (HCUP CCS) #61 for ICD-9-CM codes (trait excluded)
CPT codes for ED were dropped due to low additional yield (6.9%) among the SCD cases, difficulty of processing claims lacking admission dates, discharge dates, and primary diagnosis.
CPT codes for Inpatient Hospitalization were dropped due to low additional yield (8.7%) among the SCD cases, difficulty of processing claims lacking admission dates, discharge dates, and primary diagnosis.
HCUP cost-center clusters: RBU, SCU, NUR where facility type is outpatient or inpatient hospital
Home health care
HCUP CCS #236 for procedure codes
Blood transfusion
HCUP CCS #222 and Centers for Medicare and Medicaid Services (CMS) cost-center cluster blood processing/transfusion
SCD = sickle cell disease; ED = emergency department; RBU = routine bed units; SCU = special care units; NUR = nursery.
Other categories created but not used in case definition: Observation Stays, SCD Screening and Confirmatory Testing, Chemistry/Hematology Lab Claims, and Transracial Doppler.
∗ Codes for clinic care and preventive medicine were dropped due to low additional yield (1.1%) among the SCD cases.
† CPT codes for ED were dropped due to low additional yield (6.9%) among the SCD cases, difficulty of processing claims lacking admission dates, discharge dates, and primary diagnosis.
‡ CPT codes for Inpatient Hospitalization were dropped due to low additional yield (8.7%) among the SCD cases, difficulty of processing claims lacking admission dates, discharge dates, and primary diagnosis.
All definitions list minimum number of required claims.
Definition Number
Criteria
All SCD Claims
1
2 claims, any position
2
3 claims, any position
3
4 claims, any position
4
5 claims, any position
Interview Evaluation/Consultation SCD Claims
5
1 claim, any position
6
2 claims, any position
7
3 claims, any position
8
4 claims, any position
9
5 claims, any position
Outpatient SCD Claims
10
1 claim, any position
11
2 claims, any position
12
3 claims, any position
13
4 claims, any position
Emergency Department SCD Claims
14
1 claim, any position
15
2 claims, any position
16
3 claims, any position
17
4 claims, any position
18
1 claim, primary position
19
2 claims, primary position
20
3 claims, primary position
21
4 claims, primary position
Inpatient Hospitalization SCD Claims
22
1 claim, any position
23
2 claims, any position
24
3 claims, any position
25
4 claims, any position
26
1 claim, primary position
27
2 claims, primary position
28
3 claims, primary position
29
4 claims, primary position
Combination of SCD Claims
30
At least 1 inpatient claim, primary position OR at least 1 ED claim, primary position OR at least 4 outpatient claims, any position
31
At least 1 inpatient claim, any position OR at least 1 ED claim, any position
32
At least 1 inpatient claim, any position OR at least 1 ED claim, any position OR at least 4 SCD outpatient claims, any position
33
At least 1 inpatient claim, any position OR at least 1 ED claim, any position OR at least 3 SCD outpatient claims, any position
34
At least 1 inpatient claim, any position OR at least 1 ED claim, any position OR at least 2 SCD outpatient claims, any position
35
At least 1 inpatient claim, any position OR at least 1 ED claim, any position OR at least 1 outpatient claim, any position
36
At least 1 inpatient claim, any position OR at least 1 ED claim, any position OR at least 1 outpatient claim, any position OR at least 1 home health care claim, any position OR at least 5 blood transfusions with a SCD diagnosis OR had at least 300 days of antibiotics (age 0-5 yr)
37
At least 1 inpatient claim, any position OR 2 claims for either outpatient or ED, any position
SCD = sickle cell disease; ED = emergency department.
∗ All definitions list minimum number of required claims.
We selected a subset of individuals initially identified in our SCD claims database to test the accuracy of the alternative case definitions. Our testing subset included children in 2010 to 2011 who were 1 to 18 years who had at least 1 Medicaid SCD claim in either year and had a Michigan NBS result available. NBS results were available for all children born in Michigan from 1987 to 2010; children born outside the state of Michigan were excluded. In addition, we required continuous enrollment in Michigan Medicaid with no other forms of health insurance during the year of the SCD claim. Eligible children were linked to Michigan birth certificates using child's name, birth date, and sex. Unlinked records were manually reviewed to attempt to locate a birth record. Birth records were subsequently linked to newborn screening (NBS) records maintained by the Michigan Department of Community Health using common variables.
Records that were not automatically linked were reviewed manually to identify additional matches. Using NBS records, children were classified as confirmed SCD (HbSS, HbS/β-thalassemia, HbSC, and other variants), no SCD, or unknown status. Infants in Michigan with an abnormal hemoglobin result on their NBS are referred to a hematologist for confirmatory testing and medical management, if needed. A child is classified as having confirmed SCD after receipt of disease confirmation by the NBS follow-up program.
Children with normal hemoglobin results or abnormal hemoglobin results on their NBS but not SCD (eg, sickle cell trait, HbH disease) were classified as not SCD. Children without a documented NBS result in Michigan were classified as unknown status and excluded from further analysis.
Testing of Alternative Case Definitions
We tested alternative case definitions in 2 phases. The initial phase of testing occurred among children meeting eligibility criteria in 2010 who had Medicaid claims that reported an SCD diagnosis code. These children were linked to NBS results and included individuals with confirmed SCD as well as others who had been confirmed as not SCD cases. Measures of performance were calculated for each of the 37 candidate case definitions, including sensitivity, specificity, positive predictive value, and the area under the receiver operating characteristic (ROC) curve. We used the confirmed SCD diagnosis from Michigan NBS administrative records as the gold standard. On the basis of the results of these measures of performance, we identified 4 case definitions as the strongest candidates for additional investigation. In the second phase of testing, we conducted a subsequent validation of these 4 case definitions using a set of candidate SCD cases identified independently in 2011. Once again, we calculated performance measures to identify which case definitions provided the most accurate identification of children with SCD. In addition, we explored the changes in the predictive values of this final case definition by adding indicators of SCD-related medications to the definition: had hydroxyurea and/or received 300+ days of antibiotics from ages 0 to 5 years.
Results
Initial case definition testing was conducted among the subset of children continuously enrolled in Michigan Medicaid in 2010 who met the eligibility criteria (n = 1033). After linkage to NBS records, 605 (59%) were confirmed SCD, 333 (32%) were confirmed not SCD, and 95 (9%) were unknown (and subsequently excluded). Among children with confirmed SCD (n = 605), 49% were boys, 87% were black, and 61% were HbSS. The average (standard deviation) age was 9.7 (5.5) years (Table 3).
Table 3Characteristics of Children With an SCD Claim in 2010 or 2011
Measures of performance varied widely among the 37 case definitions (Table 4). Sensitivity ranged from 6.3% (definition 29) to 99.3% (definition 36), specificity from 52% (definition 36) to 99.7% (definitions 20, 21, 24, 27, 28), positive predictive value from 79% (definition 36) to 99.3% (definitions 26, 27), and area under the ROC curve from 0.53 (definition 29) to 0.92 (definition 6). The top 4 case definitions based on the measures of performance were definitions 1, 2, 6, and 37 (Figure).
Table 4Measures of Performance for Case Definitions
FigureReceiver operating characteristic (ROC) curve of alternative case definitions to identify children with sickle cell disease using administrative claims, 2010. Sensitivity and specificity determined by using Michigan newborn screening results for children continuously enrolled in Michigan Medicaid in 2010.
Further validation of these 4 case definitions was conducted among 997 children meeting eligibility criteria in 2011. In this second phase of validation, a total of 609 children (61%) were confirmed SCD, 315 (32%) were confirmed as not SCD, and 73 (7%) were unknown. Testing of these case definitions provided similar results to 2010, with case definition 2 (3 SCD claims in any position) emerging as the most accurate identification of children with SCD when compared to the gold standard of NBS (Table 5). We also considered the potential gain in accuracy that might be achieved by including the SCD-related medication categories (hydroxyurea and antibiotics prophylaxis) identified in Table 1. The addition of SCD-related medications to case definition 2 showed no appreciable improvement in accuracy and was not considered further.
Table 5Measures of Performance for Top 4 Performing Case Definitions
∗ Children continuously enrolled in Michigan Medicaid in 2011 with either confirmed SCD or no SCD as identified by newborn screening records (n = 924).
In this study, we successfully developed and tested an administrative claims–based method to accurately identify children with SCD. To our knowledge, no such method has yet been systematically tested for the identification of SCD cases at the population level. We found that a count of 3 paid SCD claims in a year—irrespective of type of service—is the most accurate administrative claims–based case definition to identify children with SCD. We evaluated numerous alternative case definitions that explored a wide array of services and coding combinations, and none was better than the selected method. The final group of case definitions that we evaluated all had a relatively high degree of accuracy and area under the ROC curve. However, the SCD case definition ultimately selected through our analysis had both higher specificity and positive predictive value; it is also straightforward to implement. By using the selected case definition, SCD cases can be identified with a high degree of accuracy by finding individuals with 3 or more SCD claims annually. Notably, the selected case definition does not require the use of pharmacy claims, which can be substantially more labor intensive to analyze, and for which we observed no additional improvements in accuracy. The simplicity and accuracy of this approach suggests a high degree of feasibility for health plan quality of care assessments based solely on administrative claims data.
Our findings are novel for the identification of SCD cases. Our findings build on similar approaches that have been used to validate claims-based case definitions for other chronic and acute conditions. Incident cases of breast cancer have previously been identified using a cancer registry as a gold standard; multiple logistic regression models that included claims-based predictors were investigated to estimate the probability of each model capturing a case.
Similarly for pneumonia cases, claims-based definitions have been compared to medical charts in several different settings to identify the most appropriate algorithms for identification of cases.
Although our study is similar to these in the development of claims-based case definitions and their comparison to an assumed gold standard, the gold standard we used may provide a more accurate and valid mechanism to identify true cases. In this study, we used a genetic testing gold standard to validate our cases. In contrast, the gold standard source used in other studies has often involved medical record review or cases included in a disease registry; both of these methods are subject to incomplete case capture.
Although a claims-driven investigation of the accuracy of alternative SCD case definitions has not previously been performed, a prior study did compare the accuracy of 1 claims-based SCD case definition to cases identified through NBS.
Incidence of invasive pneumococcal disease among individuals with sickle cell disease before and after the introduction of the pneumococcal conjugate vaccine.
Additional studies have used different administrative claims methods to identify children SCD in the study population. Several studies have used various combinations of administrative claims; one was based on combinations of hospitalizations and outpatient visits,
Incidence of invasive pneumococcal disease among individuals with sickle cell disease before and after the introduction of the pneumococcal conjugate vaccine.
Such methods may introduce bias; our findings indicate that SCD case definitions such as these may lack sensitivity and miss cases, while other case definitions may not be sufficiently specific and consequently include children without SCD.
This study has several limitations. The case definitions considered in our study were examined using administrative claims for one state's (Michigan) Medicaid program. Although differences exist between states' administrative claims systems, the coding methods used as the basis for this study are widely used and are common to most claims extracts, such as the Medicaid Analytic eXtract (MAX) files.
We believe that Michigan is a favorable environment to test these methods for SCD because the incidence of SCD in Michigan is comparable to the national average rate. In Michigan, the incidence of HbSS was 0.24 per 1000 births (vs 0.26 nationally), 0.16 per 1000 births for HbSC (vs 0.14 nationally), and 0.04 per 1000 births for Hb β-thalassemia (vs 0.03 nationally).
National Newborn Screening and Genetics Resource Center in Cooperation with the Maternal and Child Health Bureau Genetic Services Branch and the Association of Public Health Laboratories. National newborn screening 2006 incidence report. Available at: http://genes-r-us.uthscsa.edu/sites/genes-r-us/files/resources/genetics/2006datareport.pdf. Accessed January 28, 2013.
Our study was limited to children insured by Medicaid with no other forms of health insurance. Although 70% of children born in Michigan with SCD (1987 to 2008) have a Medicaid ID, it is possible that health care utilization among children not fully insured by Medicaid may differ from those with private insurance. Importantly, the methods explored in this study are predicated on the completeness and accuracy of administrative claims paid for SCD-related health services; children who do not have SCD-related health care encounters or claims that are not accurately coded may reduce the specificity of this method. Furthermore, this method does not reflect potential differences in the accuracy of administrative claims to identify genetic variants of SCD. Finally, although the case definition developed in our study was shown to be accurate among children in Michigan Medicaid, the effectiveness for identification of cases of SCD over the age of 18 remains untested.
The claims-based method for identifying children with SCD serves as an important foundation for studying multiple aspects of health among the population of children with SCD. The simplicity and accuracy of the method for SCD case identification developed in this study has multiple opportunities for application within the Pediatric Quality Measures Program (PQMP) as well as state Medicaid programs. The PQMP is aimed at expanding existing pediatric quality of care measures currently available for use by public and private health care purchasers through Children's Health Insurance Program Reauthorization Act of 2009 (CHIPRA). The methods explored in our study will enable several important quality of care measures aimed at key aspects of care for children with SCD, including appropriate use of antibiotics and annual Transcranial Doppler screening. Absent NBS data, we found that Medicaid administrative claims can be used to accurately identify SCD cases. It is important to note that the claims-based method developed in this study may be of value even in states where Medicaid claims for SCD cases can be identified by linkages with NBS data. Administrative claims–based case definitions can be used to identify SCD cases without a NBS result in state databases as a result of circumstances such as a child being born in a different state. In these cases, claims can accurately identify resident SCD cases that are enrolled in Medicaid, irrespective of the geographic location of their birth. Consequently, we believe that this approach will not only support SCD quality of care assessments but can also be instrumental in assessing the comparative effectiveness of SCD treatments regimens and in supporting ongoing surveillance of state SCD populations.
Acknowledgment
Supported in part by Agency for Healthcare Research and Quality (AHRQ) Cooperative Agreement U18HS020516.
References
Hassell K.L.
Population estimates of sickle cell disease in the US.
Incidence of invasive pneumococcal disease among individuals with sickle cell disease before and after the introduction of the pneumococcal conjugate vaccine.
National Newborn Screening and Genetics Resource Center in Cooperation with the Maternal and Child Health Bureau Genetic Services Branch and the Association of Public Health Laboratories. National newborn screening 2006 incidence report. Available at: http://genes-r-us.uthscsa.edu/sites/genes-r-us/files/resources/genetics/2006datareport.pdf. Accessed January 28, 2013.
The authors declare that they have no conflict of interest.
Publication of this article was supported by the US Department of Health and Human Services and the Agency for Healthcare Research and Quality.
The views expressed in this article are those of the authors and do not necessarily represent the views of the US Department of Health and Human Services or the Agency for Healthcare Research and Quality.
00075-8&id=gr1.jpg){kind=link}