Advertisement

Toward Measurement for a Whole Child Health Policy: Validity and National and State Prevalence of the Integrated Child Risk Index

Open AccessPublished:December 07, 2021DOI:https://doi.org/10.1016/j.acap.2021.12.001

      Abstract

      Objective

      To develop, validate and estimate national and across state prevalence on a multidimensional index that assesses the complex medical, social, and relational health risks experienced by United States children.

      Methods

      Data from the National Survey of Children's Health were used to construct the Integrated Child Risk Index (ICRI) which includes medical health risk (MHR), social health risk (SHR) and relational health risk (RHR) domains. Confirmatory factor analysis and logistic regression analyses were employed to assess construct and predictive validity. Validity outcomes were child flourishing, school engagement/readiness, emergency room utilization and forgone care.

      Results

      Confirmatory factor analysis confirmed the ICRI 3-domain structure and greater correlation between MHR and RHR than MHR and SHR. Logistic regressions confirmed strong predictive validity of the ICRI for all study outcomes and ICRI scoring approaches. Nearly two-thirds of children (64.3%) with MHR also experienced SHR and/or RHR. Nearly one-third of United States children experienced risks on 2 or more ICRI domains and 15% of publicly insured children had risks on all domains (16.2%; 9.0%–25.7% across states). Significant variations were observed across states and by age, race/ethnicity, health insurance and household income.

      Conclusions

      The ICRI is a valid national and state level index associated with children's flourishing and educational preparedness and emergency and forgone care. National child health policies and Medicaid risk stratification and payment models should consider children's RHR in addition to SHR and MHR. Results call for integrated systems of care with the capacity to address medical, social and relational health risks and promote well-being. Substate and clinical applications require research.

      Keywords

      What's New
      The Integrated Child Risk Index assesses the complexity of children's health risks across medical, social and relational domains, predicts important outcomes and may help inform and evaluate policies to improve child health equity and establish integrated systems of care.
      The coronavirus pandemic has highlighted the reality that our children's physical, mental, social, and relational lives are inextricably intertwined and collectively promote or diminish both their early development and lifelong well-being.
      • Garner AS
      • Saul RA.
      Thinking Developmentally: Nurturing Wellness in Childhood to Promote Lifelong Health.
      • Marmot M.
      Social determinants of health inequalities.
      • Osher D
      • Cantor P
      • Berg J
      • et al.
      Drivers of human development: how relationships and context shape learning and development.
      • Child National Scientific Council on the Developing
      Connecting the Brain to the Rest of the Body: Early Childhood Development and Lifelong Health Are Deeply Intertwined: Working Paper No. 15.
      • Bronfenbrenner U
      • Morris PA.
      The Bioecological Model of Human Development.

      Sharfstein J. The Health of the Child and COVID-19. Milbank Quarterly Opinion [Internet]. 2020; (October 6).

      • Garner AS
      • Yogman M
      The American Academy of Pediatrics Committee on Psychosocial Aspects of Child and Family Health, the Section on Developmental and Behavioral Pediatrics, and the Council on Early Childhood.
      If nothing else, the disparities in morbidity and mortality unveiled by the pandemic confirm what our best science has told us for decades: physical and mental health do not occur in a vacuum and are strongly impacted by social inequities and the toxic stress and trauma that can arise in the presence of relational risks.
      • Marmot M.
      Social determinants of health inequalities.
      ,
      • Garner AS
      • Yogman M
      The American Academy of Pediatrics Committee on Psychosocial Aspects of Child and Family Health, the Section on Developmental and Behavioral Pediatrics, and the Council on Early Childhood.
      • Brenner M
      • Kidston C
      • Hilliard C
      • et al.
      Children's complex care needs: a systematic concept analysis of multidisciplinary language.

      Center on the Developing Child at Harvard University (2021). Three Principles to Improve Outcomes for Children and Families, 2021 Update. http://www.developingchild.harvard.edu. Accessed April 25, 2021 at 3Principles_Update2021v2.pdf.

      Socially, children must have basic needs met, like food,
      American Academy of Pediatrics Council on Community Pediatrics
      Committee on Nutrition.
      ,
      • Hatem C
      • Lee CY
      • Zhao X
      • et al.
      Food insecurity and housing instability during early childhood as predictors of adolescent mental health.
      safe housing,
      • Hatem C
      • Lee CY
      • Zhao X
      • et al.
      Food insecurity and housing instability during early childhood as predictors of adolescent mental health.
      ,
      • Fowler PJ
      • Farrell AF.
      Housing and child well being: implications for research, policy, and practice.
      and neighborhoods free from violence
      • Sheats KJ
      • Irving SM
      • Mercy JA
      • et al.
      Violence-related disparities experienced by black youth and young adults: opportunities for prevention.
      and racism.
      • Trent M
      • Dooley DG
      • Dougé J.
      The impact of racism on child and adolescent health.
      Relationally, healthy development requires the presence of safe, stable and nurturing relationships across all contexts where children learn, play and grow.
      • Garner AS
      • Saul RA.
      Thinking Developmentally: Nurturing Wellness in Childhood to Promote Lifelong Health.
      ,
      • Osher D
      • Cantor P
      • Berg J
      • et al.
      Drivers of human development: how relationships and context shape learning and development.
      ,
      • Child National Scientific Council on the Developing
      Connecting the Brain to the Rest of the Body: Early Childhood Development and Lifelong Health Are Deeply Intertwined: Working Paper No. 15.
      ,
      • Garner AS
      • Yogman M
      The American Academy of Pediatrics Committee on Psychosocial Aspects of Child and Family Health, the Section on Developmental and Behavioral Pediatrics, and the Council on Early Childhood.
      ,

      Center on the Developing Child at Harvard University (2021). Three Principles to Improve Outcomes for Children and Families, 2021 Update. http://www.developingchild.harvard.edu. Accessed April 25, 2021 at 3Principles_Update2021v2.pdf.

      ,
      • Jaffee SR
      • Bowes L
      • Ouellet-Morin I
      • et al.
      Safe, stable, nurturing relationships break the intergenerational cycle of abuse: a prospective nationally representative cohort of children in the United Kingdom.
      • Bethell CD
      • Jones J
      • Gombojav N
      • et al.
      Positive childhood experiences and adult mental and relational health in a statewide sample: associations across adverse childhood experiences levels.
      • Bethell CD
      • Gombojav N
      • Whitaker R.
      Family resilience and connection promote flourishing among US children, even amid adversity.
      • Chen Y
      • Kubzansky L
      VanderWeele T. Parental warmth and flourishing in mid-life.
      • Bethell CD
      • Newacheck P
      • Hawes E
      • et al.
      Adverse childhood experiences: assessing the impact on health and school engagement and the mitigating role of resilience.
      • Thomas PA
      • Liu H
      • Umberson D.
      Americans value the well-being of children.

      Baldari, C, Mathur, R.Study Shows Americans Agree: Poor Child Well-being is a Top Issue. First Focus: October 3, 2017. Accessed on April 25, 2021 at: https://firstfocus.org/blog/most-americans-see-child-poverty-as-top-concern-study-finds.

      Yet, the United States (US) remains persistently low in international rankings in overall child well-being.

      Organization of Economic and Community Development. How does United States compare on child well-being. November 2017. Accessed on August 12, 2020 at: http://www.oecd.org/els/family/CWBDP_Factsheet_USA.pdf.

      A synthesis of recent National Academy of Sciences expert reports reveals the urgent need for the US to prioritize policies that promote the health and well-being of its children, and each call for integrated and upstream strategies to address the constellation of child-, family-, and community-level risks that impact child health, rather than focusing on single risk factors.
      • Perrin JM
      • Duncan G
      • Diaz A
      • et al.
      Recommendations include: 1) collaborative efforts across health, education, and social services sectors; 2) team-based approaches to prevent and mitigate risks by proactively promoting child-, family-, and community-level protective factors; and 3) training, payment and performance measurement strategies focused on healthy child development and positive health and educational outcomes.
      • Perrin JM
      • Duncan G
      • Diaz A
      • et al.
      In turn, these efforts require integrated assessments that consider the complex array of children's physical, mental, social, and relational health risks associated with health and educational outcomes in order to assess, plan for and target services based on child and family needs.
      • Perrin JM
      • Duncan G
      • Diaz A
      • et al.
      • Counts N.Z.
      • Roiland R.A.
      • Halfon N.
      Proposing the ideal alternative payment model for children.
      • Alley DE
      • Ashford NC
      • Gavin AM.
      Payment innovations to drive improvements in pediatric care—the integrated care for kids model.
      • Newman N
      • Ferguson M
      • Dutton MJ
      • et al.
      In Pursuit of Whole Person Health: A Review of Social Determinants of health (SDOH) Initiatives in Medicaid Managed Care Contracts and 1115 Waivers.
      • Huffstetler AN
      • Phillips Jr., RL
      Payment structures that support social care integration with clinical care: social deprivation indices and novel payment models.

      Bethell CD, Kennedy S, Martinez-Vidal E. et al. Payment for progress: investing to catalyze child and family well-being using personalized and integrated strategies to address social and emotional determinants of health. CHA. November 2018. Accessed August 12, 2020 at: https://academyhealth.org/sites/default/files/payment_for_progress_fullreport_nov2018.pdf

      Despite growing consensus on the need for such integrated assessment approaches, prominent risk stratification and associated payment methods primarily focus narrowly on medical health risks (MHRs) – children's existing physical, mental or developmental medical diagnoses – that are associated with increased costs of medical care.
      • Perrin JM
      • Duncan G
      • Diaz A
      • et al.
      • Counts N.Z.
      • Roiland R.A.
      • Halfon N.
      Proposing the ideal alternative payment model for children.
      • Alley DE
      • Ashford NC
      • Gavin AM.
      Payment innovations to drive improvements in pediatric care—the integrated care for kids model.
      • Newman N
      • Ferguson M
      • Dutton MJ
      • et al.
      In Pursuit of Whole Person Health: A Review of Social Determinants of health (SDOH) Initiatives in Medicaid Managed Care Contracts and 1115 Waivers.
      • Huffstetler AN
      • Phillips Jr., RL
      Payment structures that support social care integration with clinical care: social deprivation indices and novel payment models.

      Bethell CD, Kennedy S, Martinez-Vidal E. et al. Payment for progress: investing to catalyze child and family well-being using personalized and integrated strategies to address social and emotional determinants of health. CHA. November 2018. Accessed August 12, 2020 at: https://academyhealth.org/sites/default/files/payment_for_progress_fullreport_nov2018.pdf

      Such narrow assessments are inherently reactive (vs preventive), biased against children with poorer access to diagnostic services and/or who experience negative impacts, perhaps due to social and relational health risks, but lack a physical, mental or developmental medical diagnoses. Predominant medical cost-driven and diagnostic-based methods typically do not account for the mitigatable community-level social health risks (SHR) that pose threats to children's health (eg, poverty,
      • Marmot M.
      Social determinants of health inequalities.
      ,
      • Pascoe JM
      • Wood DL
      • Duffee JH
      • et al.
      Mediators and adverse effects of child poverty in the United States.
      ,

      Braveman P, Heck K, Egerter S, et al. Economic hardship in childhood: a neglected issue in ACE studies? Matern Child Health J. 2018 Mar;22:308-317

      food insecurity,
      American Academy of Pediatrics Council on Community Pediatrics
      Committee on Nutrition.
      ,
      • Hatem C
      • Lee CY
      • Zhao X
      • et al.
      Food insecurity and housing instability during early childhood as predictors of adolescent mental health.
      exposure to community violence
      • Sheats KJ
      • Irving SM
      • Mercy JA
      • et al.
      Violence-related disparities experienced by black youth and young adults: opportunities for prevention.
      or racism
      • Trent M
      • Dooley DG
      • Dougé J.
      The impact of racism on child and adolescent health.
      ,
      • Obermeyer Z
      • Powers B
      • Vogeli C
      • et al.
      Dissecting racial bias in an algorithm used to manage the health of populations.
      ) or the family-level relational health risks (RHR) that threaten children's well-being (eg, Adverse Childhood Experiences [ACEs]
      • Garner AS
      • Saul RA.
      Thinking Developmentally: Nurturing Wellness in Childhood to Promote Lifelong Health.
      • Marmot M.
      Social determinants of health inequalities.
      • Osher D
      • Cantor P
      • Berg J
      • et al.
      Drivers of human development: how relationships and context shape learning and development.
      ,
      • Garner AS
      • Yogman M
      The American Academy of Pediatrics Committee on Psychosocial Aspects of Child and Family Health, the Section on Developmental and Behavioral Pediatrics, and the Council on Early Childhood.
      ,

      Center on the Developing Child at Harvard University (2021). Three Principles to Improve Outcomes for Children and Families, 2021 Update. http://www.developingchild.harvard.edu. Accessed April 25, 2021 at 3Principles_Update2021v2.pdf.

      ,
      • Jaffee SR
      • Bowes L
      • Ouellet-Morin I
      • et al.
      Safe, stable, nurturing relationships break the intergenerational cycle of abuse: a prospective nationally representative cohort of children in the United Kingdom.
      • Bethell CD
      • Jones J
      • Gombojav N
      • et al.
      Positive childhood experiences and adult mental and relational health in a statewide sample: associations across adverse childhood experiences levels.
      • Bethell CD
      • Gombojav N
      • Whitaker R.
      Family resilience and connection promote flourishing among US children, even amid adversity.
      • Chen Y
      • Kubzansky L
      VanderWeele T. Parental warmth and flourishing in mid-life.
      • Bethell CD
      • Newacheck P
      • Hawes E
      • et al.
      Adverse childhood experiences: assessing the impact on health and school engagement and the mitigating role of resilience.
      • Thomas PA
      • Liu H
      • Umberson D.
      ,
      • Felitti VJ
      • Anda RF
      • Nordenberg D
      • et al.
      Relationship of childhood abuse and household dysfunction to many of the leading causes of death in adults. The Adverse Childhood Experiences (ACE) Study.
      ,
      • Oh DL
      • Jerman P
      • Silvério Marques S
      • et al.
      Systematic review of pediatric health outcomes associated with childhood adversity.
      or caregivers who lack support or are distressed).
      • Osher D
      • Cantor P
      • Berg J
      • et al.
      Drivers of human development: how relationships and context shape learning and development.
      ,
      • Child National Scientific Council on the Developing
      Connecting the Brain to the Rest of the Body: Early Childhood Development and Lifelong Health Are Deeply Intertwined: Working Paper No. 15.
      ,
      • Sameroff AJ
      • Bartko WT
      • Baldwin A
      • et al.
      Family and social influences on the development of child competence.
      ,
      • Neece CL
      • Green SA
      • Baker BL.
      Parenting stress and child behavior problems: a transactional relationship across time.
      With the exception of interpersonal safety, a recent review of evidence on social risks screening and interventions conducted for the US Preventive Services Task Force limited the concept of social risks to income-associated factors.
      • Eder M
      • Henninger M
      • Durbin S
      • et al.
      Screening and interventions for social risk factors: technical brief to support the US Preventive Services Task Force.
      This review found evidence that interventions to address such risks may reduce health care costs but that few studies examined associations with health outcomes. Also found was lack of validation for most multidomain social risk screening tools and documentation that these tools – with the exception of interpersonal safety – do not include relational health risks, like ACEs or lack of social/emotional support. This also appears to be true for risk assessment approaches being employed in the Integrated Care for Kids (InCK) state demonstrations.
      • Newman N
      • Ferguson M
      • Dutton MJ
      • et al.
      In Pursuit of Whole Person Health: A Review of Social Determinants of health (SDOH) Initiatives in Medicaid Managed Care Contracts and 1115 Waivers.
      • Huffstetler AN
      • Phillips Jr., RL
      Payment structures that support social care integration with clinical care: social deprivation indices and novel payment models.

      Bethell CD, Kennedy S, Martinez-Vidal E. et al. Payment for progress: investing to catalyze child and family well-being using personalized and integrated strategies to address social and emotional determinants of health. CHA. November 2018. Accessed August 12, 2020 at: https://academyhealth.org/sites/default/files/payment_for_progress_fullreport_nov2018.pdf

      However, research documents independent associations between the health of adults and children and such relational health risks, even among those lacking income-associated social risks.
      • Bethell CD
      • Jones J
      • Gombojav N
      • et al.
      Positive childhood experiences and adult mental and relational health in a statewide sample: associations across adverse childhood experiences levels.
      ,
      • Felitti VJ
      • Anda RF
      • Nordenberg D
      • et al.
      Relationship of childhood abuse and household dysfunction to many of the leading causes of death in adults. The Adverse Childhood Experiences (ACE) Study.
      Further emphasizing the importance of considering relational health risks is a recent study revealing a 2.8 times higher rate of mental, emotional and behavioral health problems among US children when they experience 2-4 relational health risks but do not experience social health risks.
      • Bethell CD
      • Garner A
      • Blackwell CK
      • et al.
      Social and relational health risks and common mental health problems among US children: the mitigating role of family resilience and connection to promote socio-emotional and school outcomes.
      The importance of a distinct focus on preventing and addressing relational health risks is also strongly recommended by the American Academy of Pediatrics in its 2021 revised policy statement on childhood toxic stress that documents the foundational role of relational health to fostering healthy, resilient children, and recommends the proactive assessment and promotion of relational health for all children.
      • Garner AS
      • Yogman M
      The American Academy of Pediatrics Committee on Psychosocial Aspects of Child and Family Health, the Section on Developmental and Behavioral Pediatrics, and the Council on Early Childhood.
      In response to the gaps in research on integrated assessment approaches for children,
      • Eder M
      • Henninger M
      • Durbin S
      • et al.
      Screening and interventions for social risk factors: technical brief to support the US Preventive Services Task Force.
      • Bethell CD
      • Garner A
      • Blackwell CK
      • et al.
      Social and relational health risks and common mental health problems among US children: the mitigating role of family resilience and connection to promote socio-emotional and school outcomes.
      AAP Council on Community Pediatrics. Poverty and Child Health in the United States
      Capturing Social and Behavioral Domains and Measures in Electronic Health Records: Phase 2.
      • Yule K
      • Houston J
      • Grych J
      Resilience in children exposed to violence: a meta-analysis of protective factors across ecological contexts.
      • Traub F
      • Boynton-Jarrett R.
      Modifiable resilience factors to childhood adversity for clinical pediatric practice.
      this study aimed to 1) design and evaluate the construct validity of an index that integrates information about the complexity of the risks children experience across medical (physical/mental health problems and functional difficulties), social (economic hardship, food insecurity, community safety, racism), and relational (Adverse Childhood Experiences, parent/caregiver distress) health domains; 2) evaluate predictive validity of the index using positive health and health care utilization outcomes; 3) examine associations among risk domains to document their interacting and/or independent impacts; and 4) estimate national and across state prevalence across child subgroups.

      Methods

      Data and Population

      Data from the 2016–2019 National Survey of Children's Health (NSCH) were used.
      • Ghandour RM
      • Jones JR
      • Lebrun-Harris LA
      • et al.
      Bethell CD The Design and Implementation of the 2016 National Survey of Children’s Health.
      ,

      Child and Adolescent Health Measurement Initiative (CAHMI). 2016-2017 National Survey of Children's Health SPSS Codebook, Version 1.0. 2018. Available at: www.childhealthdata.org. Accessed November 5, 2020.

      The NSCH is an annual survey led by the US Health Resources and Services Administration's (HRSA) Maternal and Child Health Bureau (MCHB) in collaboration with the US Bureau of the Census.
      • Ghandour RM
      • Jones JR
      • Lebrun-Harris LA
      • et al.
      Bethell CD The Design and Implementation of the 2016 National Survey of Children’s Health.
      Unlike some state-level efforts to assess children's medical and social risks using system-level data,
      • Reuland CP
      • Collins J
      • Chiang L
      • et al.
      Oregon's approach to leveraging system-level data to guide a social determinants of health-informed approach to children's healthcare.
      the NSCH has the advantage of providing state data at the child level across numerous topics and enables stratification across a wide range of child characteristics. Here we used the combined 2016–2018 NSCH data (n = 102,341) to create and validate the Integrated Child Risk Index (ICRI), and then used the 2019 NSCH data (n = 29,344) to assess the reliability of study findings. Data were weighted to be representative of the nation and each state's population. Missing value rates were less than 3% for any individual item used in the study, which is well under the suggested 5%–10%.

      Bethell CD, Blackwell C, Gombojav N, et al. Technical Appendix for “Toward Measurement for a Whole Child Health Policy”: Development, Validation and Prevalence on the Integrated Child Risk Index. Accessible at:https://www.cahmi.org/docs/default-source/default-document-library/_academic-peds-icri-technical-appendix_2-4-22.pdf?sfvrsn=771a1714_0. Accessed December 3, 2021; 2021

      See Technical Appendix A1-A2 for details.

      Bethell CD, Blackwell C, Gombojav N, et al. Technical Appendix for “Toward Measurement for a Whole Child Health Policy”: Development, Validation and Prevalence on the Integrated Child Risk Index. Accessible at:https://www.cahmi.org/docs/default-source/default-document-library/_academic-peds-icri-technical-appendix_2-4-22.pdf?sfvrsn=771a1714_0. Accessed December 3, 2021; 2021

      Approach to the Design of the Integrated Child Risk Index (ICRI)

      The ICRI is based on the eco-bio-developmental model
      • Garner AS
      • Saul RA.
      Thinking Developmentally: Nurturing Wellness in Childhood to Promote Lifelong Health.
      of child health and builds on a body of work undertaken by the Child and Adolescent Health Measurement Initiative (CAHMI) and its leadership of the National Maternal and Child Health Measurement Research Network (MCH MRN) and the MCH MRN's Positive and Relational Health and Social Determinants of Health Technical Working Groups.

      Bethell, C, Johnson, K, Endsley Hrivnak, P, et al. National Maternal and Child Health Strategic Measurement Agenda, Child and Adolescent Health Measurement Initiative, Johns Hopkins University: August 2018. Accessed on November 5, 2020 at:https://www.cahmi.org/docs/default-source/resources/maternal-and-child-health-national-strategic-measurement-agenda-full-report.pdf?sfvrsn=fe258d7c_0

      ,

      Burner, C, Johnson, K, Bethell, C. Next steps in family-focused screening to address social determinants of health for young children in pediatric primary care: Report of the Social Determinants of Health (SDOH) Technical Working Group of the Maternal and Child Health Measurement Research Network (MCH-MRN). 2018. Accessed on Nov. 17, 2020 at:https://www.cahmi.org/docs/default-source/resources/next-steps-in-family-focused-screening-to-address-social-determinants-of-health-for-young-children-in-pediatric-primary-care-(2018).pdf?sfvrsn=8e6d882d_0

      The purpose of this initial work was to inform risk stratification approaches under consideration by states applying for funds to implement the Center for Medicare and Medicaid Services (CMS) Integrated Care for Kids (InCK) model.
      • Alley DE
      • Ashford NC
      • Gavin AM.
      Payment innovations to drive improvements in pediatric care—the integrated care for kids model.
      ,
      • Bethell C
      • Gombojav N
      Overview of the Whole Child Risk Index (WCRI): A summary measure to support statewide integrated systems of care efforts focused on whole child population health improvement.

      Bruner, C, Johnson, K, Bethell, C. Risk stratification for children in Medicaid: achieving the promise of prevention. April 2019. Accessed on December 8, 2020 at:https://www.inckmarks.org/docs/RiskStratificationandYoungChildrenFinalMay3pdf.pdf

      • Evans G
      • Li D
      • Whipple S.
      Cumulative risk and child development.
      ICRI development married cumulative risk (CR) and factor analytic approaches to create a child-level cumulative risk index with 3 distinct domains of risk (MHR, SHR, RHR).
      • Brumley LD
      • Brumley BP
      • Jaffee SR.
      Comparing cumulative index and factor analytic approaches to measuring maltreatment in the National Longitudinal Study of Adolescent to Adult Health.
      ,
      • Sameroff A
      • Seifer R
      • Barocas R
      • et al.
      Intelligence quotient scores of 4-year-old children: social-environmental risk factors.
      While CRs emphasize risk quantity and diversity and have higher predictive value across many different outcomes compared to individual risk variables, they lack specificity in terms of types of risks experienced. Alternatively, factor analyses capture risk intensity and differentiate between risk types through the identification of latent risk factors but are highly sample dependent.
      • Brumley LD
      • Brumley BP
      • Jaffee SR.
      Comparing cumulative index and factor analytic approaches to measuring maltreatment in the National Longitudinal Study of Adolescent to Adult Health.
      ,
      • Sameroff A
      • Seifer R
      • Barocas R
      • et al.
      Intelligence quotient scores of 4-year-old children: social-environmental risk factors.
      Our multifaceted approach adopts the simplicity of CR while leveraging the nationally and state level representative nature of the NSCH dataset to evaluate the ICRI factor structure and set forth national and across state prevalence.

      Selection of Individual Risk Measures Within ICRI Domains

      NSCH items and measures used to create individual ICRI measures were drawn from previously validated instruments and underwent the NSCH's standard mixed-method development process to optimize the face, construct, and convergent/divergent validity of items and multi-item NSCH measures.
      • Ghandour RM
      • Jones JR
      • Lebrun-Harris LA
      • et al.
      Bethell CD The Design and Implementation of the 2016 National Survey of Children’s Health.
      ,

      Child and Adolescent Health Measurement Initiative (CAHMI). 2016-2017 National Survey of Children's Health SPSS Codebook, Version 1.0. 2018. Available at: www.childhealthdata.org. Accessed November 5, 2020.

      To select individual measures for the ICRI, we built on Sameroff and colleague's approach, whereby risk measures were selected based on: 1) prominent models and measures with significant literature basis on the risk's impact on child health and development; and 2) high reliability of individual risk measures.
      • Sameroff AJ
      • Seifer R
      • McDonough SC.
      Contextual contributors to the assessment of infant mental health.
      ,
      • Kraemer HC
      • Lowe KK
      • Kupfer DJ.
      To Your Health.
      We used dichotomous high/low cutoff scores for each measure, with specific criteria based on values known to be predictive of child health and development outcomes or clinical/diagnostic criteria.
      • Sameroff AJ
      • Bartko WT
      • Baldwin A
      • et al.
      Family and social influences on the development of child competence.
      ,
      • McDowell I.
      Measuring Health: A Guide to Rating Scales and Questionnaires.
      Sensitivity analysis of individual items was conducted. When setting cutoffs, we took a conservative approach that erred on the side of positive predictive value over negative predictive value, such that if a child was positively identified on any measure, there would be unarguable evidence that the child was at risk (eg, children met cutoff criteria if they had “poor or fair” overall health status or “poor or fair” caregiver mental health status, even though “good” ratings often suggest children are at increased risk compared to those with “very good” and “excellent” ratings
      • Levinson D
      • Kaplan G.
      What does Self Rated Mental Health Represent.
      ,
      • Bethell CD
      • Blumberg SJ
      • Stein RE
      • et al.
      Taking stock of the CSHCN screener: a review of common questions and current reflections.
      ).

      Risk Domains

      A total of 12 health risk measures were included. Below, we provide an overview of each measure by risk domain. Due to space limitations, further details about each measure is provided in Technical Appendix A3-A5.
      • Ghandour RM
      • Jones JR
      • Lebrun-Harris LA
      • et al.
      Bethell CD The Design and Implementation of the 2016 National Survey of Children’s Health.

      Bethell CD, Blackwell C, Gombojav N, et al. Technical Appendix for “Toward Measurement for a Whole Child Health Policy”: Development, Validation and Prevalence on the Integrated Child Risk Index. Accessible at:https://www.cahmi.org/docs/default-source/default-document-library/_academic-peds-icri-technical-appendix_2-4-22.pdf?sfvrsn=771a1714_0. Accessed December 3, 2021; 2021

      The Medical Health Risk (MHR) domain includes 4 measures that identify children experiencing: 1) a more complex special health care need based on responses to the validated CSHCN Screener;
      • Berry JG
      • Hall M
      • Cohen E
      • et al.
      Ways to identify children with medical complexity and the importance of why.
      2) two or more of the chronic or life-long diagnosed conditions asked about in the NSCH;

      World Health Organization, International classification of functioning, disability and health: children & youth version: ICF-CY, 2007. Accessed on November 5, 2020 at: https://apps.who.int/iris/bitstream/handle/10665/43737/9789241547321_eng.pdf?sequence=1

      3) one or more of 11 frequent, chronic and/or serious functional difficulties aligned with domains included in the International Classification of Functioning for Children and Youth;

      Child and Adolescent Health Measurement Initiative, National Data Resource Center for Child and Adolescent Health interactive NSCH data query on child health status. Accessed November 5, 2020 at; https://www.childhealthdata.org/browse/survey/results?q=7608&r=1

      and 4) fair or poor overall health status.

      Bethell CD, Carle A, Hudziak J, et al. Methods to assess adverse childhood experiences of children and families: toward approaches to promote child well-being in policy and practice. Acad Pediatr. 2017 Sep 1;17:S51-69.

      The Social Health Risk (SHR) domain includes 4 measures that identify children whose caregivers reported that they: 1) sometimes or often could not afford enough food to eat;
      American Academy of Pediatrics Council on Community Pediatrics
      Committee on Nutrition.
      ,
      • Hatem C
      • Lee CY
      • Zhao X
      • et al.
      Food insecurity and housing instability during early childhood as predictors of adolescent mental health.
      2) somewhat often or very often found it hard to cover the costs of basics needs, including housing;
      • Pascoe JM
      • Wood DL
      • Duffee JH
      • et al.
      Mediators and adverse effects of child poverty in the United States.
      ,

      Braveman P, Heck K, Egerter S, et al. Economic hardship in childhood: a neglected issue in ACE studies? Matern Child Health J. 2018 Mar;22:308-317

      3) lived in an unsafe neighborhood or where the child was a victim of or witnessed violence;
      • Sheats KJ
      • Irving SM
      • Mercy JA
      • et al.
      Violence-related disparities experienced by black youth and young adults: opportunities for prevention.
      ,
      • Yule K
      • Houston J
      • Grych J
      Resilience in children exposed to violence: a meta-analysis of protective factors across ecological contexts.
      and 4) witnessed their child being treated or judged unfairly due to his or her race or ethnic group.
      • Trent M
      • Dooley DG
      • Dougé J.
      The impact of racism on child and adolescent health.
      ,
      • Obermeyer Z
      • Powers B
      • Vogeli C
      • et al.
      Dissecting racial bias in an algorithm used to manage the health of populations.
      The Relational Health Risk (RHR) domain includes 4 measures that identify children based on risks to the safety, stability and nurturing qualities of their relationships in the home. The RHR domains identifies children who: 1) experienced 2 or more of 6 household level adverse childhood experiences (ACEs) using the validated NSCH_ACEs indicator and cutoff shown to predict poorer outcomes;
      • Marie-Mitchell A
      • Watkins HBR
      • Copado IA
      • et al.
      Use of the Whole Child Assessment to identify children at risk of poor outcomes.
      ,
      • Schieve L
      • Boulet S
      • Kogan M
      • Yeargin-Allsopp M
      • Boyle C
      • Visser S
      • Blumberg S
      • Rice C.
      Parenting aggravation and autism spectrum disorders: 2007 National Survey of Children's Health.
      2) had 1 or 2 caregivers with fair/poor mental health;
      • Sameroff AJ
      • Bartko WT
      • Baldwin A
      • et al.
      Family and social influences on the development of child competence.
      • Neece CL
      • Green SA
      • Baker BL.
      Parenting stress and child behavior problems: a transactional relationship across time.
      • Eder M
      • Henninger M
      • Durbin S
      • et al.
      Screening and interventions for social risk factors: technical brief to support the US Preventive Services Task Force.
      • Bethell CD
      • Garner A
      • Blackwell CK
      • et al.
      Social and relational health risks and common mental health problems among US children: the mitigating role of family resilience and connection to promote socio-emotional and school outcomes.
      3) had a caregiver report frequent aggravation with their child;
      • Kuhn ES
      • Laird RD.
      Family support programs and adolescent mental health: review of evidence.
      and 4) had a caregiver who lacked emotional support or was not coping well.
      • Thomas PA
      • Liu H
      • Umberson D.
      ,
      • Neece CL
      • Green SA
      • Baker BL.
      Parenting stress and child behavior problems: a transactional relationship across time.
      ,
      • Billioux A
      • Verlander K
      • Anthony S
      • et al.
      Standardized Screening for Health-related Social Needs in Clinical Settings: The Accountable Health Communities Screening Tool, Discussion Paper.
      Together, the social and relational risk measures represent 11 of the 13 topics addressed in the Accountable Health Communities (AHC) Health-Related Social Needs (HRSN) Screening Tool and all but one of the topics addressed in the recent USPSTF evidence review.
      • Eder M
      • Henninger M
      • Durbin S
      • et al.
      Screening and interventions for social risk factors: technical brief to support the US Preventive Services Task Force.
      ,

      Dubowitz H.The Safe Environment for Every Kid model: Promotion of children's health, development, and safety, and prevention of child neglect. Ped Annals. 2014Nov;43:e271-7.

      Moore, KA, Bethell, CD, Murphy, D, et al. Flourishing from the start: What it is and How it Can be Measures. Child Trends. Mar 7, 2017. Accessed on November 5, 2020 at: https://www.childtrends.org/publications/flourishing-start-can-measured

      See Technical Appendix B for a comparison of ICRI measures and other social determinants of health assessments.

      Bethell CD, Blackwell C, Gombojav N, et al. Technical Appendix for “Toward Measurement for a Whole Child Health Policy”: Development, Validation and Prevalence on the Integrated Child Risk Index. Accessible at:https://www.cahmi.org/docs/default-source/default-document-library/_academic-peds-icri-technical-appendix_2-4-22.pdf?sfvrsn=771a1714_0. Accessed December 3, 2021; 2021

      Predictive Validity Outcome Measures

      Healthcare Utilization

      Hospital Emergency Room Utilization (ER) assesses whether children had at least one emergency room visit (ER) in the past 12 months based on caregiver response to the question, “During the past 12 months, how many times did this child visit a hospital emergency room?”

      Child and Adolescent Health Measurement Initiative (CAHMI). 2016-2017 National Survey of Children's Health SPSS Codebook, Version 1.0. 2018. Available at: www.childhealthdata.org. Accessed November 5, 2020.

      Forgone Care assesses whether a child had to forgo receiving needed healthcare in the past 12 months based on caregiver response to the question, “During the past 12 months, was there any time when this child needed health care, but it was not received?”

      Child and Adolescent Health Measurement Initiative (CAHMI). 2016-2017 National Survey of Children's Health SPSS Codebook, Version 1.0. 2018. Available at: www.childhealthdata.org. Accessed November 5, 2020.

      Positive Health and Educational Preparedness and Engagement

      Child Flourishing Index (CFI) evaluates whether children demonstrate characteristics associated with key flourishing constructs and was developed and validated for use in the NSCH.
      • Bethell CD
      • Gombojav N
      • Whitaker R.
      Family resilience and connection promote flourishing among US children, even amid adversity.
      ,
      • Moore KA
      • Murphy K
      • Bradley BA.
      Summary Report: The Development of a Pilot Recommended National Outcome Measure for Healthy and Ready to Learn.
      The CFI varied by child's age and assesses child curiosity and interest in learning, positive social and emotional characteristics, ability to focus and complete tasks and self-regulation. Using validated scoring methods, dichotomous indicators were created, where a score of 4 for 6-month to 5-year-olds and a score of 3 for 6-17-year-olds was coded as “1” (“flourishing”) and lesser scores were coded as “0” (“not flourishing”). Due to changes in the NSCH for CFI items between 2017 and 2018, only data from 2016 and 2017 was used in regression analyses using the CFI as the dependent variable. See Technical Appendix A5 for measurement details.

      Bethell CD, Blackwell C, Gombojav N, et al. Technical Appendix for “Toward Measurement for a Whole Child Health Policy”: Development, Validation and Prevalence on the Integrated Child Risk Index. Accessible at:https://www.cahmi.org/docs/default-source/default-document-library/_academic-peds-icri-technical-appendix_2-4-22.pdf?sfvrsn=771a1714_0. Accessed December 3, 2021; 2021

      Educational preparedness and engagement assesses whether or a not a child was prepared to engage in school (ages 3–5) or met criteria for school engagement (ages 6-17). Children ages 3–5 were classified as “prepared to engage in school” if their caregivers reported they were very or completely confident that their child was ready for or would be successful in school – an item highly correlated with a more robust school readiness indicator.

      Child and Adolescent Health Measurement Initiative (CAHMI). 2016-2017 National Survey of Children's Health SPSS Codebook, Version 1.0. 2018. Available at: www.childhealthdata.org. Accessed November 5, 2020.

      ,
      • Cohen J.
      Statistical Power Analysis for the Behavioral Sciences.
      School-age children (age 6–17) met criteria for school engagement if their caregivers reported always/definitely true or usually that their child “cares about doing well in school” and “does all required homework.”

      Child and Adolescent Health Measurement Initiative (CAHMI). 2016-2017 National Survey of Children's Health SPSS Codebook, Version 1.0. 2018. Available at: www.childhealthdata.org. Accessed November 5, 2020.

      Readiness to engage/school engagement was coded as “1” for meeting criteria and “0” for not meeting criteria. See Technical Appendix A5 for measurement details.

      Bethell CD, Blackwell C, Gombojav N, et al. Technical Appendix for “Toward Measurement for a Whole Child Health Policy”: Development, Validation and Prevalence on the Integrated Child Risk Index. Accessible at:https://www.cahmi.org/docs/default-source/default-document-library/_academic-peds-icri-technical-appendix_2-4-22.pdf?sfvrsn=771a1714_0. Accessed December 3, 2021; 2021

      Demographic Measures

      Demographic variables were measured using standard NSCH categories

      Child and Adolescent Health Measurement Initiative (CAHMI). 2016-2017 National Survey of Children's Health SPSS Codebook, Version 1.0. 2018. Available at: www.childhealthdata.org. Accessed November 5, 2020.

      and included: child age (0–5, 6–11, 12–17); sex (male = 1, female = 0); race and ethnicity (Hispanic, Non-Hispanic Black, Non-Hispanic White, Other/Multi-Racial, Asian, Non-Hispanic, American Indian/Alaska Native, Non-Hispanic, Native Hawaiian/Other Pacific Islander, Non-Hispanic); household income, calculated as a percentage of the federal poverty level (0%–99%, 100%–199%, 200%–399%, ≥ 400%); and type of health insurance (public, uninsured, private).

      Analytic Procedures

      Confirmatory Factor Analysis

      We first computed Pearson's correlations among all 12 ICRI measures to evaluate information redundancy using standard intervals established in the literature to evaluate the strength of correlations (r = 0, no correlation; r = below +/-0.10, low; r = +/-0.30, moderate; r ≥ +/-0.50, large; r = 1, perfect correlation).
      • Hu L
      • Bentler PM
      Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A.
      We then conducted a 3-factor CFA model allowing for correlated factors, where each factor represented one of the 3 risk domains, each comprised of 4 individual risk items. We compared this model with a unidimensional one (ie, all 12 items on one factor). We evaluated model fit using multiple fit indices with standard acceptable fit criteria: χ2, comparative fit index (CFI) and Tucker-Lewis Index (TLI) >0.95; root mean-square error of approximation (RMSEA) <0.08; and standardized root mean squared residual (SRMR) <0.08.
      • Hu L
      • Bentler PM
      Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A.
      • MacCallum RC
      • Zhang S
      • Preacher KJ
      • et al.
      On the practice of dichotomization of quantitative variables.
      • Kline RB.
      Principles and Practice of Structural Equation Modeling.
      • Vest JR
      • Menachemi N
      • Grannis SJ
      • et al.
      Impact of risk stratification on referrals and uptake of wraparound services that address social determinants: a stepped wedged trial.
      We further evaluated associations among ICRI domains by estimating prevalence of MHR by the SHR and RHR children experience.

      ICRI Scoring

      A goal for the ICRI was to create a dynamic instrument useful for epidemiological research, population health policymaking and management and with relevance as an assessment model for clinical practice. As such, we conducted a thorough investigation of potential scoring options and discuss the benefits and trade-offs for each one in Table 1. The 3 options include Domain Count (number of domains with at least one risk), Domain Combination (8 mutually exclusive categories of combinations of MHR, SHR, RHR) and Cumulative Risk (count of individual risks). Prevalence and validity analyses were conducted separately for each option and results were compared to understand the utility of each option in providing a robust and pragmatic scoring system for the ICRI.
      Table 1Description of ICRI Scoring Options and the Benefits and Tradeoffs of Each Option
      Scoring OptionDescriptionBenefitsTradeoffs/Solutions
      1. Domain CountTotal number of domains experienced (0–3), where higher values represent greater complexityStraightforward, single metricLacks specificity on which domains are experienced and the density of risks (ie, how many) within each domain. Can easily report domains and single items as needed.
      2. Domain CombinationsEight mutually-exclusive combinations of domains based on scores within any single domain:
      • -
        None (0 risks)
      • -
        Only medical (≥ 1 MHR, 0 SHR, 0 RHR)
      • -
        Only social (≥ 1 SHR, 0 MHR, 0 RHR)
      • -
        Only relational (≥ 1 RHR, 0 MHR, 0 SHR)
      • -
        Medical and social only (≥ 1 MHR, ≥1 SHR, 0 RHR)
      • -
        Medical and relational only (≥ 1 MHR, ≥ 1 RHR, 0 SHR)
      • -
        Social and relational only (≥ 1 SHR, ≥ 1 RHR, 0 MHR)
      • -
        Medical, social, and relational (≥ 1 MHR, ≥ 1 SHR, ≥ 1 RHR)
      Specific, enables identification of risk profilesRequires a large sample size to ensure sufficient power to assess combinations with lower prevalences.
      3. Cumulative RiskTotal number of individual risks a child experienced out of the 12 possible risks, collapsed into a categorical variable with a range of 0-5 or more given the skewness of the data (93% of children experienced fewer than 5 risks).Simple, intuitiveLacks complexity by not specifying which domains a child experienced. Lacks specificity as to which criteria are met

      Predictive Validity

      To evaluate the predictive validity of the ICRI, for each ICRI scoring option and study outcome we conducted logistic regressions using the ICRI as the primary predictor. For health care utilization, the reference category for ICRI scores was the lowest level of risk. For positive outcomes the reference category was the highest level of risk. Each model controlled for children's sex, age, race/ethnicity, household income, and insurance type. Results are presented as adjusted odds ratios [aORs] with 95% confidence intervals (95% CI).

      National and State Prevalence

      National and state prevalence was calculated for each ICRI scoring option, individual domain and risk measure and compared by child age, sex, race/ethnicity, income, and type of health insurance using chi squares tests of differences. Nested t-tests were employed to test the significance of differences between the prevalence of children in the nation versus each state who experience at least 2 ICRI domains. This was evaluated for all children and separately for children in each state who lived in households with incomes below 200% FPL to approximate prevalence for more economically vulnerable children. Finally, we compared findings using 2016–2018 NSCH with those using the 2019 NSCH as a final test of reliability.

      Results

      Confirmatory Factor Analysis Validation

      The 12 ICRI measures were not statistically redundant. Each of the 66 Pearson's correlations were significant (P < .05), yet all but 5 was low to moderate. Four sets of risk criteria exceeded the moderate correlation classification, and one correlation was large (r = 0.596; “more complex CSHCN” and “2+ chronic conditions”). See Technical Appendix A4.

      Bethell CD, Blackwell C, Gombojav N, et al. Technical Appendix for “Toward Measurement for a Whole Child Health Policy”: Development, Validation and Prevalence on the Integrated Child Risk Index. Accessible at:https://www.cahmi.org/docs/default-source/default-document-library/_academic-peds-icri-technical-appendix_2-4-22.pdf?sfvrsn=771a1714_0. Accessed December 3, 2021; 2021

      Results from the 3-factor CFA confirmed good model fit (χ2(51) = 1,560.14, P < .001; RMSEA = 0.02; SRMR = 0.08; CFI = 0.96; TLI = 0.95; Figure 1, Panel A). All paths were significant (P < .05), but the parameter estimate of caregiver emotional support/coping was low (β = 0.10) and accounted for a negligible amount of variance (R2 = 0.01, SE = 0.00). A revised CFA without this variable was conducted but the model fit was near identical (χ2(41) = 1,398.23, P < .001; RMSEA = 0.02; SRMR = 0.08; CFI = 0.97; TLI = 0.96), as were parameter estimate values for the remaining 11 risk items. We also examined a unidimensional model with all items loading on a single latent factor, but that fit was inferior (χ2(54) = 4385.47, P < .001; RMSEA = 0.03; SRMR = 0.11; CFI = 0.90; TLI = 0.88; Figure 1, Panel B). The SHR and RHR domains were highly correlated (r = 0.88), yet we retained the distinct risk domains given: 1) the qualitative difference between SHRs and RHRs and their potentially different intervention approaches (societal, familial, both), and 2) the higher correlation between RHR and MHR (r = 0.78) compared to SHR (r = 0.43) and potentially different associations with study outcomes. In addition, only 40.6% of children with RHR also experienced SHR such that independent impacts of RHR on MHR and study outcomes would be obscured by combining RHR with SHR.
      Figure 1
      Figure 1Standardized results from Confirmatory Factor Analyses (CFA) on the Integrated Child Risk Index (ICRI): Results from a 3 domain and unidimensional modela Data: National Survey of Children's Health. aWCCI=Whole Child Complexity Index; Diffs=1 or more functional difficulties; Conds=2 or more chronic conditions; Poorhlth= fair or poor overall health status; CSHCN=complex special healthcare needs; Discrim=discriminated against because of race/ethnicity; Food=sometimes/often could not afford food; Nbhdviol=unsafe neighborhood or witnessed/experienced violence; Hardship=difficulty meeting basic needs; Mental=caregiver poor/fair mental health status; Ace2=child experienced 2 or more ACEs; Aggrav=caregiver frequent aggravation or anger toward child; Cope=caregiver lacked emotional support/not coping well. b Path coefficients represent factor loading coefficients (standard errors) on the respective factors. All paths were significant at P < .01.
      The retainment of separate SHR and RHR domains was further corroborated by findings showing both independent and interacting associations between MHR and children's SHR and RHR. Specifically, as shown in Figure 2, prevalence of MHR was 24.9% for children with neither SHR nor RHR and was highest for children with 2–4 RHR and 2–4 SHR (69.9%). Yet, even with no SHR, 54.4% of children with 2–4 RHR had MHR and 65.2% with 2–4 RHR and only 1 SHR experienced MHR. Comparatively, 46.6% of children with 2–4 SHR and no RHR experienced MHR. All adjusted odds ratios were significant. See Technical Appendix C4

      Bethell CD, Blackwell C, Gombojav N, et al. Technical Appendix for “Toward Measurement for a Whole Child Health Policy”: Development, Validation and Prevalence on the Integrated Child Risk Index. Accessible at:https://www.cahmi.org/docs/default-source/default-document-library/_academic-peds-icri-technical-appendix_2-4-22.pdf?sfvrsn=771a1714_0. Accessed December 3, 2021; 2021

      for detailed regression findings showing all variations resulted in significant adjusted odds ratios.
      Figure 2
      Figure 2Prevalence of US children who experienced any ICRI Medical Health Risk (MHR) by their Relational Health Risk (RHR) and Social Health Risk (SHR) criteria count scores. Data: National Survey of Children's Health. Notes. All prevalence rates are weighted to represent the US child population. Adjusted Odds Ratios (aOR) are adjusted for age, sex, race/ethnicity, income and insurance coverage type. saORs are statistically significant after adjusting for age, sex, race/ethnicity, income and insurance coverage type.

      Predictive Validity

      Healthcare Utilization

      Children who experienced risks across more ICRI domains or individual criteria were systematically more likely to visit the emergency room and have forgone care in the past 12 months (Table 2). For example, children experiencing 2 domains were twice as likely to visit the emergency room (26.7% vs 13.4%; aOR: 2.05; 95% CI: 1.83–2.29) and 9.2 times more likely to have forgone care (4.6% vs 0.5%; aOR: 7.21; 95% CI: 5.01–10.36) compared to children without risks on any domain. The adjusted odds of forgone care were 6.12 (95% CI 3.77–9.93) times greater for children with only SHR and RHR (no MHR) compared to no risks.
      Table 2Associations Between Children Experiencing Each Study Outcome (Emergency Room Visits, Forgone Health Care, Child Flourishing, School Engagement/Ready to Engage in School) and Their Integrated Child Risk Index (ICRI) and ICRI Individual Medical, Social and Relational Health Risk Component Scores
      Data source: 2016-2018 (3 years combined) National Survey of Children's Health.
      All Children
      Data source: 2016-2018 (3 years combined) National Survey of Children's Health.
      Children Age 0–17 Who Experienced 1+ Hospital Emergency Room Visits in the Past 12 Months
      Data source: 2016-2018 (3 years combined) National Survey of Children's Health.
      Children Age 0–17 Who Experienced Forgone Health Care in the Past 12 Months
      Data source: 2016-2018 (3 years combined) National Survey of Children's Health.
      Children Age 6

      Months –17 Years Who Met Child Flourishing Index Criteria
      Data source: 2016-2017 (2 years combined) National Survey of Children's Health.
      Children Age 3–17 Who Met Criteria for Being Engaged (Age 6–17) or Caregiver Was Confident Child Was Prepared to Engage (age 3–5) in School
      Data source: 2016-2017 (2 years combined) National Survey of Children's Health.
      PercentPercentAdjusted Odds Ratio
      Adjusted Odds Ratios (aORs) are adjusted for age, sex, race/ethnicity, income and insurance coverage type. See Appendix A544 for detailed description of each study outcome variable. See Technical Appendix D144for the prevalence of US children who experienced Integrated Child Risk Index criteria by child responses to each outcome measure. See Technical Appendix D244 for the prevalence of study outcomes by child's age, race/ethnicity and household income and insurance status and type groups. See Technical Appendix D3-D944 for regression analysis details.
      PercentAdjusted Odds Ratio
      Adjusted Odds Ratios (aORs) are adjusted for age, sex, race/ethnicity, income and insurance coverage type. See Appendix A544 for detailed description of each study outcome variable. See Technical Appendix D144for the prevalence of US children who experienced Integrated Child Risk Index criteria by child responses to each outcome measure. See Technical Appendix D244 for the prevalence of study outcomes by child's age, race/ethnicity and household income and insurance status and type groups. See Technical Appendix D3-D944 for regression analysis details.
      PercentAdjusted Odds Ratio
      Adjusted Odds Ratios (aORs) are adjusted for age, sex, race/ethnicity, income and insurance coverage type. See Appendix A544 for detailed description of each study outcome variable. See Technical Appendix D144for the prevalence of US children who experienced Integrated Child Risk Index criteria by child responses to each outcome measure. See Technical Appendix D244 for the prevalence of study outcomes by child's age, race/ethnicity and household income and insurance status and type groups. See Technical Appendix D3-D944 for regression analysis details.
      PercentAdjusted Odds Ratio
      Adjusted Odds Ratios (aORs) are adjusted for age, sex, race/ethnicity, income and insurance coverage type. See Appendix A544 for detailed description of each study outcome variable. See Technical Appendix D144for the prevalence of US children who experienced Integrated Child Risk Index criteria by child responses to each outcome measure. See Technical Appendix D244 for the prevalence of study outcomes by child's age, race/ethnicity and household income and insurance status and type groups. See Technical Appendix D3-D944 for regression analysis details.
      All US children100%19.9%N/A3.1%N/A47.8%N/A69.1%N/A
      Experienced 1+ ICRI criteria63.7%23.5%1.74 (1.60–1.89)4.5%6.22 (4.47–8.65)40.1%Ref63.3%Ref
      Scoring Option 1: Domain Count: Number of ICRI domains experienced
        036.3%13.4%Ref0.5%Ref61.8%4.81 (3.98–5.81)80.6%4.87 (4.11–5.78)
        134.9%19.0%1.43 (1.30– 1.57)2.1%3.08 (2.11–4.48)48.5%3.08 (2.56–3.72)72.3%3.21 (2.74–3.77)
        220.0%26.7%2.05 (1.83–2.29)4.6%7.21 (5.01–10.36)34.9%1.84 (1.51–2.24)57.9%1.70 (1.44–2.01)
        38.8%34.5%2.92 (2.55–3.35)13.8%22.15 (15.50–31.65)21.2%Ref43.8%Ref
      Scoring Option 2: Domain Combinations: Mutually exclusive combinations
      No ICRI risks experienced36.3%13.4%Ref0.5%Ref61.8%4.59 (3.80–5.54)80.6%4.63 (3.90–5.48)
      Only Medical Health Risks12.0%23.4%2.03 (1.82–2.27)2.4%3.87 (2.57–5.84)41.3%2.12 (1.74–2.60)65.2%2.20 (1.83–2.63)
      Only Social Health Risks7.5%19.2%1.25 (1.07–1.46)2.4%3.63 (2.22–5.93)51.4%3.20 (2.59–3.96)73.5%3.29 (2.66–4.07)
      Only Relational Health Risks15.4%15.4%1.07 (0.92–1.23)1.8%2.24 (1.31– 3.84)52.2%3.73 (3.02–4.61)77.0%4.08 (3.38–4.93)
      Only Medical & Social Risks4.7%34.8%2.81 (2.35–3.35)5.7%9.65 (6.38–14.61)36.2%1.81 (1.41–2.34)55.6%1.51 (1.19–1.92)
      Only Medical & Relational Risks8.1%26.0%2.07 (1.81–2.38)4.1%6.64 (4.46–9.88)27.3%1.23 (0.98–1.55)50.5%1.26 (1.04–1.53)
      Only Social & Relational Risks7.2%22.1%1.45 (1.22–1.73)4.3%6.12 (3.77–9.93)41.2%2.57 (2.03–3.25)66.2%2.41 (1.94– 3.00)
      Medical, Social & Relational Risks8.8%34.5%2.80 (2.44–3.22)13.8%21.81 (15.26 –31.15)21.2%Ref43.8%Ref
      Scoring Option 3: Cumulative Risk: Number of ICRI criteria experienced
        036.3%13.4%Ref0.5%Ref61.8%7.12 (5.73–8.84)80.6%6.65 (5.53 –8.02)
        127.4%17.1%1.26 (1.13–1.40)1.9%2.52 (1.66–3.82)51.7%5.10 (4.10–6.35)75.8%5.20 (4.33 –6.25)
        215.0%23.2%1.75 (1.55–1.97)2.7%4.28 (2.78–6.58)41.5%3.45 (2.75–4.33)66.8%3.35 (2.75–4.08)
        3 to 414.3%29.1%2.45 (2.17–2.76)5.5%9.19 (6.43–13.15)30.2%2.27 (1.80–2.86)52.9%1.89 (1.57–2.28)
        5 to 127.1%37.8%3.54 (3.07–4.07)16.6%29.10 (20.27–41.77)15.0%Ref36.1%Ref
      Medical Health Risk ICRI Domain: criteria count
      aORs are not adjusted for other domains.
        066.4%15.5%Ref1.4%Ref56.0%2.53 (2.33–2.75)77.2%2.67 (2.45–2.91)
        1+33.6%28.6%2.10 (1.95–2.28)6.3%4.30 (3.46–5.34)31.5%Ref54.0%Ref
      Social Health Risk ICRI Domain: criteria count
      aORs are not adjusted for other domains.
        071.8%16.9%Ref1.5%Ref52.7%1.62 (1.47–1.78)73.8%1.77 (1.60–1.95)
        1+28.2%27.3%1.50 (1.37–1.64)7.0%4.08 (3.32–5.00)36.9%Ref58.9%Ref
      Relational Health Risk ICRI Domain: criteria count
      aORs are not adjusted for other domains.
        060.5%17.8%Ref1.5%Ref54.2%1.53 (1.41–1.67)74.4%1.59 (1.45–1.74)
        1+39.5%23.1%1.20 (1.10–1.30)5.5%2.74 (2.24–3.35)38.2%Ref61.7%Ref
      NOTES All percentages are weighted to represent the child population nationally and in each state.
      low asterisk Data source: 2016-2018 (3 years combined) National Survey of Children's Health.
      Data source: 2016-2017 (2 years combined) National Survey of Children's Health.
      Adjusted Odds Ratios (aORs) are adjusted for age, sex, race/ethnicity, income and insurance coverage type. See Appendix A5

      Bethell CD, Blackwell C, Gombojav N, et al. Technical Appendix for “Toward Measurement for a Whole Child Health Policy”: Development, Validation and Prevalence on the Integrated Child Risk Index. Accessible at:https://www.cahmi.org/docs/default-source/default-document-library/_academic-peds-icri-technical-appendix_2-4-22.pdf?sfvrsn=771a1714_0. Accessed December 3, 2021; 2021

      for detailed description of each study outcome variable. See Technical Appendix D1

      Bethell CD, Blackwell C, Gombojav N, et al. Technical Appendix for “Toward Measurement for a Whole Child Health Policy”: Development, Validation and Prevalence on the Integrated Child Risk Index. Accessible at:https://www.cahmi.org/docs/default-source/default-document-library/_academic-peds-icri-technical-appendix_2-4-22.pdf?sfvrsn=771a1714_0. Accessed December 3, 2021; 2021

      for the prevalence of US children who experienced Integrated Child Risk Index criteria by child responses to each outcome measure. See Technical Appendix D2

      Bethell CD, Blackwell C, Gombojav N, et al. Technical Appendix for “Toward Measurement for a Whole Child Health Policy”: Development, Validation and Prevalence on the Integrated Child Risk Index. Accessible at:https://www.cahmi.org/docs/default-source/default-document-library/_academic-peds-icri-technical-appendix_2-4-22.pdf?sfvrsn=771a1714_0. Accessed December 3, 2021; 2021

      for the prevalence of study outcomes by child's age, race/ethnicity and household income and insurance status and type groups. See Technical Appendix D3-D9

      Bethell CD, Blackwell C, Gombojav N, et al. Technical Appendix for “Toward Measurement for a Whole Child Health Policy”: Development, Validation and Prevalence on the Integrated Child Risk Index. Accessible at:https://www.cahmi.org/docs/default-source/default-document-library/_academic-peds-icri-technical-appendix_2-4-22.pdf?sfvrsn=771a1714_0. Accessed December 3, 2021; 2021

      for regression analysis details.
      § aORs are not adjusted for other domains.

      Positive Health and Educational Preparedness/Engagement

      Children with risks in fewer ICRI domains or on fewer individual ICRI measures were more likely to meet child flourishing index (CFI) and educational preparedness (3–5-year-olds) or school engagement (6–17-year-olds) criteria (Table 2). Compared to children experiencing all 3 risk domains, those without risks were about 3 times more likely to meet CFI criteria (61.8% vs 21.2%; aOR: 4.81, 95% CI 3.98–5.81) and nearly 2 times more likely to meet educational preparedness/engagement criteria (80.6% vs 43.8%; aOR 4.87, 95% CI 4.11–5.78).
      See Technical Appendix C1 and D6-D9 for detailed regression findings by ICRI subdomains.

      Bethell CD, Blackwell C, Gombojav N, et al. Technical Appendix for “Toward Measurement for a Whole Child Health Policy”: Development, Validation and Prevalence on the Integrated Child Risk Index. Accessible at:https://www.cahmi.org/docs/default-source/default-document-library/_academic-peds-icri-technical-appendix_2-4-22.pdf?sfvrsn=771a1714_0. Accessed December 3, 2021; 2021

      Of note, associations between study outcomes and race/ethnicity and household income were largely insignificant after adjusting for children's ICRI scores.

      National and Across State Prevalence on the Integrated Child Health Index

      Overall, 28.8% of all children and 46% of publicly insured children experienced risks on 2 or more ICRI domains, and 8.8% of all US children had risks on all 3 domains. This was 16.2% for publicly insured children with a range of 9.0% to 25.7% across states (Table 3, Scoring Option 1). For all ICRI scoring options, prevalence was higher for older, non-white, lower income, and publicly insured children (P < .05). Smaller differences were observed by child sex, with male children having generally higher rates of risk on the ICRI across all scoring options. This was driven by higher rates of MHR for male children. Prevalence of children experiencing risks on 2 or more ICRI domains ranged from 22.5% for White children to 43.2% for American Indian/Alaskan Native children, and from 47.1% for children with poverty level household incomes to 14.3% for children with household incomes 400% or greater than the Federal Poverty Level. See Table 3 and Technical Appendix C2 and C3.

      Bethell CD, Blackwell C, Gombojav N, et al. Technical Appendix for “Toward Measurement for a Whole Child Health Policy”: Development, Validation and Prevalence on the Integrated Child Risk Index. Accessible at:https://www.cahmi.org/docs/default-source/default-document-library/_academic-peds-icri-technical-appendix_2-4-22.pdf?sfvrsn=771a1714_0. Accessed December 3, 2021; 2021

      Table 3National and Across-state Prevalence of Children Experiencing Integrate Child Risk Index (ICRI) Criteria for Each ICRI Scoring Option and on Each of the 3 ICRI Domains (Medical, Social and/or Relational Health Risk): by Child's Insurance Status/type and Race/ethnicity
      Data source: 2016-2018 (3 years combined) National Survey of Children's Health.
      ,
      Variations within child subgroups are all statistically significant. Statistical significance of differences is set at P < .05 and is based on tests of independence comparing the percentage of children meeting ICRI scoring option criteria by children's age, household income and race/ethnicity.
      All Children, Age 0-17 YearsChild Insurance Status and TypeState Range for Publicly Insured Children (%)Child Race/Ethnicity
      Publicly Insured
      Includes children with any public health insurance, even if they also had some form of private insurance as well.
      Privately InsuredUninsuredHispanicWhite, Non-HispanicBlack-Non-HispanicAsian, Non-HispanicAmerican Indian/Alaska Native, Non-HispanicNative Hawaiian/Other Pacific Islander, Non-HispanicMulti-race, Non-Hispanic
      All US children100%35.9%57.8%6.3%N/A24.9%51.1%13.2%4.6%0.4%0.1%5.0%
      Experienced 1+ WCCI criteria63.7%81.1%51.6%73.5%72.3–88.174.7%55.0%74.3%69.4%71.8%81.3%65.3%
      Scoring Option 1: Domain Count: Number of domains in which any risks are experienced
        036.3%18.9%48.4%26.5%11.9–27.725.3%45.0%25.7%30.6%28.2%18.7%34.7%
        134.9%35.2%34.2%38.3%24.9–41.538.5%32.4%35.1%46.6%28.5%38.9%31.0%
        220.0%29.7%13.4%23.2%21.4–35.525.6%15.8%25.1%17.7%29.8%28.2%23.2%
        38.8%16.2%3.9%12.0%9.0–25.710.5%6.7%14.1%5.2%13.4%14.2%
      Estimate has a 95% confidence interval width exceeding 30 percentage points or 1.3 times the estimate and may not be reliable. See Appendix Part C3 for prevalence for each individual ICRI criteria.
      11.1%
      Scoring Option 2: Domain Combinations: Mutually exclusive groups
      No ICRI risks experienced36.3%18.9%48.4%26.5%11.9–27.725.3%45.0%25.7%30.6%28.2%18.7%34.7%
      Only Medical Health Risks12.0%8.5%14.6%8.1%3.5
      Estimate has a 95% confidence interval width exceeding 30 percentage points or 1.3 times the estimate and may not be reliable. See Appendix Part C3 for prevalence for each individual ICRI criteria.
      14.8
      8.8%14.2%10.9%9.8%7.3%8.4%
      Estimate has a 95% confidence interval width exceeding 30 percentage points or 1.3 times the estimate and may not be reliable. See Appendix Part C3 for prevalence for each individual ICRI criteria.
      11.1%
      Only Social Health Risks7.5%9.6%6.1%8.2%6.3–19.27.0%6.9%10.5%4.4%10.1%
      Estimate has a 95% confidence interval width exceeding 30 percentage points or 1.3 times the estimate and may not be reliable. See Appendix Part C3 for prevalence for each individual ICRI criteria.
      9.0%
      Estimate has a 95% confidence interval width exceeding 30 percentage points or 1.3 times the estimate and may not be reliable. See Appendix Part C3 for prevalence for each individual ICRI criteria.
      9.8%
      Only Relational Health Risks15.4%17.1%13.6%22.1%7.9–25.622.7%11.3%13.7%32.3%11.0%21.5%
      Estimate has a 95% confidence interval width exceeding 30 percentage points or 1.3 times the estimate and may not be reliable. See Appendix Part C3 for prevalence for each individual ICRI criteria.
      10.2%
      Only Medical & Social Risks4.7%7.3%3.2%3.2%4.1–10.55.2%3.9%7.2%2.5%5.5%10.5%
      Estimate has a 95% confidence interval width exceeding 30 percentage points or 1.3 times the estimate and may not be reliable. See Appendix Part C3 for prevalence for each individual ICRI criteria.
      5.5%
      Only Medical & Relational Risks8.1%10.1%6.6%9.1%6.0–16.210.2%7.0%8.0%9.1%9.9%4.1%
      Estimate has a 95% confidence interval width exceeding 30 percentage points or 1.3 times the estimate and may not be reliable. See Appendix Part C3 for prevalence for each individual ICRI criteria.
      8.2%
      Only Social & Relational Risks7.2%12.3%3.6%11.0%7.5–16.710.3%4.9%9.9%6.0%14.5%13.7%
      Estimate has a 95% confidence interval width exceeding 30 percentage points or 1.3 times the estimate and may not be reliable. See Appendix Part C3 for prevalence for each individual ICRI criteria.
      9.4%
      Medical, Social & Relational Risks8.8%16.2%3.9%12.0%9.0–25.710.5%6.7%14.1%5.2%13.4%14.2%
      Estimate has a 95% confidence interval width exceeding 30 percentage points or 1.3 times the estimate and may not be reliable. See Appendix Part C3 for prevalence for each individual ICRI criteria.
      11.1%
      Scoring Option 3: Criterion Count: Number of ICRI criteria experienced
        036.3%18.9%48.4%26.5%11.9–27.725.3%45.0%25.7%30.6%28.2%18.7%34.7%
        127.4%27.1%26.8%33.1%16.3–32.832.4%24.6%26.1%41.2%16.4%30.7%22.4%
        215.0%19.0%12.0%17.8%14.2–26.918.2%12.5%17.9%15.4%17.8%15.3%
      Estimate has a 95% confidence interval width exceeding 30 percentage points or 1.3 times the estimate and may not be reliable. See Appendix Part C3 for prevalence for each individual ICRI criteria.
      16.2%
        3 to 414.3%21.5%9.7%13.9%15.9–34.016.5%12.2%17.9%10.5%24.0%26.4%16.6%
        5 to 127.1%13.5%3.0%8.7%6.1–20.47.5%5.6%12.3%2.4%13.6%8.9%
      Estimate has a 95% confidence interval width exceeding 30 percentage points or 1.3 times the estimate and may not be reliable. See Appendix Part C3 for prevalence for each individual ICRI criteria.
      10.2%
      Medical Health Risk ICRI Domain: criteria count
        1+33.6%42.2%28.4%31.9%28.4–55.034.5%31.9%40.1%26.9%36.8%36.8%36.0%
        066.4%57.8%71.6%68.1%45.0–71.665.5%68.1%59.9%73.1%63.2%63.2%64.0%
      Social Health Risk ICRI Domain: criteria count
        1+28.2%45.4%16.8%34.3%37.9–57.133.0%22.5%41.7%18.2%43.6%47.3%35.8%
        071.8%54.6%83.2%65.7%42.9–62.167.0%77.5%58.3%81.8%56.4%52.7%64.2%
      Relational Health Risk ICRI Domain: criteria count
        1+39.5%55.5%27.7%53.6%44.0–65.453.6%29.8%45.7%52.7%48.9%53.4%38.8%
        060.5%44.5%72.3%46.4%34.6–56.046.4%70.2%54.3%47.3%51.1%46.6%61.2%
      low asterisk Data source: 2016-2018 (3 years combined) National Survey of Children's Health.
      Variations within child subgroups are all statistically significant. Statistical significance of differences is set at P < .05 and is based on tests of independence comparing the percentage of children meeting ICRI scoring option criteria by children's age, household income and race/ethnicity.
      Includes children with any public health insurance, even if they also had some form of private insurance as well.
      § Estimate has a 95% confidence interval width exceeding 30 percentage points or 1.3 times the estimate and may not be reliable. See Appendix Part C3 for prevalence for each individual ICRI criteria.
      The prevalence of children who experienced risks on 2 or more ICRI domains ranged from 23.3% to 38.4% across US states. This range was 35.6%–53.2% among children who lived in households with incomes less than 200% Federal Poverty Level. See Figure 3. See Technical Appendix E2

      Bethell CD, Blackwell C, Gombojav N, et al. Technical Appendix for “Toward Measurement for a Whole Child Health Policy”: Development, Validation and Prevalence on the Integrated Child Risk Index. Accessible at:https://www.cahmi.org/docs/default-source/default-document-library/_academic-peds-icri-technical-appendix_2-4-22.pdf?sfvrsn=771a1714_0. Accessed December 3, 2021; 2021

      for detailed findings.
      Figure 3
      Figure 3US map showing the prevalence of children experiencing 2 or more ICRI domains by state among children age 0–17 who live in households with incomes that are below 200% of the Federal Poverty Level (FPL)a Nationwide: 42.2%; State range: 35.6% (HI) - 53.2% (RI). aICRI domains: Medical Health Risks, Social Health Risks, Relational Health Risks; bShading indicates results of nested t-tests comparing each state to the nation (see legend).

      Comparison with 2019 National Survey of Children's Health

      Predictive validity and prevalence findings on the ICRI domain count scoring option were consistent using either the 2016–2018 NSCH or the 2019 NSCH data. Overall prevalence rates were no more than 0.5% lower in 2019 across all domain count score categories (0, 1, 2, 3). The small differences found were driven by changes to the wording of the “economic hardship” measure in the 2019 NSCH – changes that led to fewer children being identified as experiencing this social health risk. See Technical Appendix E4

      Bethell CD, Blackwell C, Gombojav N, et al. Technical Appendix for “Toward Measurement for a Whole Child Health Policy”: Development, Validation and Prevalence on the Integrated Child Risk Index. Accessible at:https://www.cahmi.org/docs/default-source/default-document-library/_academic-peds-icri-technical-appendix_2-4-22.pdf?sfvrsn=771a1714_0. Accessed December 3, 2021; 2021

      for detailed findings.

      Discussion

      This study sought to develop a parsimonious and intuitive child-level risk index that acknowledges the complexity of children's medical, social, and relational health risks. To achieve this, we leveraged the practicality of cumulative risk indices and the granularity of factor analytic techniques to allow for risk domains to be evaluated separately and in combination with each other. While still a data reduction approach, the ICRI allows for the possibility of different associations among risk domains and provides more specificity about the types of risks versus just total quantity of individual risks. The ICRI domain count scoring option showed strong validity consistent with that found for the cumulative risk scoring option yet may be more intuitive. The ICRI provides valid national and state level information about the complexity of children's risks associated with positive health, educational preparedness/engagement and emergency and forgone care. This information can be used to understand needs, document and track inequities, tailor and compare services quality and outcomes across child subgroups and help guide the design of integrated systems of care in states. The NSCH is collected annually, enabling the ICRI to track the impact of policies and innovations aiming to reduce the medical, social and relational health risks children experience and to promote health equity, including the implementation of integrated models of care and utilization of comprehensive, family centered primary care medical homes. Substate ICRI findings (eg, county or city) using the NSCH can be estimated using synthetic estimation methods. States can also elect to increase the NSCH sample in their state to enable more robust geographic and/or child subgroup analysis and tracking over time.
      The ICRI complements emerging state-level approaches to assess complex risks of children using system-level data, like that set forth in Oregon.
      • Reuland CP
      • Collins J
      • Chiang L
      • et al.
      Oregon's approach to leveraging system-level data to guide a social determinants of health-informed approach to children's healthcare.
      Interestingly, our Oregon ICRI findings using the NSCH were consistent with findings from the Oregon systems-level data algorithm. The ICRI identified 15.8% of Oregon's publicly insured children with risks on all 3 of its medical, social and relational health domains whereas the Oregon algorithm identified 22.1% of such children meeting both its medical and social risk criteria. Here, the ICRI identified fewer children, perhaps because it uses a higher bar to qualify for risk and was potentially more granular due to use of child-level, person-reported data collection methods. In addition, variables we recommend the ICRI be stratified by (eg, limited English proficiency) are included as risks in the Oregon model.
      Whereas prior research on comprehensive child risk assessment has been limited by sample-dependency,
      • Brumley LD
      • Brumley BP
      • Jaffee SR.
      Comparing cumulative index and factor analytic approaches to measuring maltreatment in the National Longitudinal Study of Adolescent to Adult Health.
      we developed the ICRI using the large, nationally representative 2016–2018 NSCH combined data sample of US children (0–17 years) and confirmed generalizability with a second nationally representative sample using the 2019 NSCH data. Based on its psychometric properties and our validity findings, we conclude that the ICRI is a robust, holistic assessment tool providing important information at the national and state level. Including 12 measures with statistically distinct contributions, the ICRI can be considered a minimal item index. With its multiple scoring options, the ICRI provides rich information to understand levels and differences in needs across states and subgroups of children based on their demographic characteristics, insurance type and status, and the numerous health care quality, education and community variables also included in the NSCH.
      Findings have 4 key implications for promoting the health of children. First, results provide empirical support for assessing RHR separately rather than combining RHR and SHR into a single domain. Nearly 60% of children with RHR did not experience SHR (59.4%). Yet, compared to children with no RHR, these children with RHR and no SHR had a prevalence of MHR that was 1.26–2.18 times greater depending on whether they had only 1 or 2–4 RHR. (Figure 2). RHR and SHR also require tailored intervention approaches, even when they are co-occurring. For instance, interventions to address food insecurity alone will differ from those addressing the presence of food insecurity among children who also experience ACEs and parents that are distressed. Second, with nearly two-thirds (63.4%) of all US children with MHR also experiencing RHR and SHR, results urge that RHR and SHR be assessed for all children with MHR. These risks not only contribute to the presence and severity of the numerous physical, mental, developmental health problems and functional difficulties assessed in the ICRI but can also hinder child and family capacities to access needed services or effectively respond to treatment recommendations.
      • Mann C
      • Eder J.
      Don't forget the kids: care transformations that meet the needs of children.
      Third, results call for efforts to address SHR and RHR even when they do not co-occur with MHR because they are independently associated with poor outcomes in health, educational preparedness/engagement and emergency and foregone care. Finally, study findings show that the absence of risks on the ICRI does not automatically imply the presence of positive outcomes-and vice versa. For example, 38.2% of children with no ICRI risks did not meet flourishing criteria. And while children with risks on all 3 ICRI domains had 79% lower adjusted odds of flourishing compared to children with no risks, 21.2% of these children still met flourishing criteria. (Table 3) Overall, this study adds to existing evidence
      • Bethell CD
      • Jones J
      • Gombojav N
      • et al.
      Positive childhood experiences and adult mental and relational health in a statewide sample: associations across adverse childhood experiences levels.
      ,
      • Bethell CD
      • Gombojav N
      • Whitaker R.
      Family resilience and connection promote flourishing among US children, even amid adversity.
      supporting a strengths-based approach to assessment that includes risks but also considers children's protective factors and positive health, like those measured in the child flourishing and school preparedness indicators used in this study.

      Limitations

      This study has several limitations. First, analyses use cross-sectional data preventing confirmation of causal relationships between the ICRI and study outcomes. Yet, the nationally representative NSCH data used here draw on previously validated measures and/or have undergone standardized NSCH validity analysis. The NSCH also provides important measurement generalizability and epidemiological insights which are not feasible to do longitudinally on such a large scale. Nonetheless, improvements to measures used may improve the ICRI, especially in the area of relational health risk assessment. Second, the ICRI may underestimate the prevalence of children's risks because: 1) measures represent higher levels of risk (eg, children with 2+ ACEs or more complex special health care needs, etc.), and 2) positive caregiver reporting bias may lower prevalence of risks.

      World Health Organization, International classification of functioning, disability and health: children & youth version: ICF-CY, 2007. Accessed on November 5, 2020 at: https://apps.who.int/iris/bitstream/handle/10665/43737/9789241547321_eng.pdf?sequence=1

      Conclusions

      The ICRI is a valid indicator associated with health care access and utilization and positive outcomes in US children. Available national and state ICRI data may inform existing and emerging policy efforts to build and finance the integrated systems of care required to reduce children's complex health risks, foster health equity and promote child, family and community well-being.
      • Garner AS
      • Yogman M
      The American Academy of Pediatrics Committee on Psychosocial Aspects of Child and Family Health, the Section on Developmental and Behavioral Pediatrics, and the Council on Early Childhood.
      ,
      • Perrin JM
      • Duncan G
      • Diaz A
      • et al.
      ,
      • Counts N.Z.
      • Roiland R.A.
      • Halfon N.
      Proposing the ideal alternative payment model for children.
      ,

      Bethell CD, Kennedy S, Martinez-Vidal E. et al. Payment for progress: investing to catalyze child and family well-being using personalized and integrated strategies to address social and emotional determinants of health. CHA. November 2018. Accessed August 12, 2020 at: https://academyhealth.org/sites/default/files/payment_for_progress_fullreport_nov2018.pdf

      ,
      • Bethell C
      • Gombojav N
      Overview of the Whole Child Risk Index (WCRI): A summary measure to support statewide integrated systems of care efforts focused on whole child population health improvement.
      ,

      The Healthcare Transformation Task Force and the Public Health Leadership Forum, Partnering to Catalyze Comprehensive Community Wellness: An actionable framework for health care and public health collaboration. 2020. Accessed on Dec 22 2020 at: https://hcttf.org/wp-content/uploads/2018/06/Comprehensive-Community-Wellness-Report.pdf

      • McCabe MA
      • Leslie L
      • Counts N
      • et al.
      Pediatric integrated primary care as the foundation for healthy development across the lifespan.
      • Brewster AL
      • Fraze TK
      • Gottlieb LM
      • et al.
      The role of value-based payment in promoting innovation to address social risks: a cross-sectional study of social risk screening by US physicians.
      • Murray SD
      • Hurley J
      • Ahmed SR.
      Supporting the whole child through coordinated policies, processes, and practices.
      • Teisberg E
      • Wallace S
      • O'Hara S
      Defining and implementing value-based health care: a strategic framework.
      • Mann C
      • Ferguson M.
      Caring for the Whole Child: A New Way to Finance Initiatives to Improve Children's Health and Well-being. Issue Brief.
      Massachusetts Child and Adolescent Health Initiative
      Moving to the Vanguard on Pediatric Care: Child and Adolescent Health Initiative Recommendations for the MassHealth Section 1115.
      • Bruner C.
      Medicaid, Managed Care, and Value-Based Payment Systems: State Roles in Contracting for Healthy Child Development.
      • Wolfe I
      • Satherley RM
      • Scotney E
      • et al.
      Integrated care models and child health: a meta-analysis.
      Additional studies are required to evaluate local and clinical applications. Studies are forthcoming to evaluate the impact of malleable mitigating factors to protect against the complex risks experienced by many US children today.

      Acknowledgments

      The authors acknowledge Phillip Sherlock, PhD, for his consultation in analytic procedures.
      Financial statement: Authors acknowledge funding from the Robert Wood Johnson Foundation through a grant (#75448) to Johns Hopkins University and the Health Resources and Services Administration grant to Johns Hopkins University (Grant Number: UA6MC30375).
      Authorship Statement: While MB Davis is an employee of the RWJF, neither RWJF nor HRSA officially participated in the development or completion of this manuscript.

      Appendix. Supplementary Data

      References

        • Garner AS
        • Saul RA.
        Thinking Developmentally: Nurturing Wellness in Childhood to Promote Lifelong Health.
        American Academy of Pediatrics, Itasca, IL2020
        • Marmot M.
        Social determinants of health inequalities.
        Lancet. 2005; 365: 1099-1104
        • Osher D
        • Cantor P
        • Berg J
        • et al.
        Drivers of human development: how relationships and context shape learning and development.
        Applied Developmental Science. 2020; 24: 6-36
        • Child National Scientific Council on the Developing
        Connecting the Brain to the Rest of the Body: Early Childhood Development and Lifelong Health Are Deeply Intertwined: Working Paper No. 15.
        Center on the Developing Child at Havard University, Cambridge, MA2020
      1. (In:)
        • Bronfenbrenner U
        • Morris PA.
        The Bioecological Model of Human Development.
        (editors)in: Lerner RM Damon W Handbook of Child Psychology: Theoretical Models of Human Development. John Wiley & Sons Inc, Hoboken, NJ2006: 793-828
      2. Sharfstein J. The Health of the Child and COVID-19. Milbank Quarterly Opinion [Internet]. 2020; (October 6).

        • Garner AS
        • Yogman M
        The American Academy of Pediatrics Committee on Psychosocial Aspects of Child and Family Health, the Section on Developmental and Behavioral Pediatrics, and the Council on Early Childhood.
        Preventing Childhood Toxic Stress: Partnering With Families and Communities to Promote Relational Health. Pediatrics. 2021 (in press)
        • Brenner M
        • Kidston C
        • Hilliard C
        • et al.
        Children's complex care needs: a systematic concept analysis of multidisciplinary language.
        Eur J Pediatr. 2018; 177: 1641-1652
      3. Center on the Developing Child at Harvard University (2021). Three Principles to Improve Outcomes for Children and Families, 2021 Update. http://www.developingchild.harvard.edu. Accessed April 25, 2021 at 3Principles_Update2021v2.pdf.

        • American Academy of Pediatrics Council on Community Pediatrics
        Committee on Nutrition.
        Promoting Food Security for All Children Pediatrics, Oct 2015, peds: 2015-3301
        • Hatem C
        • Lee CY
        • Zhao X
        • et al.
        Food insecurity and housing instability during early childhood as predictors of adolescent mental health.
        J Fam Psychol. 2020; 34: 721-730
        • Fowler PJ
        • Farrell AF.
        Housing and child well being: implications for research, policy, and practice.
        Am J Community Psychol. 2017; 60: 3-8https://doi.org/10.1002/ajcp.12154
        • Sheats KJ
        • Irving SM
        • Mercy JA
        • et al.
        Violence-related disparities experienced by black youth and young adults: opportunities for prevention.
        Am J Prev Med. 2018; 55: 462-469
        • Trent M
        • Dooley DG
        • Dougé J.
        The impact of racism on child and adolescent health.
        Pediatrics. 2019; 144:e20191765https://doi.org/10.1542/peds.2019-1765
        • Jaffee SR
        • Bowes L
        • Ouellet-Morin I
        • et al.
        Safe, stable, nurturing relationships break the intergenerational cycle of abuse: a prospective nationally representative cohort of children in the United Kingdom.
        J Adolesc Health. 2013; 53: S4-10
        • Bethell CD
        • Jones J
        • Gombojav N
        • et al.
        Positive childhood experiences and adult mental and relational health in a statewide sample: associations across adverse childhood experiences levels.
        JAMA pediatrics. 2019; 173e193007-e
        • Bethell CD
        • Gombojav N
        • Whitaker R.
        Family resilience and connection promote flourishing among US children, even amid adversity.
        Health Aff. 2019; 38: 729-737
        • Chen Y
        • Kubzansky L
        VanderWeele T. Parental warmth and flourishing in mid-life.
        Soc Sci Med. 2019; 220: 65-72
        • Bethell CD
        • Newacheck P
        • Hawes E
        • et al.
        Adverse childhood experiences: assessing the impact on health and school engagement and the mitigating role of resilience.
        Health Aff (Millwood). 2014; 33: 2106-2115
        • Thomas PA
        • Liu H
        • Umberson D.
        Family Relationships and Well-being. 1. Innov Again, 2017https://doi.org/10.1093/geroni/igx025 (Nov2017 Nov 11)
      4. Baldari, C, Mathur, R.Study Shows Americans Agree: Poor Child Well-being is a Top Issue. First Focus: October 3, 2017. Accessed on April 25, 2021 at: https://firstfocus.org/blog/most-americans-see-child-poverty-as-top-concern-study-finds.

      5. Organization of Economic and Community Development. How does United States compare on child well-being. November 2017. Accessed on August 12, 2020 at: http://www.oecd.org/els/family/CWBDP_Factsheet_USA.pdf.

        • Perrin JM
        • Duncan G
        • Diaz A
        • et al.
        Principles And Policies To Strengthen Child And Adolescent Health And Well-Being: Study describes National Academies of Sciences, Engineering, and Medicine reports on poverty, mental, emotional, and behavioral health, adolescence, and young family health and education. 39. Health Affairs, 2020: 1677-1683
        • Counts N.Z.
        • Roiland R.A.
        • Halfon N.
        Proposing the ideal alternative payment model for children.
        JAMA pediatrics. 2021; 175: 669-670
        • Alley DE
        • Ashford NC
        • Gavin AM.
        Payment innovations to drive improvements in pediatric care—the integrated care for kids model.
        JAMA Pediatr. 2019; 173: 717-718https://doi.org/10.1001/jamapediatrics.2019.1703
        • Newman N
        • Ferguson M
        • Dutton MJ
        • et al.
        In Pursuit of Whole Person Health: A Review of Social Determinants of health (SDOH) Initiatives in Medicaid Managed Care Contracts and 1115 Waivers.
        Manatt, Phelps, & Phillips, LLP, New York, NY2020
        • Huffstetler AN
        • Phillips Jr., RL
        Payment structures that support social care integration with clinical care: social deprivation indices and novel payment models.
        Am J Prev Med. 2019; 57 (1S82-S8)
      6. Bethell CD, Kennedy S, Martinez-Vidal E. et al. Payment for progress: investing to catalyze child and family well-being using personalized and integrated strategies to address social and emotional determinants of health. CHA. November 2018. Accessed August 12, 2020 at: https://academyhealth.org/sites/default/files/payment_for_progress_fullreport_nov2018.pdf

        • Pascoe JM
        • Wood DL
        • Duffee JH
        • et al.
        Mediators and adverse effects of child poverty in the United States.
        Pediatrics. 2016; 137:e20160340https://doi.org/10.1542/peds.2016-0340
      7. Braveman P, Heck K, Egerter S, et al. Economic hardship in childhood: a neglected issue in ACE studies? Matern Child Health J. 2018 Mar;22:308-317

        • Obermeyer Z
        • Powers B
        • Vogeli C
        • et al.
        Dissecting racial bias in an algorithm used to manage the health of populations.
        Science. 2019; 366: 447-453
        • Felitti VJ
        • Anda RF
        • Nordenberg D
        • et al.
        Relationship of childhood abuse and household dysfunction to many of the leading causes of death in adults. The Adverse Childhood Experiences (ACE) Study.
        Am J Prev Med. 1998 May; 14: 245-258
        • Oh DL
        • Jerman P
        • Silvério Marques S
        • et al.
        Systematic review of pediatric health outcomes associated with childhood adversity.
        BMC Pediatrics. 2018; 18: 83
        • Sameroff AJ
        • Bartko WT
        • Baldwin A
        • et al.
        Family and social influences on the development of child competence.
        Families, Risk, and Competence. Lawrence Erlbaum, Mahwah, NJ1998: 161-185
        • Neece CL
        • Green SA
        • Baker BL.
        Parenting stress and child behavior problems: a transactional relationship across time.
        Am J Intellect Dev Disabil. 2012; 117: 48-66https://doi.org/10.1352/1944-7558-117.1.48
        • Eder M
        • Henninger M
        • Durbin S
        • et al.
        Screening and interventions for social risk factors: technical brief to support the US Preventive Services Task Force.
        JAMA. 2021; (Published online September 01)https://doi.org/10.1001/jama.2021.12825
        • Bethell CD
        • Garner A
        • Blackwell CK
        • et al.
        Social and relational health risks and common mental health problems among US children: the mitigating role of family resilience and connection to promote socio-emotional and school outcomes.
        Child and Adol. Psychiatric Clinics of N. A. Jan 2022; 31: 45-70
        • AAP Council on Community Pediatrics. Poverty and Child Health in the United States
        Capturing Social and Behavioral Domains and Measures in Electronic Health Records: Phase 2.
        (Pediatrics. 2016; 137(4):e20160339 42) National Academies Press, Washington, DC2014 (Committee on the Recommended Social and Behavioral Domains and Measures for Electronic Health Records, Board on Population Health and Public Health Practice, IOM)
        • Yule K
        • Houston J
        • Grych J
        Resilience in children exposed to violence: a meta-analysis of protective factors across ecological contexts.
        Clin Child Fam Psychol Rev. 2019 Sep; 22: 406-431
        • Traub F
        • Boynton-Jarrett R.
        Modifiable resilience factors to childhood adversity for clinical pediatric practice.
        Pediatrics. 2017; 139:e20162569https://doi.org/10.1542/peds.2016-2569
        • Ghandour RM
        • Jones JR
        • Lebrun-Harris LA
        • et al.
        Bethell CD The Design and Implementation of the 2016 National Survey of Children’s Health.
        Maternal Child Health Journal. 2018; 22: 1093-1102
      8. Child and Adolescent Health Measurement Initiative (CAHMI). 2016-2017 National Survey of Children's Health SPSS Codebook, Version 1.0. 2018. Available at: www.childhealthdata.org. Accessed November 5, 2020.

        • Reuland CP
        • Collins J
        • Chiang L
        • et al.
        Oregon's approach to leveraging system-level data to guide a social determinants of health-informed approach to children's healthcare.
        BMJ Innovations. 2020; (Published Online First: 23 October 2020)https://doi.org/10.1136/bmjinnov-2020-000452
      9. Bethell CD, Blackwell C, Gombojav N, et al. Technical Appendix for “Toward Measurement for a Whole Child Health Policy”: Development, Validation and Prevalence on the Integrated Child Risk Index. Accessible at:https://www.cahmi.org/docs/default-source/default-document-library/_academic-peds-icri-technical-appendix_2-4-22.pdf?sfvrsn=771a1714_0. Accessed December 3, 2021; 2021

      10. Bethell, C, Johnson, K, Endsley Hrivnak, P, et al. National Maternal and Child Health Strategic Measurement Agenda, Child and Adolescent Health Measurement Initiative, Johns Hopkins University: August 2018. Accessed on November 5, 2020 at:https://www.cahmi.org/docs/default-source/resources/maternal-and-child-health-national-strategic-measurement-agenda-full-report.pdf?sfvrsn=fe258d7c_0

      11. Burner, C, Johnson, K, Bethell, C. Next steps in family-focused screening to address social determinants of health for young children in pediatric primary care: Report of the Social Determinants of Health (SDOH) Technical Working Group of the Maternal and Child Health Measurement Research Network (MCH-MRN). 2018. Accessed on Nov. 17, 2020 at:https://www.cahmi.org/docs/default-source/resources/next-steps-in-family-focused-screening-to-address-social-determinants-of-health-for-young-children-in-pediatric-primary-care-(2018).pdf?sfvrsn=8e6d882d_0

        • Bethell C
        • Gombojav N
        Overview of the Whole Child Risk Index (WCRI): A summary measure to support statewide integrated systems of care efforts focused on whole child population health improvement.
        (Working Paper). April 2019;
      12. Bruner, C, Johnson, K, Bethell, C. Risk stratification for children in Medicaid: achieving the promise of prevention. April 2019. Accessed on December 8, 2020 at:https://www.inckmarks.org/docs/RiskStratificationandYoungChildrenFinalMay3pdf.pdf

        • Evans G
        • Li D
        • Whipple S.
        Cumulative risk and child development.
        Psychological bulletin. 2013; 139: 1342-1396
        • Brumley LD
        • Brumley BP
        • Jaffee SR.
        Comparing cumulative index and factor analytic approaches to measuring maltreatment in the National Longitudinal Study of Adolescent to Adult Health.
        Child abuse & neglect. 2019; 87: 65-76
        • Sameroff A
        • Seifer R
        • Barocas R
        • et al.
        Intelligence quotient scores of 4-year-old children: social-environmental risk factors.
        Pediatrics. 1987; 79: 343-350
      13. (In:)
        • Sameroff AJ
        • Seifer R
        • McDonough SC.
        Contextual contributors to the assessment of infant mental health.
        (editors)in: DelCarmen-Wiggins R Carter A Handbook of Infant, Toddler, and Preschool Mental Health Assessment. Oxford University Press, New York, NY2004: 61-76
        • Kraemer HC
        • Lowe KK
        • Kupfer DJ.
        To Your Health.
        Oxford University Press, New York, NY2005
        • McDowell I.
        Measuring Health: A Guide to Rating Scales and Questionnaires.
        3rd ed. Oxford University Press, New York2006
        • Levinson D
        • Kaplan G.
        What does Self Rated Mental Health Represent.
        J Public Health Res. 2014; 3 (Published 2014 Dec 2): 287https://doi.org/10.4081/jphr.2014.287
        • Bethell CD
        • Blumberg SJ
        • Stein RE
        • et al.
        Taking stock of the CSHCN screener: a review of common questions and current reflections.
        Acad Pediatr. 2015; 15: 165-176
        • Berry JG
        • Hall M
        • Cohen E
        • et al.
        Ways to identify children with medical complexity and the importance of why.
        J Pediatr. 2015; 167: 229-237
      14. World Health Organization, International classification of functioning, disability and health: children & youth version: ICF-CY, 2007. Accessed on November 5, 2020 at: https://apps.who.int/iris/bitstream/handle/10665/43737/9789241547321_eng.pdf?sequence=1

      15. Child and Adolescent Health Measurement Initiative, National Data Resource Center for Child and Adolescent Health interactive NSCH data query on child health status. Accessed November 5, 2020 at; https://www.childhealthdata.org/browse/survey/results?q=7608&r=1

      16. Bethell CD, Carle A, Hudziak J, et al. Methods to assess adverse childhood experiences of children and families: toward approaches to promote child well-being in policy and practice. Acad Pediatr. 2017 Sep 1;17:S51-69.

        • Marie-Mitchell A
        • Watkins HBR
        • Copado IA
        • et al.
        Use of the Whole Child Assessment to identify children at risk of poor outcomes.
        Child Abuse Negl. 2020 Jun; 104 (Epub 2020 Apr 11. PMID: 32289566)104489https://doi.org/10.1016/j.chiabu.2020.104489
        • Schieve L
        • Boulet S
        • Kogan M
        • Yeargin-Allsopp M
        • Boyle C
        • Visser S
        • Blumberg S
        • Rice C.
        Parenting aggravation and autism spectrum disorders: 2007 National Survey of Children's Health.
        Disability and health journal. 2011; 4: 143-152https://doi.org/10.1016/j.dhjo.2010.09.002
        • Kuhn ES
        • Laird RD.
        Family support programs and adolescent mental health: review of evidence.
        Adolesc Health Med Ther. 2014; 5 (Published 2014 Jul 10): 127-142https://doi.org/10.2147/AHMT.S48057
        • Billioux A
        • Verlander K
        • Anthony S
        • et al.
        Standardized Screening for Health-related Social Needs in Clinical Settings: The Accountable Health Communities Screening Tool, Discussion Paper.
        National Academy of Medicine, Washington, DC2017
      17. Dubowitz H.The Safe Environment for Every Kid model: Promotion of children's health, development, and safety, and prevention of child neglect. Ped Annals. 2014Nov;43:e271-7.

      18. Moore, KA, Bethell, CD, Murphy, D, et al. Flourishing from the start: What it is and How it Can be Measures. Child Trends. Mar 7, 2017. Accessed on November 5, 2020 at: https://www.childtrends.org/publications/flourishing-start-can-measured

        • Moore KA
        • Murphy K
        • Bradley BA.
        Summary Report: The Development of a Pilot Recommended National Outcome Measure for Healthy and Ready to Learn.
        Child Trends, Bethesda, MD2017 (by request from [email protected])
        • Cohen J.
        Statistical Power Analysis for the Behavioral Sciences.
        2nd ed. Hillsdale, NJ: Erlbaum, 1988
        • Hu L
        • Bentler PM
        Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A.
        Multidisciplinary Journal. 1999; 6: 1-55
        • MacCallum RC
        • Zhang S
        • Preacher KJ
        • et al.
        On the practice of dichotomization of quantitative variables.
        Psychological Methods. 2002; 7: 19-40
        • Kline RB.
        Principles and Practice of Structural Equation Modeling.
        Guilford Publications, New York, NY2015
        • Vest JR
        • Menachemi N
        • Grannis SJ
        • et al.
        Impact of risk stratification on referrals and uptake of wraparound services that address social determinants: a stepped wedged trial.
        Am J Prev Med. 2019 l. 1999; 6: 1-55
        • Mann C
        • Eder J.
        Don't forget the kids: care transformations that meet the needs of children.
        Acad Pediatr. 2019; 19: 865-867
      19. The Healthcare Transformation Task Force and the Public Health Leadership Forum, Partnering to Catalyze Comprehensive Community Wellness: An actionable framework for health care and public health collaboration. 2020. Accessed on Dec 22 2020 at: https://hcttf.org/wp-content/uploads/2018/06/Comprehensive-Community-Wellness-Report.pdf

        • McCabe MA
        • Leslie L
        • Counts N
        • et al.
        Pediatric integrated primary care as the foundation for healthy development across the lifespan.
        Clinical Practice in Pediatric Psychology. 2020; 8: 278-287
        • Brewster AL
        • Fraze TK
        • Gottlieb LM
        • et al.
        The role of value-based payment in promoting innovation to address social risks: a cross-sectional study of social risk screening by US physicians.
        Milbank Q. 2020; 77 (October 20 Block, R. Whole Person Health Reform Intiative Launched in Pennsylvania. Oct 23, 2020. Accessed on Dec 22, 2020 at:)
        • Murray SD
        • Hurley J
        • Ahmed SR.
        Supporting the whole child through coordinated policies, processes, and practices.
        J Sch Health. 2015; 85 (PMID: 26440821; PMCID: PMC4606772): 795-801https://doi.org/10.1111/josh.12306
        • Teisberg E
        • Wallace S
        • O'Hara S
        Defining and implementing value-based health care: a strategic framework.
        Acad Med. 2020; 95: 682-685
        • Mann C
        • Ferguson M.
        Caring for the Whole Child: A New Way to Finance Initiatives to Improve Children's Health and Well-being. Issue Brief.
        Manatt, Phelps & Phillips, LLP, New York, NY2020
        • Massachusetts Child and Adolescent Health Initiative
        Moving to the Vanguard on Pediatric Care: Child and Adolescent Health Initiative Recommendations for the MassHealth Section 1115.
        Waiver Renewal. 2020; (Accessed April 26, 2021. Available at:)
        • Bruner C.
        Medicaid, Managed Care, and Value-Based Payment Systems: State Roles in Contracting for Healthy Child Development.
        InCK Marks Working Paper Series. 2021; (InCK Marks Initiative. Accessed April 26, 2021 at: https://img1.wsimg.com/blobby/go/35bcb6ed-a93b-41a3-9ad9-273bceb67f08/downloads/ValueBasedContractsWorkingPaper8docx.pdf?ver=1620231363217)
        • Wolfe I
        • Satherley RM
        • Scotney E
        • et al.
        Integrated care models and child health: a meta-analysis.
        Pediatrics. 2020; 145 (PMID: 31888959)e20183747https://doi.org/10.1542/peds.2018-3747