A Systematic Review of the Methods Used to Evaluate Child Psychiatry Access Programs



      There is a well-documented gap between the need for and availability of mental health services for children nationwide. To address this gap, over 30 regional Child Psychiatry Access Programs (CPAPs) provide psychiatric consultation and other services to primary care providers.


      Summarize the methods used to evaluate CPAPs in the United States.

      Data Sources

      PubMed, PsychInfo, CINAHL, and reference checking.

      Study Appraisal Methods

      A systematic literature review was conducted searching 3 databases. The search produced 307 unique articles, 278 were excluded for irrelevance, leaving 29 for data extraction. Data extracted included author(s), publication year, provider types, CPAP formats, study sample, design, outcomes examined, results, and limitations. Articles were also appraised for quality using the Johns Hopkins Nursing Evidence-Based Practice Evidence Level and Quality Guide.


      The 29 articles evaluated 13 unique CPAPs. Most evaluations used nonexperimental observational designs (68.9%), 6.9% used quasi-experimental designs, and none used true experimental designs. Evaluations examined the following outcomes: usage of program services (82.8%), provider satisfaction (48.3%), provider comfort/confidence with managing mental health concerns (31.0%), provider practice change (24.1%), patient outcomes (13.7%), and family satisfaction (6.9%). Outcomes were measured using surveys, qualitative interviews, or insurance claims data.


      Review was limited to articles published in English in 3 databases or identified by reference checking.


      Evaluations of CPAPs have largely been descriptive in nature, focusing primarily on program usage and provider satisfaction. Few studies have examined the impact of CPAPs on patients, families, or health systems. Future studies should evaluate the broader impacts of CPAPs.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Academic Pediatrics
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Whitney DG
        • Peterson MD
        US national and state-level prevalence of mental health disorders and disparities of mental health care use in children.
        JAMA Pediatr. 2019; 173: 389-391
        • Copeland WE
        • Wolke D
        • Shanahan L
        • et al.
        Adult functional outcomes of common childhood psychiatric problems a prospective, longitudinal study.
        JAMA Psychiatry. 2015; 72: 892-899
        • National Research Council
        Preventing Mental, Emotional, and Behavioral Disorders Among Young People: Progress and Possibilities.
        National Academies Press, Washington, DC2009
        • Busch SH
        • McCue Horwitz S
        Access to mental health services: are uninsured children falling behind?.
        Ment Health Serv Res. 2004; 6: 109-116
        • Kolko DJ
        • Perrin E
        The integration of behavioral health interventions in children's health care: services, science, and suggestions.
        J Clin Child Adolesc Psychol. 2014; 43: 216-228
        • US Department of Health and Human Services
        National projections of supply and demand for selected behavioral health practitioners: 2013-2025.
        2016 (Available at: Accessed August 6, 2019.)
        • Kim WJ
        • Bechtold D
        • Brooks BA
        • et al.
        Child and adolescent psychiatry workforce: a critical shortage and national challenge.
        Acad Psychiatry. 2003; 27: 277-282
        • McBain RK
        • Kofner A
        • Stein BD
        • et al.
        Growth and distribution of child psychiatrists in the United States: 2007–2016.
        Pediatrics. 2019; 144e20191576
        • Williams J
        • Palmes G
        • Klinepeter K
        • et al.
        Referral by pediatricians of children with behavioral health disorders.
        Clin Pediatr (Phila). 2005; 44: 343-349
        • World Health Organization
        Atlas: Child and Adolescent Mental Health Resources: Global Concerns, Implications for the Future.
        WHO Press, Geneva, Switzerland2005
        • Heneghan A
        • Garner AS
        • Storfer-Isser A
        • et al.
        Pediatrician's role in providing mental health care for children and adolescents: do pediatricians and child and adolescent psychiatrists agree?.
        J Dev Behav Pediatr. 2008; 29: 262-269
        • Wissow LS
        • van Ginneken N
        • Chandna J
        • et al.
        Integrating children's mental health into primary care.
        Pediatr Clin North Am. 2016; 63: 97-113
        • Horwitz SM
        • Storfer-Isser A
        • Kerker BD
        • et al.
        Barriers to the identification and management of psychosocial problems: changes from 2004 to 2013.
        Acad Pediatr. 2015; 15: 613-620
        • Ader J
        • Stille CJ
        • Keller D
        • et al.
        The medical home and integrated behavioral health: advancing the policy agenda.
        Pediatrics. 2015; 135: 909-917
        • Asarnow JR
        • Rozenman M
        • Wiblin J
        • et al.
        Integrated medical-behavioral care compared with usual primary care for child and adolescent behavioral health: a meta-analysis.
        JAMA Pediatr. 2015; 169: 929-937
        • Burkhart K
        • Asogwa K
        • Muzaffar N
        • et al.
        Pediatric integrated care models: a systematic review.
        Clin Pediatr (Phila). 2020; 59: 148-153
        • Spencer AE
        • Platt RE
        • Bettencourt AF
        • et al.
        Implementation of off-site integrated care for children: a scoping review.
        Harv Rev Psychiatry. 2019; 27: 342-353
        • Van Cleave J
        • Holifield C
        • Perrin JM
        Primary care providers’ use of a child psychiatry telephone support program.
        Acad Pediatr. 2018; 18: 266-272
        • NNCPAP
        National network of child psychiatry access programs.
        2019 (Available at:)
        Date accessed: October 28, 2019
        • Stock KM
        Addressing trauma and children's mental health through child psychiatry access programs: an introductory guide for state health advocates.
        2019 (Available at: Accessed October 28, 2019)
        • Hilt RJ
        • Romaire MA
        • Michael GM
        • et al.
        The partnership access line: evaluating a child psychiatry consult program in Washington state.
        JAMA Pediatr. 2013; 167: 162-168
        • Sarvet BD
        • Wegner L
        Developing effective child psychiatry collaboration with primary care: leadership and management strategies.
        Child Adolesc Psychiatr Clin N Am. 2010; 19: 139-148
        • US Department of Health and Human Services
        Justification of estimates for appropriations committees: fiscal year 2020.
        2019 (Available at: Accessed October 15, 2019.)
        • Health Resources & Services Administration
        Pediatric mental health care access program fact sheet.
        2019 (Available at: Accessed October 16, 2019.)
        • Platt RE
        • Spencer AE
        • Burkey MD
        • et al.
        What's known about implementing co-located paediatric integrated care: a scoping review.
        Int Rev Psychiatry. 2018; 30: 242-271
      1. Covidence Systematic Review Software [Computer program]. Melbourne, Australia: Veritas Health Innovation. Available at: Accessed August 16, 2020.

        • Dang D
        • Dearholt S
        Johns Hopkins Nursing Evidence-Based Practice: Model and Guidelines.
        3rd ed. Sigma Theta Tau International, Indianapolis, Ind2018
        • Moher D
        • Liberati A
        • Tetzlaff J
        • et al.
        Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.
        PLOS Med. 2009; 6e1000097
        • Hobbs Knutson K
        • Wei MH
        • Straus JH
        • et al.
        Medico-legal risk associated with pediatric mental health telephone consultation programs.
        Adm Policy Ment Heal Ment Heal Serv Res. 2014; 41: 215-219
        • Walter HJ
        • Kackloudis G
        • Trudell EK
        • et al.
        Enhancing pediatricians’ behavioral health competencies through child psychiatry consultation and education.
        Clin Pediatr (Phila). 2018; 57: 958-969
        • Cama S
        • Knee A
        • Sarvet B
        Impact of child psychiatry access programs on mental health care in pediatric primary care: measuring the parent experience.
        Psychiatr Serv. 2019;
        • Malas N
        • Klein E
        • Tengelitsch E
        • et al.
        Exploring the telepsychiatry experience: primary care provider perception of the Michigan child collaborative care (MC3) program.
        Psychosomatics. 2018; : 1-11
        • Marcus SM
        • Malas NM
        • Quigley JM
        • et al.
        Partnerships with primary care for the treatment of preschoolers.
        Child Adolesc Psychiatr Clin N Am. 2017; 26: 597-609
        • Marcus S
        • Malas N
        • Dopp R
        • et al.
        The Michigan child collaborative care program: building a telepsychiatry consultation service.
        Psychiatr Serv. 2019; 70
        • Archbold T
        The psychiatric assistance line. One solution to the child and adolescent mental health crisis.
        Minn Med. 2015; 98: 42-44
        • Stein BD
        • Kofner A
        • Vogt WB
        • et al.
        A national examination of child psychiatric telephone consultation programs’ impact on children's mental health care utilization.
        J Am Acad Child Adolesc Psychiatry. 2019; 58: 1016-1019
        • Kaye DL
        • Fornari V
        • Scharf M
        • et al.
        Description of a multi-university education and collaborative care child psychiatry access program: New York State's CAP PC.
        Gen Hosp Psychiatry. 2017; 48: 32-36
        • Gadomski AM
        • Wissow LS
        • Palinkas L
        • et al.
        Encouraging and sustaining integration of child mental health into primary care: interviews with primary care providers participating in Project TEACH (CAPES and CAP PC) in NY.
        Gen Hosp Psychiatry. 2014; 36: 555-562
        • Barclay RP
        • Hilt RJ
        • Garrison M
        A statewide pediatric psychiatry consultation to primary care program and the care of children with trauma-related concerns.
        J Behav Health Serv Res. 2015; 43: 691-699
        • Harrison J
        • Wasserman K
        • Steinberg J
        • et al.
        The five S's: a communication tool for child psychiatric access projects.
        Curr Probl Pediatr Adolesc Health Care. 2016; 46: 411-419
        • Hilt RJ
        • Barclay RP
        • Bush J
        • et al.
        A statewide child telepsychiatry consult system yields desired health system changes and savings.
        Telemed e-Health. 2015; 21: 533-537
        • Arora PG
        • Connors EH
        • Coble K
        • et al.
        Pediatric primary care providers’ use of behavioral health consultation.
        Psychiatr Serv. 2017; 68: 531-534
        • Platt R
        • Pustilnik S
        • Connors E
        • et al.
        Severity of mental health concerns in pediatric primary care and the role of child psychiatry access programs.
        Gen Hosp Psychiatry. 2018; 53: 12-18
        • Hobbs Knutson K
        • Masek B
        • Bostic JQ
        • et al.
        Clinicians’ utilization of child mental health telephone consultation in primary care: findings from Massachusetts.
        Psychiatr Serv. 2014; 65: 391-394
        • Straus JH
        • Sarvet B
        Behavioral health care for children: the Massachusetts child psychiatry access project.
        Health Aff. 2014; 33: 2153-2161
        • Van Cleave J
        • Le T-T
        • Perrin JM
        Point-of-care child psychiatry expertise: the Massachusetts Child Psychiatry Access Project.
        Pediatrics. 2015; 135: 834-841
        • Pidano AE
        • Slater CM
        • Dale LP
        • et al.
        Availability of telephone-based child psychiatry consultation: implications from a survey of pediatric providers in two states.
        J Child Fam Stud. 2016; 25: 2607-2615
        • Sarvet BD
        • Ravech M
        • Straus JH
        Massachusetts child psychiatry access project 2.0: a case study in child psychiatry access program redesign.
        Child Adolesc Psychiatr Clin N Am. 2017; 26: 647-663
        • Castro JL
        • Taylor JL
        • Golden WE
        Psych TLC: a new Arkansas resource for pediatric mental health.
        J Ark Med Soc. 2011; 107: 202-203
        • Sarvet B
        • Gold J
        • Bostic JQ
        • et al.
        Improving access to mental health care for children: the Massachusetts Child Psychiatry Access Project.
        Pediatrics. 2010; 126: 1191-1200
        • Dvir Y
        • Wenz-Gross M
        • Jeffers-Terry M
        • et al.
        An assessment of satisfaction with ambulatory child psychiatry consultation services to primary care providers by parents of children with emotional and behavioral needs: the Massachusetts child psychiatry access project University of Massachusetts parent satisfaction study.
        Front Psychiatry. 2012; 3: 1-7
        • Sheldrick RC
        • Mattern K
        • Perrin EC
        Pediatricians’ perceptions of an off-site collaboration with child psychiatry.
        Clin Pediatr (Phila). 2012; 51: 546-550
        • Connor DF
        • McLaughlin TJ
        • Jeffers-Terry M
        • et al.
        Targeted child psychiatric services: a new model of pediatric primary clinician—child psychiatry collaborative care.
        Clin Pediatr (Phila). 2006; 45: 423-434
        • Sarvet B
        • Gold J
        • Straus JH
        Bridging the divide between child psychiatry and primary care: the use of telephone consultation within a population-based collaborative system.
        Child Adolesc Psychiatr Clin N Am. 2011; 20: 41-53
        • Kerker BD
        • Chor KHB
        • Hoagwood KE
        • et al.
        Detection and treatment of mental health issues by pediatric PCPs in New York state: an evaluation of project TEACH.
        Psychiatr Serv. 2015; 66: 430-433
        • Center for Public Representation
        Rosie D: Reforming the mental health system in Massachusetts.
        2008 (Available at: Accessed October 28, 2019)
        • Craig P
        • Dieppe P
        • Macintyre S
        • et al.
        Developing and evaluating complex interventions: the new medical research council guidance.
        BMJ. 2008; 337: a1655
        • Wissow LS
        • Brown JD
        • Hilt RJ
        • et al.
        Evaluating integrated mental health care programs for children and youth.
        Child Adolesc Psychiatr Clin N Am. 2017; 26: 795-814
        • Archer J
        • Bower P
        • Gilbody S
        • et al.
        Collaborative care for depression and anxiety problems.
        Cochrane Database Syst Rev. 2012;
        • Oude Wesselink SF
        • Lingsma HF
        • et al.
        Effects of government supervision on quality of integrated diabetes care: a cluster randomized controlled trial.
        Med Care. 2015; 53 ( 784-791
        • Mellahi K
        • Harris LC
        Response rates in business and management research: an overview of current practice and suggestions for future direction.
        Br J Manag. 2016; 27: 426-437
        • Groves RM
        • Fowler FJJ
        • Couper MP
        • et al.
        Survey Methodology.
        2nd ed. John Wiley & Sons Inc, Hoboken, NJ2009
        • Turk A
        • Heneghan C
        • Nunan D
        Non-response bias. Catalogue of bias collaboration.
        2019 (Available at: Accessed June 1, 2020)