Advertisement

The Referral and Consultation Entrustable Professional Activity: Defining the Components in Order to Develop a Curriculum for Pediatric Residents

      Approaches to the assessment of outcomes in medical education have changed over the last decade. In 2001 the 6 Accreditation Council for Graduate Medical Education (ACGME) competency domains and 48 competencies were introduced as the framework for assessment, and more recently, the milestones have expanded the framework with detailed descriptions of progression in competency.
      • Hicks P.J.
      • Schumacher D.J.
      • Benson B.J.
      • et al.
      The Pediatrics Milestones: conceptual framework, guiding principles, and approach to development.
      The pediatrics education community is also beginning to accept the concept of entrustable professional activities (EPAs), which describe the broad scope of work that is expected of a physician. As described by ten Cate and Scheele,
      • ten Cate O.
      • Scheele F.
      Competency-based postgraduate training: can we bridge the gap between theory and clinical practice?.
      the EPA “concept allows faculty to make competency-based decisions on the level of supervision required by trainees.” Put another way, EPA's provide a synthetic framework that requires changes in the amount of supervision provided in the clinical setting as residents' abilities progress.
      Management of the referral and consultation process in both general and subspecialty practice is critical for achieving the goals of the triple aim: improving the patient experience of care (including quality and satisfaction), improving the health of populations, and reducing the per capita cost of health care.

      Institute for Healthcare Improvement. The IHI triple aim. Available at: http://www.ihi.org/Engage/Initiatives/TripleAim/Pages/default.aspx. Accessed June 11, 2014.

      • Thibault G.E.
      Reforming health professions education will require culture change and closer ties between classroom and practice.
      In addition, the medical home concept calls for coordination of care between primary care providers and subspecialists to ensure seamless delivery of care for patients.
      Aligning resident education with the principles of the medical home for optimal patient care by judicious use of specialty consultation, avoidance of unnecessary referrals, and optimal management of this interface is more pressing than ever if the goals of the triple aim are to be reached. Recognizing the importance of this EPA for health care delivery and outcomes, both the Association of Pediatric Program Directors and the American Board of Pediatrics have identified referral and consultation as an essential EPA for graduating pediatric residents.

      Association of Pediatric Program Directors. Important points about entrustable professional activities (EPAs). Available at: https://www.appd.org/home/PDF/Newborn_EPA_mapped_to_MilestonesVignetttes_for_APPDmembers.pdf

      Although examples of good practice and desirable behaviors related to referral and consultation have been described,
      • Forrest C.B.
      A typology of specialists’ clinical roles.
      the components that constitute this professional activity have not been defined. Here we describe the following: 1) the process we used to define the components of the referral and consultation EPA; 2) the translation of the components of the EPA into curricular goals and objectives that are mapped to competency domains; 3) possible ways to deliver the curriculum to pediatric residents; 4) suggestions for a program of assessment that will provide the evidence necessary to make entrustment decisions; and 5) implications for faculty development.

      Defining the Components of the Referral and Consultation EPA

      To define the components of the referral and consultation EPA, we first performed a literature review to base our work on existing knowledge about the referral and consultation process. We then used qualitative and survey methods to ask key stakeholders—patients and families, primary care pediatricians, subspecialty pediatricians, other specialists such as occupational therapists and pediatric residents—about their experiences with the referral and consultation process.

       Literature Review

      The literature provides ample evidence that there is room for improvement in the referral and consultation process from the viewpoint of patients and providers.
      • Bondenheimer T.
      Coordinating care: a perilous journey through the health care system.
      • Chen A.H.
      • Yee H.F.
      Improving the primary care-specialty interface: getting from here to there.
      Patients often experience challenges when referred from primary care to other health care providers, including confusion about the urgency or necessity of the referral,
      • Forrest C.B.
      Pediatric subspecialty referral completion.
      difficulty navigating the health care system to get needed appointments, timely approval by insurance,
      • Forrest C.B.
      Pediatric subspecialty referral completion.
      and inadequate communication among providers.
      • Forrest C.B.
      • Glade G.B.
      • Baker A.E.
      • et al.
      Coordination of specialty referrals and physician satisfaction with referral care.
      • Mehotra A.
      • Forrest C.B.
      • Yin C.
      Dropping the baton: specialty referrals in the United States.
      Providers often rely on patients to be the purveyors of information,
      • Stille C.J.
      • Primack W.A.
      • McLaughlin T.J.
      • et al.
      Parents as information intermediaries between primary care and specialty physicians.
      and they also report suboptimal communication at the interface between primary and specialty care.
      • Forrest C.B.
      • Glade G.B.
      • Baker A.E.
      • et al.
      Coordination of specialty referrals and physician satisfaction with referral care.
      • Stille C.J.
      • McLaughlin T.J.
      • Primack W.A.
      • et al.
      Determinants and impact of generalist–specialist communication about pediatric outpatient referrals.
      The literature further highlights that these concerns, as well as systems issues such as limited access to specialists, often result in delays in care, lack of completion of the referral, or unnecessary referrals; the end result is poor patient care and injudicious use of limited resources.
      • Bondenheimer T.
      Coordinating care: a perilous journey through the health care system.
      • Mehotra A.
      • Forrest C.B.
      • Yin C.
      Dropping the baton: specialty referrals in the United States.
      This review informed the focus of the qualitative data gathering with stakeholders.

       Qualitative Data Collection With Stakeholders

      We conducted focus groups with families, primary care pediatricians, and subspecialist pediatric providers with the goal of understanding their experience with the referral and consultation process, the components that are essential for success, and the components that are often overlooked. We used the constant comparative approach to qualitative analysis from grounded theory methodology
      • Corbin J.
      • Strauss A.
      Basics of Qualitative Research.
      • Glaser B.G.
      • Strauss A.L.
      The Discovery of Grounded Theory: Strategies for Qualitative Research.
      to analyze focus group transcripts. After coding, themes emerged and a grounded theory was developed. Our findings corroborate those reported in the literature but also add new information: families described the energy and emotion expended in navigating the system, the lack of communication among providers, and the access issues that pose barriers to optimal care. Providers echoed concerns about time and energy expended and lack of communication at the primary care/specialty care interface, and they agreed on a need to educate the next generation of practitioners to manage subspecialty problems in primary care, to recognize red flags for referral, and to improve communication and coordination of care.

       Pediatric Resident Survey

      Second- and third-year residents (n = 132) in the authors' home institutions completed an online survey about referral and consultation. Insights from the literature review and focus groups informed the survey questions. Residents identified gaps in their mastery of the components of the referral and consultation EPA, and 84% desired additional education about the referral and consultation process, including the following: communication, logistics and process of a referral, knowledge related to decision to refer, and ongoing management. Many residents did not think they were prepared to communicate with families who resist referral, communicate about insurance/cost factors or appointment access, prepare the family by explaining the logistics of a referral, or explain how to make an appointment. In addition, most residents thought they were inadequately trained to ensure communication with specialists and to provide them with all necessary prereferral data. Furthermore, only a few residents had a standard way to follow up on patients, and as a result, many never learned what happened at the visit. Even when they learned what happened, many residents did not know what the next steps in management should be. Residents suggested that a curriculum include the following: 1) feedback for residents from specialists about the appropriateness and ongoing management of their referrals; 2) faculty development to prepare primary care and specialist preceptors to teach about referral and consultation; and 3) standardization of the referral and consultation system.

      Development of Curricular Elements

      Three overarching components of the referral process emerged from the literature review and stakeholder input: 1) make an appropriate decision to refer; 2) make the referral and ensure its completion; and 3) provide appropriate postreferral patient care, coordination, and follow-up. In addition, 17 related action items or behaviors emerged. We translated the 3 components into the goals and the 17 action items into specific, measurable objectives of the curriculum that reflect the requisite knowledge, skills, and attitudes that must be acquired for entrustment in the referral and consultation EPA. The objectives are listed in the Table mapped to their relevant competency domains (eg, patient care) and associated specific activities (eg, patient care 1: gather essential and accurate information about the patient). We refer to those activities as they are numbered in the current terminology of the Pediatric Milestone Project.
      TableCurricular Elements for Referral and Consultation EPA
      Competency
      Make Appropriate Decision to Refer
       Gather essential and accurate information about the patient through history, eliciting the family/patient's view and agenda, physical examination, appropriate laboratory testingPC 1
       Apply medical knowledge to reach preliminary problem identificationMK 1
       Engage the family/patient in an exchange to arrive at a shared mental model of the problemsICS 1,2
       Identify knowledge gaps and access information such as guidelines, expert opinion, evidenceMK I, PBLI 1, 6, PPD 1
       Demonstrate the ability to negotiate with the family/patient: acknowledge concernsICS 1, P 1
       Make a decision whether or not to refer by applying all relevant gathered data, evidence, and family/patient considerationsPC 4, 5, PPD 8
      Make the referral and ensure its completion
       Decide on the urgency of the referralPC 4, 9, MK 1, ICS 2, P 1
       Articulate a summary of the patient that identifies the referral question and/or requestPC 4, MK 1, ICS 2
       Perform the appropriate prereferral assessment (laboratory tests, imaging, etc)PC 4, MK 1
       Choose the appropriate specialty for consultationMK 1, PC 4, 9, ICS 1
       Recognize and manage the logistics of the referral (how patient will make an appointment, access to specialty given identified urgency, insurance issues, etc)PC 3, SBP1
       Clarify plan with the family and ensure conceptual understanding and agreement on logistics of the plan, including reason for referral, expected time frame, logistics, and roles for family/patient, referring provider, consultantPC 3, 9, P 1, SBP 2
       Communicate with the consultant, using appropriate level of communication (phone, written referral, etc) and providing a clear referral requestPC 3, 5, ICS 3
       Troubleshoot logistical problems in ensuring the referral completion by clarifying roles and, when necessary, recruiting assistance.PC 3, ICS 1, SBP 1, 2, PPD 5
      Provide appropriate postreferral patient care, coordination, and follow-up
       Access consultant report to identify any next steps necessary in patient care, including clarification of findings, procedures done, recommendations, and/or ongoing management with family/patientPC 5, ICS 1, 2, SBP 2
       Provide ongoing patient carePC 5
       Refer back to specialist for indicated follow-up and/or complications in course requiring further consultationPC 5, ICS 1, 3, SBP 1, 2
      PC indicates patient care; MK, medical knowledge; PBL, practice-based learning and improvement; ICS, interpersonal and communication skills; P, professionalism; SBP, systems-based practice; and PPD, personal and professional development.

      Proposed Curricular Implementation

      Having identified the components of the referral and consultation EPA and verified the need to provide deliberate education about those components, we have begun the planning process for a curriculum that will be delivered in primary care (eg, continuity clinic, outpatient block), subspecialty core, and elective rotations. In the primary care setting, relevant skills will be taught, practiced, and assessed in the course of routine patient care. Each resident will be expected to acquire the knowledge and skills to make appropriate referrals for a list of common conditions that were generated by experienced primary care and subspecialty physicians from 2 of the authors' (LL, JH) institutions using a modified Delphi process. In subspecialty rotations, residents will critically examine the 3 components of the referral and consultation process by interviewing families and the subspecialist physician or other specialist who is seeing the patient.

      Assessment of Educational Outcomes

      Evidence about resident performance, which will provide the data on which entrustment decisions for the referral and consultation EPA are based, will be gathered in several ways and from a number of sources, including the following: 1) pre- and posttests of online case-based modules about commonly referred conditions to assess knowledge; 2) direct observation by faculty to assess communication and management skills; 3) resident reflections on the patient experience with referral and consultation; 4) parent/patient feedback and satisfaction; 5) feedback from consultants; and 6) tracking of referral patterns for common conditions. We have developed 2 structured tools for resident assessment that we plan to pilot. First is a tool to record and guide faculty feedback to residents in the primary care setting. They will record the context of the patient encounter and the referral components observed or discussed along with corrective and reinforcing feedback. Second is a tool to prompt reflection in the subspecialty setting about patient and family experiences with the referral and consultation process that will guide resident discussion with the subspecialists and allow them to record their findings and reflections about what went well and what could have been improved.
      We plan to collate the narrative and contextual data from each of these tools over time and synthesize them with other data to make decisions about each resident's overall performance, level of competency, and readiness for entrustment in the referral and consultation process.

      Faculty and Resident Development

      In order for the referral and consultation curriculum to succeed, residents and faculty must engage in a productive learning partnership. This will require a purposeful faculty and resident development program. Skilled educators will coach residents to take ownership of their learning by reflecting on their performance and creating appropriate learning goals related to the components of the referral and consultation EPA. Faculty will receive coaching in observing residents during the referral and consultation process, providing effective reinforcing and corrective feedback, and helping residents develop and meet their learning goals.

      Conclusions and Next Steps

      The methods used to define this EPA and the associated curriculum serve as an example of how residency and fellowship training programs can define the components of and develop curricular content for other EPAs. The process of literature review followed by needs assessment with key stakeholders—patients, families, and health care providers—generates comprehensive insight into the behaviors and skills required for entrustment. This process also helps identify the educational gaps that must be filled to meet those needs. In regard to the referral and consultation process, our findings demonstrate consensus among families, providers, and residents that education about this process is necessary regardless of a resident's career choice. With the knowledge, skills, and attitudes necessary for the referral and consultation EPA clearly defined, residency training programs can now embark on implementing curricula to teach the skills required and collect relevant assessments to provide the evidence to support entrustment decisions. A deliberate focus on this EPA holds the promise of eventually improving patient care and outcomes as we equip our soon-to-be subspecialist and primary care providers to best manage the referral and consultation process.

      Acknowledgments

      Work on this project was awarded design support by the Initiative for Innovation in Pediatric Education, American Board of Pediatrics.

      References

      1. American Board of Pediatrics; Accreditation Council for Graduate Medical Education The Pediatric Milestones Project. Accreditation Council for Graduate Medical Education, Chicago, Ill2012
        • Hicks P.J.
        • Schumacher D.J.
        • Benson B.J.
        • et al.
        The Pediatrics Milestones: conceptual framework, guiding principles, and approach to development.
        J Grad Med Educ. 2010; 2: 410-418
        • ten Cate O.
        • Scheele F.
        Competency-based postgraduate training: can we bridge the gap between theory and clinical practice?.
        Acad Med. 2007; 82: 542-547
      2. Institute for Healthcare Improvement. The IHI triple aim. Available at: http://www.ihi.org/Engage/Initiatives/TripleAim/Pages/default.aspx. Accessed June 11, 2014.

        • Thibault G.E.
        Reforming health professions education will require culture change and closer ties between classroom and practice.
        Health Aff (Millwood). 2013; 32: 1928-1932
      3. Kirschner N. Greenlee M.C. The Patient-Centered Medical Home Neighbor: The Interface of the Patient-Centered Medical Home With Specialty/Subspecialty Practices. American College of Physicians, Philadelphia, Pa2010
      4. Association of Pediatric Program Directors. Important points about entrustable professional activities (EPAs). Available at: https://www.appd.org/home/PDF/Newborn_EPA_mapped_to_MilestonesVignetttes_for_APPDmembers.pdf

        • Forrest C.B.
        A typology of specialists’ clinical roles.
        Arch Intern Med. 2009; 169: 1062-1068
        • Bondenheimer T.
        Coordinating care: a perilous journey through the health care system.
        N Engl J Med. 2008; 358: 1064-1071
        • Chen A.H.
        • Yee H.F.
        Improving the primary care-specialty interface: getting from here to there.
        Arch Intern Med. 2009; 169: 1024-1026
        • Forrest C.B.
        Pediatric subspecialty referral completion.
        J Pediatr. 2011; 158: 3-4
        • Forrest C.B.
        • Glade G.B.
        • Baker A.E.
        • et al.
        Coordination of specialty referrals and physician satisfaction with referral care.
        Arch Pediatr Adolesc Med. 2000; 154: 499-506
        • Mehotra A.
        • Forrest C.B.
        • Yin C.
        Dropping the baton: specialty referrals in the United States.
        Milbank Q. 2011; 89: 39-68
        • Stille C.J.
        • Primack W.A.
        • McLaughlin T.J.
        • et al.
        Parents as information intermediaries between primary care and specialty physicians.
        Pediatrics. 2007; 120: 1238-1246
        • Stille C.J.
        • McLaughlin T.J.
        • Primack W.A.
        • et al.
        Determinants and impact of generalist–specialist communication about pediatric outpatient referrals.
        Milbank Q. 2006; 118: 1341-1349
        • Corbin J.
        • Strauss A.
        Basics of Qualitative Research.
        3rd ed. Sage Publications, Thousand Oaks, Calif2008
        • Glaser B.G.
        • Strauss A.L.
        The Discovery of Grounded Theory: Strategies for Qualitative Research.
        Aldine Publishing Company, Hawthorne, NY1967