Abstract
Background
Care of children with medical complexity (CMC) involves unique knowledge, skills, and attitudes. Previous work has identified curricular priorities in complex care for pediatricians yet there are no established competency frameworks to guide curriculum development. We aimed to develop and appraise Entrustable Professional Activities (EPAs) for the care of CMC with multistakeholder involvement.
Methods
We recruited complex care practitioners to develop EPAs using a template for elaborating descriptive elements. A team of clinicians, educators, trainees, and family leaders refined EPAs and mapped content to the Accreditation Council for Graduate Medical Education Milestones. We conducted virtual focus groups to assess whether EPAs represented the essential skills of pediatricians caring for CMC. Focus group data were analyzed using content analysis.
Results
Content experts developed 11 EPAs for the care of CMC describing knowledge, skills, and attitudes required for attaining competency. EPAs were mapped to 21 of the 21 (100%) reporting pediatric milestones. Focus group participants endorsed and refined EPA content. Categories of feedback included clarifying medical knowledge, expanding on interpersonal communication skills, emphasizing systems-based practice, and affirming family partnership.
Conclusions
A systematic approach to developing EPAs for the care of CMC provides a guide for curriculum development and assessment in complex care.
Keywords
What's New
We describe the development of entrustable professional activities (EPAs) for the care of children with medical complexity through engagement of clinicians, educators, trainees and family leaders. These EPAs provide an important framework for curriculum development and assessment in complex care.
Children with medical complexity (CMC) are a subset of the pediatric population characterized by chronic multisystem disease, high service needs, and functional impairment often requiring the use of medical technology.
1
With improvements in care this population of children continues to grow, with a concomitant rise in health care utilization,2
, 3
, 4
resulting in pediatricians routinely caring for CMC in both inpatient and ambulatory settings. While care of CMC is recognized as an important aspect of pediatric training,5
there are no established competency frameworks to guide curriculum development and assessment in this area.American Board of Pediatrics. Entrustable professional activities for general pediatrics. 2019. Available at: https://www.abp.org/entrustable-professional-activities-epas. Accessed May 30, 2019.
6
,7
Entrustable Professional Activities (EPAs) provide an established framework for operationalizing competency-based education and defining high-quality patient care. EPAs are activities that are observable, essential to the profession, and often require integration of multiple competencies.
8
An EPA includes a description of the knowledge, skills and attitudes that are necessary to perform the activity.9
While many specialties, including pediatrics, have developed EPAs to help inform training and assessment strategies, the general pediatrics EPAs are intentionally broad in scope,10
offering little guidance on specific, practical skills required to care for CMC.American Board of Pediatrics. Entrustable Professional Activities for General Pediatrics. Available at: https://www.abp.org/entrustable-professional-activities-epas. Accessed June 27, 2020.
The authors are unaware of any other subspecialty in pediatrics that addresses the comprehensive care of CMC. There are 17 EPAs developed in Hospice and Palliative Medicine
11
that include common principles and overlapping areas of practice (eg, providing a decision-making framework for increasing technological dependence, care planning, and exploring the family context).12
EPAs for the care of CMC would expand on the scope of existing EPAs and provide a more comprehensive guide for trainees, training programs and other stakeholders.A recent North American Delphi study identified curricular priorities in pediatric complex care, including management of multisystem clinical conditions like aspiration, as well as medical technology care, care coordination and advocacy.
13
Yet it is unclear what skill development looks like before, at, and after graduation and transition to practice. Creating rich descriptions of the clinical activities required of general pediatricians using an EPA framework would help clarify what it means to attain competence in the care of this pediatric population. Therefore, we developed EPAs for the care of CMC in partnership with key stakeholders in complex care training, including clinicians, educators, trainees, and family leaders. We also explored their alignment with actual practice through focus groups of clinicians and advocates who care for CMC.Methods
We used a multistage approach to EPA development, modeled after the 5-phase process described by Chen et al for the development of EPAs for entry into clerkship: identification of content domains, domain mapping and confirmation, content description and expert consultation, assurance of appropriate content, and finalization and stakeholder review.
14
As prior work had already been done to establish content domains for pediatric complex care,13
we focused on the remaining phases in order to describe and refine EPAs for the care of CMC.EPA Content Description and Domain Mapping
We recruited complex care clinicians through an international complex care e-mail listserv, and through the Academic Pediatric Association's complex care special interest group at the Pediatric Academic Societies annual meetings in 2018 and 2019 to serve as content experts, given their focused practice in complex care. Content experts self-identified domains for which they had clinical experience, identifying collaborators as appropriate. Using snowball sampling, we also made targeted outreach to national experts in EPA content domains. We facilitated a virtual training meeting discussing the concept of EPAs and describing their components, and provided information on EPA development methods via e-mail communication. We developed and provided a template based on guidelines for EPA descriptions, including specifications of the activity, domains of competence critical to entrustment, and necessary knowledge, skills and attitudes.
9
We recruited 18 content experts from 9 institutions across the United States to develop 11 EPA descriptions.The study team representing complex care clinicians (KH, DH, NS), training program leaders (DH, NS), trainees (CCF), and parents/family leaders (CC, BF) reviewed and refined each draft EPA. We summarized feedback from multiple meetings and collaborated with content experts to revise EPAs. All authors conducted editorial review to standardize language and formatting, eliminate redundancies, and streamline content.
One investigator (DH) performed initial mapping of EPA content to the Accreditation Council for Graduate Medical Education (ACGME) pediatric milestones.
15
Mapping was reviewed with other team members (KH, NS) to establish consensus.The Accreditation Council for Graduate Medical Education and The American Board of Pediatrics. The Pediatrics Milestones Project. 2017. Available at: http://www.acgme.org/Specialties/Milestones/pfcatid/16/Pediatrics. Accessed August 10, 2017.
Assurance of Appropriate Content
EPA appraisal involved affirming that the EPAs accurately represented the practice of pediatric complex care. From November to December 2020, utilizing recruitment via invitations sent to the complex care listserv, we conducted 6 virtual focus groups. We asked participants to assess whether the EPA content represented the essential knowledge, skills and attitudes of a pediatrician caring for CMC. Each focus group was comprised of 5 to 8 interprofessional participants and was facilitated by 2 to 3 members of the study team, including our family leaders. Focus groups included complex care pediatricians, pediatric subspecialists, therapists, advanced practice providers, and clinical researchers/advocates. Focus groups were 60 minutes and included a brief introduction to the EPA framework and review of 2 EPAs. Participants provided feedback verbally and on a shared web-based application. Facilitators summarized feedback during focus groups to check understanding and ensure the summary resonated with participants and reflected key points of the discussion. All focus groups were audio-recorded and facilitators took notes for the purposes of reflection and comparison between facilitators.
Finalizing EPA Content
Two authors (KH, DH) summarized and categorized data from focus groups using conventional content analysis.
16
The authors independently compiled comments from the web-based application and facilitator notes and identified initial codes. The authors discussed relationships between codes and grouped codes into broader categories. The data were then reanalyzed using the coding scheme and illustrative examples were identified for each category.We integrated focus group feedback to further refine EPAs through multiple meetings. Final EPAs were reviewed and approved by the content experts and study team.
Results
Content experts developed 11 EPAs for the care of CMC with rich description of activity-specific knowledge, skills, and attitudes required for attaining competency (EPA titles in Table 1, full details of EPAs displayed in Appendix). We mapped the EPAs to 21 of the 21 (100%) reporting pediatric milestones from the ACGME (Table 1).
Table 1Entrustable Professional Activities (EPAs) for the Care of CMC With Mapping to Accreditation Council for Graduate Medical Education Milestones
| EPA Titles | PC1 | PC2 | PC3 | PC4 | PC5 | MK1 | SBP1 | SBP2 | SBP3 | PBLI1 | PBLI2 | PBLI3 | PBLI4 | PROF1 | PROF2 | PROF3 | PROF4 | PROF5 | PROF6 | ICS1 | ICS2 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Evaluate and manage feeding difficulties and nutritional concerns for CMC | X | X | X | X | X | X | X | X | X | X | X | X | X | X | |||||||
| Evaluate and manage pain and irritability in CMC | X | X | X | X | X | X | X | X | X | X | X | X | X | X | |||||||
| Provide routine care for CMC with feeding tubes and troubleshoot common issues | X | X | X | X | X | X | X | X | X | X | X | X | X | ||||||||
| Design and implement a developmentally appropriate transition process to adult care for CMC | X | X | X | X | X | X | X | X | X | X | X | ||||||||||
| Facilitate goals of care discussions and introduce the concepts of palliative and hospice care for CMC | X | X | X | X | X | X | X | X | X | X | X | X | X | X | X | X | |||||
| Facilitate team-based care coordination for CMC | X | X | X | X | X | X | X | X | X | X | X | X | X | X | X | ||||||
| Manage motility disorders in CMC | X | X | X | X | X | X | X | X | X | X | X | X | X | X | |||||||
| Evaluate and manage aspiration in CMC | X | X | X | X | X | X | X | X | X | X | |||||||||||
| Develop and implement safety and emergency plans for CMC | X | X | X | X | X | X | X | X | X | ||||||||||||
| Evaluate and manage common neuromuscular and skeletal issues in CMC | X | X | X | X | X | X | X | X | X | X | X | X | |||||||||
| Advocate for CMC and their families in the community setting | X | X | X | X | X | X | X | X | X | X | X | X | X | X | X | X | X |
CMC indicates children with medical complexity; PC, patient care; MK, medical knowledge; SBP, systems-based practice, PBLI, practice-based learning and improvement; PROF, professionalism; ICS, interpersonal and communication skills.
In total, 42 individuals participated in the focus groups. The majority were general pediatricians with direct care responsibilities for CMC in inpatient and ambulatory settings (25, 60%). Participants also included pediatric subspecialists (4, 10%), members of the interprofessional care team including registered nurses, nurse practitioners, and physical therapists (5, 12%), educators within pediatric residency and fellowships (12, 29%), and individuals with research and advocacy roles and lived experience in the care of CMC (6, 14%). Focus group participants felt that the EPA descriptions resonated with them and suggested minor revisions to EPA content. No additional EPAs were identified, though some participants suggested developing an EPA related to respiratory care and equipment.
Content analysis of focus group data yielded 4 categories of feedback (Table 2). 1) Clarifying essential medical knowledge of the pediatrician who cares for CMC, (eg, identifying risk factors, managing side effects, and recognizing complications for certain clinical conditions), and refined the scope of practice for the activity; 2) Expanding on the role of the pediatrician in facilitating communication with and between clinicians and families; 3) Emphasizing the centrality of systems-based practice, including opportunities for interprofessional collaboration, care coordination, and accessing community resources; and 4) Affirming the focus on family partnership, expressing appreciation for the emphasis on shared decision-making and patient- and family-centeredness.
Table 2Focus Group Feedback Categories, Definitions, and Examples
| Category | Definition | Example |
|---|---|---|
| Clarifying essential medical knowledge | Illustrating the role of the pediatrician in the care of CMC. Often included specifying content or refining the scope of practice for a clinical activity | Discuss indications and safety considerations for bracing, perform anticipatory guidance (including use of wheelchair seatbelt, need for head and neck support, safe transfers), and recognize bracing issues (including issues with fit and skin breakdown) |
| Expanding on interpersonal communication skills | Describing the necessary practices and tools for effective communication with other clinicians and with patients/families | Include communication with families about what the pediatrician will do as an advocate, including liaising with schools, obtaining prior authorizations, requesting additional nursing hours, equipment, advocating for CMC in the foster care system, discussion with subspecialists. |
| Emphasizing systems-based practice | Identifying opportunities for the pediatrician to work effectively within systems to advance or enable equitable access to care for CMC. | Consider opportunities to advocate for inclusion when possible in different institutions (eg, school). |
| Affirming family partnership | Acknowledging the focus on collaborative partnerships with families and patient- and family-centeredness in the activity descriptions | Appreciate highlighting multiple possibilities for management and importance of shared decision-making with families. Recognize that doing nothing is also an option. |
Discussion
In this study, we describe a multistep process engaging a diverse group of key stakeholders in the development and appraisal of 11 EPAs for the care of CMC. These EPAs are intended to support learners and educators across the training continuum in focusing learning and assessment efforts in essential areas of professional practice for pediatricians in the care of CMC. They provide not only a curricular roadmap, but an assessment framework amenable to integration with existing programmatic requirements.
Our EPA development process incorporated multiple steps to generate content validity evidence.
17
We established a shared understanding with content experts in the standards and methods for EPA development, used a template outlining EPA components and provided frequent instruction and feedback. We used an iterative approach to collect, interpret, and integrate feedback from experts to inform EPA revisions, focused on assuring that the EPAs capture the practice of complex care. We also ensured that the set of EPAs aligned with an existing competency framework used in the profession. We believe that these EPAs are an accurate reflection of the essential work of pediatricians in the care of CMC.Though EPAs describe activities for the pediatrician and form the basis for decisions about entrustment for individual clinicians, it is critical to recognize the nature of complex care as team-based and with varying roles of the pediatrician dependent on context, available resources and areas of need. Focus groups highlighted opportunities to clarify the scope and nature of these activities. Care of CMC typically depends on an interprofessional team, comprising subspecialists, advanced practice providers, social workers, therapists, community-based providers, and most critical, patients and families at the center.
18
The pediatrician's role often involves identifying needs, performing initial evaluation and management of clinical issues, and partnering with other members of the care team including families. These concepts are embedded across all EPAs for the care of CMC. Furthermore, pediatricians care for CMC in various health care settings, within different models of care delivery, with variable access to subspecialists and other resources. The EPAs are written to be deliberately broad and can thus be used and adapted across training programs and care settings.Development of EPAs specific to the care of particular subpopulations in generalist specialties is not unique to pediatrics. In internal medicine, geriatricians composed 12 EPAs addressing the unique aspects of caring for geriatric patients, which describe many common areas of practice with the care of CMC, including managing multisystem clinical “syndromes” and discussing goals of care through shared decision-making.
19
These geriatric EPAs are meant to provide a framework for developing competency in the care of geriatric patients beyond what is expected of a general internist. Similarly, our hope is that pediatrics trainees, program directors and practitioners will utilize EPAs for the care of CMC to further develop and enhance their knowledge and skills in pediatric complex care. This effort may include consideration of additional training opportunities to build upon knowledge gained through pediatric residency programs. Given the growing population of CMC and the multidisciplinary nature of their care, we should aspire to ensure every graduating pediatric resident will have achieved competency in as many of these EPAs as possible. Nonetheless given multiple challenges in postgraduate medical training including limited training time with competing priorities, varied individual career goals and limited knowledge on when trainees achieve competency, assuring adequate training to achieve competency in these EPAs during residency training may not be easily accomplished.20
We propose that these EPAs be utilized as a framework, in addition to the existing general pediatrics EPAs, for cultivating a skillset in trainees who have a specific interest in complex care. For those who plan to regularly provide care to CMC, there may be opportunity to leverage existing programmatic infrastructure, such as individualized curriculum time or residency pathways, to focus knowledge and skill development in complex care and achieve competency prior to independent practice. Furthermore, given the multidisciplinary nature of care of CMC, we encourage pediatric subspecialty training program leaders to utilize these complex care EPAs during fellowship as well. Finally, if the availability of dedicated complex care fellowships expands across the nation, it is likely that achievement of competency in these EPAs will continue during additional focused training in complex care.- Schumacher DJ.
- et al.
Longitudinal assessment of resident performance using entrustable professional activities.
JAMA Netw Open. 2020; 3 (Jan.DOI.org (Crossref))e1919316https://doi.org/10.1001/jamanetworkopen.2019.19316
Our study has some important limitations. Content experts were largely based in the United States and EPAs were mapped to ACGME milestones, thus the products of this work may have limited generalizability to other countries and training programs. Our recruitment was also limited to an e-mail listserv and likely missed important contributors. However, stakeholders who participated in focus groups included practitioners from across North America with experience in various care settings, and EPA descriptions are intentionally inclusive of various care models, so we believe the EPAs largely reflect the practice of pediatricians in the care of CMC. Content domains for these 11 EPAs were based on a prior Delphi study and may not include all essential clinical activities in complex care. It was not our intent to provide an exhaustive set of EPAs, but rather to develop EPAs based on prior literature as a starting point in defining this area of clinical practice. Our sources of validity evidence are limited, and our study does not explore operationalizing these EPAs and feasibility of their use in pediatric training programs. Despite these limitations, our findings recognize complex care as a critical aspect of professional practice for every pediatrician. The described clinical activities are high-stakes and/or frequently encountered in practice, and are essential to high-quality, safe, patient-and family-centered care for CMC. We anticipate that this approach can serve as a model for developing educational frameworks in other areas of pediatric practice.
Conclusion
A systematic approach to developing EPAs for the care of CMC provides a detailed guide for curriculum development, training activities and assessment priorities in complex care. Next steps include partnering with training programs to use the EPA framework to guide curriculum development and assessment in pediatric complex care.
Acknowledgments
Thank you to the Complex Care EPA Working Group who served as content experts and authors of EPAs for the care of CMC: Ruba A. Abdelhadi, Justin Alvey, Jori F. Bogetz, Taron Davis, Laurie Glader, Emily Goodwin, Julie Hauer, Christina Imming, Mitul Kapadia, Ruchi Kaushik, Laura M. Panko, John Takayama, Justin C. Wheeler.
Funding sources: None.
Appendix. Supplementary Data
References
- Children with medical complexity: an emerging population for clinical and research initiatives.Pediatrics. 2011; 127: 529-538https://doi.org/10.1542/peds.2010-0910
- Trends in care practices, morbidity, and mortality of extremely preterm neonates, 1993-2012.JAMA. 2015; 314: 1039-1051https://doi.org/10.1001/jama.2015.10244
- 20-year survival of children born with congenital anomalies: a population-based study.Lancet. 2010; 375: 649-656https://doi.org/10.1016/S0140-6736(09)61922-X
- Patterns and costs of health care use of children with medical complexity.Pediatrics. 2012; 130: e1463-e1470https://doi.org/10.1542/peds.2012-0175
American Board of Pediatrics. Entrustable professional activities for general pediatrics. 2019. Available at: https://www.abp.org/entrustable-professional-activities-epas. Accessed May 30, 2019.
- Training health care professionals for 21st-century practice: a systematic review of educational interventions on chronic care.Acad Med. 2015; 90: 1561-1572https://doi.org/10.1097/ACM.0000000000000773
- Challenges and potential solutions to educating learners about pediatric complex care.Acad Pediatr. 2014; 14: 603-609https://doi.org/10.1016/j.acap.2014.06.004
- The recommended description of an entrustable professional activity: AMEE Guide No. 140.Med Teach. 2021; 43: 1106-1114https://doi.org/10.1080/0142159X.2020.1838465
- Nuts and bolts of entrustable professional activities.J Grad Med Educ. 2013; 5: 157-158
American Board of Pediatrics. Entrustable Professional Activities for General Pediatrics. Available at: https://www.abp.org/entrustable-professional-activities-epas. Accessed June 27, 2020.
- Development of entrustable professional activities for hospice and palliative medicine fellowship training in the United States.J Pain Symptom Manage. 2017; 54 (e1): 609-616https://doi.org/10.1016/j.jpainsymman.2017.07.003
- Care of the complex chronically ill child by generalist pediatricians: Lessons learned from pediatric palliative care.Hosp Pediatr. 2013; 3: 129-138https://doi.org/10.1542/hpeds.2012-0047
- Core curricular priorities in the care of children with medical complexity: a North American modified Delphi study.Acad Pediatr. 2020; 20: 558-564https://doi.org/10.1016/j.acap.2020.01.014
- Developing entrustable professional activities for entry into clerkship.Acad Med. 2016; 91: 247-255https://doi.org/10.1097/ACM.0000000000000988
The Accreditation Council for Graduate Medical Education and The American Board of Pediatrics. The Pediatrics Milestones Project. 2017. Available at: http://www.acgme.org/Specialties/Milestones/pfcatid/16/Pediatrics. Accessed August 10, 2017.
- Three approaches to qualitative content analysis.Qual Health Res. 2005; 15: 1277-1288https://doi.org/10.1177/1049732305276687
- Constructing approaches to Entrustable Professional Activity development that deliver valid descriptions of professional practice.Teach Learn Med. 2020; 33: 1-9https://doi.org/10.1080/10401334.2020.1784740
- Children's complex care needs: a systematic concept analysis of multidisciplinary language.Eur J Pediatr. 2018; 177: 1641-1652https://doi.org/10.1007/s00431-018-3216-9
- What is a geriatrician? American Geriatrics Society and Association of Directors of Geriatric Academic Programs end-of-training Entrustable Professional Activities for geriatric medicine.J Am Geriatr Soc. 2014; 62: 924-929https://doi.org/10.1111/jgs.12825
- Longitudinal assessment of resident performance using entrustable professional activities.JAMA Netw Open. 2020; 3 (Jan.DOI.org (Crossref))e1919316https://doi.org/10.1001/jamanetworkopen.2019.19316
Article Info
Publication History
Published online: September 29, 2021
Accepted:
September 6,
2021
Received:
June 22,
2021
Footnotes
Conflicts of Interest: The authors have no conflicts of interest to disclose.
Identification
Copyright
Copyright © 2021 by Academic Pediatric Association
