Advertisement

The Early Effects of the COVID-19 Pandemic on Pediatric Resident Education: A National Assessment

Published:November 10, 2022DOI:https://doi.org/10.1016/j.acap.2022.11.003

      Abstract

      Purpose

      Residency programs must ensure resident competence for independent practice. The COVID-19 pandemic disrupted healthcare delivery, impacting pediatric residencies. This study examines the impact on pediatric resident education.

      Methods

      The authors conducted a mixed methods national survey of pediatric residency program directors (PDs) from May- July 2020. Data analysis included descriptive statistics, χ2, Wilcox rank sum tests. Multivariable modeling identified factors associated with resident preparation for more senior roles. Thematic analysis was performed on open-ended questions about PD COVID-19 pandemic recommendations to peers, Accreditation Council for Graduate Medical Education and American Board of Pediatrics.

      Results

      Response rate was 55% (110/199). PDs reported the COVID-19 pandemic negatively affected inpatient (n= 86, 78.2%), and outpatient education (n=104, 94.5%), procedural competence (n=64; 58.2%), and resident preparation for more senior roles (n= 50, 45.5%). In bivariate analyses, increasingly negative impacts on inpatient and outpatient education were associated with an increasingly negative impact on resident preparation for more senior roles (p=0.03, p=0.008), these relationships held true in multivariable analysis. Qualitative analysis identified 4 themes from PD recommendations: 1) Clear communication from governing bodies and other leaders; 2) Flexibility within programs and from governing bodies; 3) Clinical exposure is key for competency development; 4) Online platforms are important for education, communication, and support.

      Conclusions

      The COVID-19 pandemic negatively impacted inpatient and outpatient education. When these were more negatively impacted, resident preparation for more senior roles was worse. Highlighting the importance of competency based medical education to tailor experiences ensuring each resident is competent for independent practice.

      What's New

      The COVID-19 pandemic negatively impacted inpatient and outpatient clinical education which negatively impacted resident preparation for more senior roles. This highlights the importance of competency based medical education to ensure each graduating resident is competent for independent practice.

      Key Words

      Abbreviations

      COVID-19
      Coronavirus disease 2019
      APPD
      Association of Pediatric Program Directors
      PD
      Program Director
      GME
      Graduate Medical Education
      ACGME
      Accreditation Council for Graduate Medical Education
      ABP
      American Board of Pediatrics
      ABIM
      American Board of Internal Medicine
      The COVID-19 pandemic has disrupted healthcare delivery worldwide, impacting graduate medical education (GME)
      • Clements JM
      • Burke JR
      • Hope C
      • et al.
      The quantitative impact of COVID-19 on surgical training in the United Kingdom.
      • Kim CS
      • Lynch JB
      • Cohen S
      • et al.
      One Academic Health System's Early (and Ongoing) Experience Responding to COVID-19: Recommendations From the Initial Epicenter of the Pandemic in the United States.
      • Liang ZC
      • Ooi SBS
      • Wang W.
      Pandemics and Their Impact on Medical Training: Lessons From Singapore.
      • Naifeh MM
      • Stevenson MD
      • Abramson EL
      • Aston CE
      • Li ST.
      Early Impact of the COVID-19 Pandemic on Pediatric Resident Workforce.
      . In pediatrics, major disruptions in clinical experiences included dramatic initial decreases in direct resident-patient interactions with a shift to telemedicine, and redeployment to care for adult patients
      • Naifeh MM
      • Stevenson MD
      • Abramson EL
      • Aston CE
      • Li ST.
      Early Impact of the COVID-19 Pandemic on Pediatric Resident Workforce.

      Boyle P. Hospitals Innovate Amid Dire Nursing Shortages. AAMC. https://www.aamc.org/news-insights/hospitals-innovate-amid-dire-nursing-shortages. Published 2021. Accessed.

      • Jumreornvong O
      • Yang E
      • Race J
      • Appel J
      Telemedicine and Medical Education in the Age of COVID-19.
      • Chiel L
      • Winthrop Z
      • Winn AS.
      The COVID-19 Pandemic and Pediatric Graduate Medical Education.

      Hegland A, Kemp C, Miller D, Sulaski-Wykoff A. Pediatricians show their mettle as coronavirus sweeps the nation. https://www.aappublications.org/news/2020/04/21/covidvignettes042120. Published 2020. Accessed November 6, 2020.

      • Leary JC
      • Fox LA
      • Rakoczy K
      • Ross SLP.
      We Got This and We Don't: Pediatricians Going to Battle for the "Big Children" of COVID-19.
      . In response, both the American Board of Pediatrics (ABP) and the Accreditation Council for Graduate Medical Education (ACGME) modified training requirements

      Accreditation Council for Graduate Medical Education. ACGME Response to the Coronavirus (COVID-19). https://acgme.org/Newsroom/Newsroom-Details/ArticleID/10111/ACGME-Response-to-the-Coronavirus-COVID-19. Published 2020. Accessed June 1, 2020.

      ,

      Accreditation Council of Graduate Medical Education. Three Stages of GME During the COVID-19 Pandemic. https://acgme.org/COVID-19/Three-Stages-of-GME-During-the-COVID-19-Pandemic. Published 2020. Accessed June 2, 2020.

      to allow flexibility, and programs implemented innovations in virtual learning, telehealth, and staffing structure
      • Rebecca Blankenburg MD, M
      • Patricia Poitevien MD, M
      • Javier Gonzalez del Rey MD, M
      • CAE LD
      Virtual Caf´e Study Team. Virtual Cafes: An Innovative Way for Rapidly Disseminating Educational Best Practices and Building Community During COVID-19.
      • Price DW
      • Campbell CM.
      Rapid Retooling, Acquiring New Skills, and Competencies in the Pandemic Era: Implications and Expectations for Physician Continuing Professional Development.
      • Rogers A
      • Lynch K
      • Toth H
      • Weisgerber M.
      Patient and Family Centered (Tele)rounds: The use of video conferencing to maintain family and resident involvement in rounds.
      . It is unknown if these disruptions during a critical time in skill-building for pediatric trainees, will impact the ability of residents to take on level-appropriate responsibilities, or the readiness of graduates for independent practice
      • Naifeh MM
      • Stevenson MD
      • Abramson EL
      • Aston CE
      • Li ST.
      Early Impact of the COVID-19 Pandemic on Pediatric Resident Workforce.
      ,

      Accreditation Council for Graduate Medical Education. ACGME Program Requirements for Graduate Medical Education in Pediatrics. In: ACGME, ed2020.

      ,
      • Weiss PG
      • Li ST.
      Leading Change to Address the Needs and Well-Being of Trainees During the COVID-19 Pandemic.
      .
      The ACGME tasks program directors with administering and maintaining a learning environment conducive to educating residents in each competency domain. Additionally, PDs must submit a summative evaluation, including a statement about readiness to progress to the next level, for each resident annually

      Accreditation Council for Graduate Medical Education. ACGME Program Requirements for Graduate Medical Education in Pediatrics. In: ACGME, ed2020.

      . For graduating residents, PDs must submit a final evaluation verifying that each graduate has demonstrated the necessary knowledge, skills and behaviors necessary for autonomous practice

      Accreditation Council for Graduate Medical Education. ACGME Program Requirements for Graduate Medical Education in Pediatrics. In: ACGME, ed2020.

      . PDs, as stewards of the educational mission were forced to quickly respond to changes in the clinical, and educational environments to assure continued effectiveness of their training programs
      • Weiss PG
      • Li ST.
      Leading Change to Address the Needs and Well-Being of Trainees During the COVID-19 Pandemic.
      .
      Building on our previous report on the disruptions in direct patient care reported by PDs and resident COVID-19 illness, this work used a mixed methods approach to analyze a survey of pediatric PDs to describe the changes in educational experience for residents, the effects of these disruptions on resident readiness for more senior roles, and PD recommendations for residency training moving forward. We chose a mixed methods approach to allow us to better understand how the COVID-19 pandemic affected resident and residency program experiences including resident preparation for more senior roles and autonomous practice.

      Methods

      Survey administration

      Using the LimeSurvey platform, we performed a nationally distributed, cross-sectional, electronic survey of pediatric PDs from May-July 2020. The survey was distributed via the Association of Pediatric Program Directors (APPD) list of member programs, which includes 95% (199/210) of all pediatric programs in the United States. We distributed 8 survey reminders to non-responders. We received institutional review board approval from the University of Oklahoma Health Sciences Center.

      Survey development

      We (current/former PDs and associate PDs (APDs) from 4 institutions) developed the survey after extensive literature review
      • Naifeh MM
      • Stevenson MD
      • Abramson EL
      • Aston CE
      • Li ST.
      Early Impact of the COVID-19 Pandemic on Pediatric Resident Workforce.
      . We conducted cognitive interviews followed by pilot testing of the survey with 7 current/former PDs and APDs. The APPD Research and Scholarship Learning Community and APPD LEARN (Longitudinal Educational Assessment Research Network), comprised of educators with expertise in survey methodology, suggested additional revisions. We did not analyze pilot answers. Resident workforce impacts were reported previously
      • Naifeh MM
      • Stevenson MD
      • Abramson EL
      • Aston CE
      • Li ST.
      Early Impact of the COVID-19 Pandemic on Pediatric Resident Workforce.
      .

      Survey content

      Our survey included questions about the impact of the COVID-19 pandemic on various residency program areas (e.g., inpatient and outpatient clinical education, procedural competence, resident preparation for more senior roles), changes in educational modalities (e.g., shifting to virtual programming), and use of ABP waivers

      American Board of Pediatrics. Training Disruptions Due to COVID-19 for Residents and Fellows. https://www.abp.org/sites/abp/files/pdf/cic-training-disruptions-covid-2021.pdf. Published 2020. Accessed 2022.

      for inpatient, outpatient, and continuity clinic experiences. We also asked PDs this open-ended survey question: “What COVID-19 pandemic specific recommendations would you like to share with other program directors, the ABP or the ACGME?” COVID-19 disease burden was measured using the New York Times website

      The New York Times. Coronavirus in the U.S.: Latest Map and Case Count. https://www.nytimes.com/interactive/2020/us/coronavirus-us-cases.html. Published 2020. Accessed May 28, 2020.

      reported cumulative number of cases per 100,000 population in the county where the residency program is located on the date the survey was returned. Several questions about the impact of the COVID-19 pandemic on specific aspects of pediatric residency programs were measured using Likert-type response scales ranging from very negative to very positive. For some questions, for ease of reporting, the very negative and negative responses were combined and are reported as negative and the very positive and positive responses were combined and are reported as positive. Inpatient and outpatient volume was measured using a Likert type scale. The question asked about volume in specific areas (appendix 2 survey question 6), and we assigned numerical values from -2 (corresponding to very negative) to 2 (corresponding to very positive) to each response to determine the mean for each clinical area. Traditionally inpatient areas like the neonatal and pediatric intensive care units, inpatient wards, emergency department, and inpatient subspecialty care were all included in the hospital-based volume variable while outpatient subspecialty care, continuity clinic sick and well visits were included in the outpatient volume variable.

      Quantitative data analysis

      We present group statistics as count and percent for categorical measures and median for continuous measures unless stated otherwise. We used Pearson chi-squared tests for categorical measures and Wilcox rank sum tests for continuous measures for group comparisons. We examined bivariate relationships between demographic variables like region, program size, setting, pandemic emergency stage declared, COVID-19 impact on various residency program areas including inpatient and outpatient education, resident redeployment, patient volume, and resident preparation for more senior roles. Beginning with variables with p < 0.1 in the bivariate analyses and those predetermined as possibly relevant factors, we used stepwise generalized linear regression to build a minimal best-fit model to determine factors associated with resident preparation for more senior roles with a threshold of a likelihood ratio p value < 0.05 to remain in the model. We used R for calculations (version 4.0.3).

      Qualitative data analysis

      We analyzed open-ended survey responses using thematic content analysis, an inductive qualitative method used to provide a descriptive presentation of data
      • Anderson R.
      Intuitive Inquiry: A Transpersonal Approach.
      . We used open-coding to develop an initial code list by identifying commonly used phrases and significant statements
      • Braun V
      • Clarke V.
      Qualitative Content Analysis: a guide to paths not taken.
      . We reviewed codes as a group, using an iterative process of reviewing, labeling, and re-labeling codes into categories to inform emergent themes. Themes were then named and defined based on the research team's interpretation of the data
      • Erlandson DA
      • Harris E.L.
      • Skipper B.L.
      • Allen S.D.
      Doing Naturalistic Inquiry: a guide to methods.
      . We selected final themes through consensus.

      Results

      We received responses from 55% (110/199) of programs. Over half (55%) of responses were received 5/26/2020-6/9/2020; we closed the survey on 7/27/2020. We previously published demographics of responding programs and impact of COVID-19 on clinical volume of patients in pediatric residency programs including resident redeployment
      • Naifeh MM
      • Stevenson MD
      • Abramson EL
      • Aston CE
      • Li ST.
      Early Impact of the COVID-19 Pandemic on Pediatric Resident Workforce.
      . Briefly, we found no differences between responding and non-responding programs regarding program size, geographic location, or program setting, except no military programs (of 6 possible) participated. Timing of survey completion did not differ by program size or geographic location. At the time of survey completion, the majority (63%; n=69) of programs reported being more than a week past their initial peak COVID-19 burden, with another 20% reporting they were at the initial peak or plateau. Cumulative disease burden ranged from 9.5 to 360/100,000 with a median of 78/100,000. (Supplemental Figure 1).

      Impact on pediatric resident education

      We found that the impact of the COVID-19 pandemic varied across domains of resident education (Figure 1). We previously reported that hospital-based and outpatient patient volume decreased in most clinical areas except newborn / neonatal intensive care units (NICU). In our current study, most PDs reported clinical education was negatively affected by the COVID-19 pandemic in both inpatient (n= 86, 78.2%) and outpatient (n=104, 94.5%) areas. PDs also reported the COVID-19 pandemic negatively affected procedural competence (n= 64, 58.2%) and residents’ preparation for more senior roles (n= 50, 45.5%). Most PDs reported quality of patient care was unchanged (n=87, 79.1%), and had mixed opinions on effect of the COVID-19 pandemic on the balance of service and education, resident scholarly activity and educational conferences. Most PDs (n=65, 59.1%) felt that the COVID-19 pandemic had no effect on compliance with clinical and educational work hours, with a significant minority (n=45, 40.9%) feeling that it positively affected compliance.
      Figure 1:
      Figure 1Impact Of COVID-19 Pandemic on Pediatric Resident Education
      Many programs changed how they conducted educational activities (Table 1). For most programs, bedside teaching (n=94, 85.5%), and direct observations (n=95, 86.4%) were decreased or completely cancelled, with some reporting increased remote direct observations (n=54, 49.1%). Similarly, in-person resident educational conferences were either completely cancelled or decreased (n=102, 92.7%) and remote live didactics were increased (n=103, 93.6%).
      Table 1Changes in Educational Modalities and Program Administration Activities
      Educational ModalityCompletely Cancelled(Very) DecreasedUnchanged(Very) Increased
      Bedside teaching6 (5.5%)88 (80%)14 (12.7%)2 (1.8%)
      In person Direct Observations20 (18.2%)75 (68.2%)12 (10.9%)3 (2.7%)
      Remote Direct Observations (via telemedicine)7 (6.4%)8 (7.2 %)32 (29.1%)54 (49.1%)
      In-person, resident educational conferences (noon conference, morning report)71 (64.5%)31(28.2%)7 (6.4%)1 (0.9%)
      In-person Departmental educational conferences (such as Grand Rounds)83 (75.5%)21 (19.1%)6 (5.5%)0 (0.0%)
      Online pre-recorded lectures5 (4.5%)4 (3.6%)71 (64.5%)30 (27.3%)
      Remote Live Didactics (e.g. Zoom, Microsoft Teams)2 (1.8%)4 (3.6%)1 (0.9%)103 (93.6%)
      Mock code style medical or trauma simulations50 (45.5%)31 (28.2%)16 (14.5%)13 (11.8%)
      Skills/procedure workshops60 (54.5%)28 (25.5%)18 (16.4%)4 (3.6%)
      When asked to quantify their use of ABP waivers for required rotations, the majority (66.4%; n=73) of PDs reported at least 1-10% of their residents would finish the 2019-2020 academic year completing fewer than the required 36 continuity clinics and nearly one quarter (22.7%, n=25) reported that more than 75% of their residents would finish the year completing fewer than 36 continuity clinics. In contrast, the vast majority of PDs reported they did not make use of waivers for other clinical rotation requirements, including the required outpatient rotations in adolescent medicine and developmental and behavioral pediatrics (92.7%, n= 102) or required hospital-based rotations like NICU, PICU and ED (92.7%, n=102).

      Factors associated with pediatric resident preparation for more senior roles

      Overall, PDs reported resident preparation for more senior roles was negatively impacted by the COVID-19 pandemic (figure 1). Bivariate analyses showed resident preparation for more senior roles was associated with region and pandemic emergency stage declared, inpatient and outpatient clinical education and outpatient clinical volume, and mandatory redeployment (Table 2). In our multivariable analysis, we found increasingly negative impacts on inpatient (p=0.03) and outpatient clinical education (p=0.009) were associated with increasingly negative impacts on resident preparation for more senior roles (Table 3). Inpatient and outpatient volume, though not significantly associated with resident preparation for more senior roles in our multivariable model, were correlated with inpatient (r=0.1) and outpatient (r=0.5) education, respectively.
      Table 2Bivariate Analyses: Factors Associated with Resident Preparation for More Senior Roles
      Resident Preparation

      for More Senior Roles
      (Very)

      Negative
      Unchanged(Very)

      Positive
      p-value
      Overall50 (45.5%)51 (46.4%)9 (8.2%)
      Region0.020
       Northeast17 (45.9%)13 (35.1%)7 (18.9%)
       Midwest17 (54.8%)12 (38.7%)2 (6.5%)
       South8 (30.8%)18 (69.2%)0 (0.0%)
       West8 (50.0%)8 (50.0%)0 (0.0%)
      Program size0.23
       Small17 (48.6%)15 (42.9%)3 (8.6%)
       Medium22 (52.4%)19 (45.2%)1 (2.4%)
       Large11 (33.3%)17 (51.5%)5 (15.2%)
      Program setting0.051
       In a hospital that also cares for adults27 (62.8%)12 (27.9%)4 (9.3%)
       In a system that also cares for adults12 (34.3%)20 (57.1%)3 (8.6%)
       Independent Children's Hospital7 (36.8%)12 (63.2%)0 (0.0%)
       Community Hospital4 (30.8%)7 (53.8%)2 (15.4%)
      Pandemic Emergency Stage0.042
       Stage 1 (business as usual)2 (25.0%)6 (75.0%)0 (0.0%)
       Stage 2 (increased clinical demand)33 (53.2%)27 (43.5%)2 (3.2%)
       Stage 3 (switch to focus on patient care)14 (38.9%)15 (41.7%)7 (19.4%)
      Inpatient Clinical Ed0.028
       (Very) Negative45 (52.3%)38 (44.2%)3 (3.5%)
       Unchanged3 (18.8%)11 (68.8%)2 (12.5%)
       (Very) Positive2 (25.0%)2 (25.0%)4 (50.0%)
      Outpatient Clinical Ed0.0008
       Very Negative22(57.9%)15(39.5%)1(2.6%)
       Negative28(42.4%)33(50.0%)5(7.6%)
       Unchanged, Positive, Very.Positive03(50.0%)3(50.0%)
      Volume
       Hospital based [mean±SD]-1.0 ± 0.3-0.9 ± 0.3-1.0 ± 0.50.33
       Outpatient [mean±SD]-1.7 ± 0.5-1.5 ± 0.5-1.1 ± 0.80.040
      COVID Case Burden75 [45-146]70 [50-105]92 [92-92]0.43
      Mandatory Redeployment0.0016
       Yes9 (50.0%)4 (22.2%)5 (27.8%)
       No41 (44.6%)47 (51.1%)4 (4.3%)
      Resident Illness0.48
       None16 (43.2%)19 (51.4%)2 (5.4%)
       1-10% mild28 (50.9%)23 (41.8%)4 (7.3%)
       More6 (33.3%)9 (50.0%)3 (16.7%)
      Table 3Factors Associated with Resident Preparation for More Senior Roles
      Model termβ95% confidence intervalWald test, p valueLikelihood Ratio test, p value
      (Intercept)-0.83[-1.07,-0.60]< 0.0001<0.0001
      Impact on Inpatient Clinical Education0.025
      Very Negative, Negative0
      Unchanged0.48[0.10,0.87]0.016
      Positive, Very Positive0.84[0.33,1.36]0.0017
      Impact on Outpatient Clinical Education0.0088
      Very Negative0
      Negative0.31[0.01,0.60]0.042
      Unchanged, Positive, Very Positive1.33[0.73,1.94]< 0.0001

      PD recommendations

      Thematic analysis of PD recommendations to their peers, the ACGME and ABP revealed four themes related to resident competence (Table 4): 1) Increased communication and transparency, including in guidance from accrediting bodies (ABP and ACGME) and national organizations (APPD), is vital for PDs: “Understanding and clear direction from these regulatory bodies has been very appreciated as it allows us to see a clear path forward and plan accordingly.”. 2) Flexibility is important both within residency programs and especially from governing bodies, including in meeting ACGME/ABP requirements: “I would truly encourage flexibility and trust that an accredited program can evaluate competencies in all residents without the need for a possible full 36 months.”. Some responding PDs used this opportunity to make suggestions for permanent increased flexibility and a move towards competency- based education. 3) Pediatric residency program leadership must ensure sufficient, in person clinical exposure to the typical breadth and depth of pediatric care to achieve competence. Simulation, telemedicine, and care for adult patients may supplement clinical learning, but are not adequate replacements for traditional in-person pediatric experiences: “Keep residents involved in learning opportunities and patient care opportunities at all cost!”. 4) Continue to use online platforms to facilitate education, communication, and support: “Changing educational curricula to virtual format, …, making virtual social connections”.
      Table 4Program Director Recommendations to Their Peers and Governing Bodies During the COVID-19 Pandemic
      ThemeRepresentative Quotes
      Increased communication and transparency, including in guidance from accrediting bodies, (ABP and ACGME) and national organizations (APPD) is paramount.
      • The ACGME and ABP were rapidly responsive to our needs as programs, and made excellent decisions in the face of what was so unexpected and chaotic. Communication was excellent and well thought out. We are very grateful for that.
      • APPD chats were very helpful, more official guidance from ACGME that was more detailed than the 3 broad characterizations of effect on patient care that came directly from RRC
        residency review committee
        . Reducing clinic numbers was very useful, providing some additional curricula and learning tools from the ABP would have been helpful as well.
      • Understanding and clear direction from these regulatory bodies has been very appreciated as it allows us to see a clear path forward and plan accordingly. For example, the ABPs support and understanding of modified operations and a means of applying for waivers was tremendously helpful. The AAMCs clear stance on virtual interviews early on has allowed us to start planning for recruitment season. I know that much uncertainty remains, but the support, understanding, and clear direction from these bodies is very appreciated.
      Flexibility is important both within residency programs and especially from governing bodies, including in meeting ACGME/ ABP requirements.
      • Be flexible and adaptable. Take the silver linings where they are – virtual conferences and meetings has really been a plus. I've learned that I was much more capable of handling a disaster than I ever thought I was.
      • The leniency the ACGME showed with the decrease in continuity clinics was perfect. They should continue to grant us leeway in meeting requirements.
      • I would truly encourage flexibility and trust that an accredited program can evaluate competencies in all residents without the need for a possible full 36 months. Many times there is not much difference in a resident that has completed 31 vs. 33 vs. 36 months for example. We should adapt requirements to be more flexible just as we are doing for the pandemic. This piece should be a positive permanent outcome.
      • I think we will still need to look at all the program requirements going forward, since some institutions will not be able to return to regular rotations for some time - this particularly effects ability to do procedures, to rotate in subspecialty clinics that might have different rules for trainees and continuity clinic visits.
      • Have a library of online self- study Individualized Curriculum options available for use.
      Pediatric residency program leadership must ensure sufficient, in person clinical exposure to the typical breadth and depth of pediatric care to achieve competence. Simulation, telemedicine, and care for adult patients may supplement but are not adequate replacements for traditional in person experiences.
      • PDs need to advocate for residents to stay involved with patient care during a pandemic. They need to learn how to care for COVID patients, of course with good precautions and safety.
      • Keep residents involved in learning opportunities and patient care opportunities at all cost! Having skilled caring physicians at home was harder than protecting them in the hospital. Especially keep the outpatient clinic setting going, sorry our residents spent so many weeks without outpatient care experience (although there was very little to be had at the time.
      • Resident physicians need to see patients in person. Institutions are cutting funding for residencies (perhaps due to COVID related losses, perhaps because they are looking at their next quarter, annual report… own bonus pool); they are not looking long term at physician shortages and regardless of what they "say", they do not "care" the same as those who are physicians who see patients regularly and teach residents.
      • Although telehealth has been a means to care for children during the pandemic, it in no way offers the same level of excellence in care that face to face patient contact is capable of providing.
      • It would be inappropriate to follow actions making telehealth a mandatory part of our RRC
        residency review committee
        's requirements, as this has not been a means to provide excellence in care. It is appealing because it is different and utilizes technology, but it is incomplete and not a good substitute for traditional care models.
      Continue to use online platforms to facilitate education, communication, and support
      • Educational (zoom) conferences were a source of wellness in maintaining some semblance of normalcy.
      • Re-arranging schedules, changing educational curricula to virtual format, getting updates from 1 person in the leadership, making virtual social connections, being available.
      low asterisk residency review committee

      Discussion

      Early in the COVID-19 pandemic, nearly all participating pediatric program directors reported a negative impact on resident preparation for more senior roles. This was especially true when the PD reported that inpatient and outpatient clinical education were more negatively affected. During the pandemic, decreases in the number of both inpatient and outpatient visits in hospitals and clinics were widely reported in pediatrics
      • Naifeh MM
      • Stevenson MD
      • Abramson EL
      • Aston CE
      • Li ST.
      Early Impact of the COVID-19 Pandemic on Pediatric Resident Workforce.
      ,
      • Chiel L
      • Winthrop Z
      • SW A
      The COVID-19 Pandemic and Pediatric Graduate Medical Education.
      • Santoli JM
      • Lindley MC
      • DeSilva MB
      • et al.
      Effects of the COVID-19 Pandemic on Routine Pediatric Vaccine Ordering and Administration - United States, 2020.
      • Abuali M
      • Bonner R
      • Irigoyen M.
      Operationalizing an academic pediatric practice during the COVID-19 crisis.
      . While our multivariable analysis, did not find an association between decreases in hospital based and outpatient volume with a negative impact on resident preparation for more senior roles, decreases in volume were associated with a negative impact on inpatient and outpatient education. Assessment of the effect on education likely included other factors in addition to volume, making volume alone a less sensitive predictor.
      Changes in the clinical environment with a resultant negative effect on resident preparation for their next phase of practice during the COVID-19 pandemic have been described in other subspecialties, such as anesthesia and surgical specialties
      • Columbus AB
      • Breen EM
      • Abelson JS
      • et al.
      What Just Happened to My Residency?" The Effect of the Early Coronavirus Disease 2019 Pandemic on Colorectal Surgical Training.
      • Van Heest A
      • Brandt AM
      • Dyer G
      • Homer CJ
      • Murray PM.
      COVID-19: Impact on Orthopaedic Graduate Medical Education in the U.S.: AOA Critical Issues Symposium.
      • Mazandi V
      • Gordon E.
      Resident Education and Redeployment During a Disaster.
      • Abdelsattar JM
      • Coleman JR
      • Nagler A
      • et al.
      Lived Experiences of Surgical Residents During the COVID-19 Pandemic: A Qualitative Assessment.
      . Many studies describing resident experiences during the pandemic mention loss of typical experiences and increases in alternative learning experiences including virtual education and education around public health measures and critical care
      • Lucey CR
      • Johnston SC.
      The Transformational Effects of COVID-19 on Medical Education.
      ,
      • Hall AK
      • Nousiainen MT
      • Campisi P
      • et al.
      Training disrupted: Practical tips for supporting competency-based medical education during the COVID-19 pandemic.
      . Pediatrics may be similar in some ways to these surgical specialties as it is difficult to learn pediatric specific care in the absence of pediatric patients.
      We found that most programs (66%) requested ABP waivers for some of their residents and that continuity clinic was significantly disrupted in most pediatric programs during the COVID-19 pandemic. Continuity clinic has long been considered a vital part of residency training in general pediatrics, with a minimum of 36 clinic sessions required annually

      Accreditation Council for Graduate Medical Education. ACGME Program Requirements for Graduate Medical Education in Pediatrics. In: ACGME, ed2020.

      . Pediatrics, like internal medicine, has many subspecialty training programs, including most recently pediatric hospital medicine. The primary goal of pediatric residency training is to train general, outpatient pediatricians who typically practice in an outpatient office setting, making a robust continuity clinic experience key. Maintaining resident outpatient clinic experiences or quickly shifting to offer telehealth experiences for short periods of time will be key in case of future disruptions.
      The long- term effect of so many missed core experiences remains to be seen, but if it recurs, PDs may need to focus on creative augmentation or replacement of some of these experiences with alternative activities such as simulation or alternative practice sites. In response to such unprecedented disruptions to residency training, PDs must lead educational change to ensure resident competence
      • Weiss PG
      • Li ST.
      Leading Change to Address the Needs and Well-Being of Trainees During the COVID-19 Pandemic.
      . In our qualitative analysis, PDs described the importance of residents continuing in person clinical care for patients, emphasizing that there is no real substitute for in person encounters. Interestingly, Columbus et. al. described a survey study of colorectal surgery residents that found that while residents were concerned about a number of missed operative opportunities, the vast majority reported feeling they would be prepared for independent practice at the end of their residency. Whether pediatric residents feel similarly and if that translates into competency is unknown.
      In our qualitative analysis, PDs desired more support from governing bodies in the way of online curricula and guidelines for resident participation in clinical care, which might help to ensure basic standards are met across programs, regardless of regional variation in patient care patterns. The ABP in-training exam scores and ABP initial certifying exam pass rates for 2021 were significantly lower than prior years

      American Board of Pediatrics. Exam Pass Rates Initial Certifying https://www.abp.org/sites/public/files/pdf/exam-pass-rates-init-cert.pdf. Published 2022. Accessed 4/14/2022, 2022.

      . While the exact cause of this drop is unknown, and is likely multifactorial, limitations in hands-on pediatric patient experiences and disruption to in-person educational sessions for residents throughout much of their residency may have played a role. If programs continue to have virtual conferences, innovative virtual learning strategies may be necessary to ensure residents acquire knowledge effectively.
      • Chertoff JD
      • Zarzour JG
      • Morgan DE
      • Lewis PJ
      • Canon CL
      • Harvey JA.
      The Early Influence and Effects of the Coronavirus Disease 2019 (COVID-19) Pandemic on Resident Education and Adaptations.
      In comparing pediatrics to medicine, the American Board of Internal Medicine (ABIM) also reported an overall decrease in pass rates for first time test takers; however, their rates in 2021 were higher than those reported for pediatrics (Pediatrics 81%

      American Board of Pediatrics. Exam Pass Rates Initial Certifying https://www.abp.org/sites/public/files/pdf/exam-pass-rates-init-cert.pdf. Published 2022. Accessed 4/14/2022, 2022.

      versus Medicine 88%

      American Board of Internal Medicine. First-Time Taker Pass Rates. https://www.abim.org/Media/yeqiumdc/certification-pass-rates.pdf. Published 2022. Accessed 4/14/2022, 2022.

      ). This may reflect the fact that internal medicine residency programs were generally able to preserve more aspects of adult in-person care (even if it shifted to COVID related care) compared to pediatrics, where volumes were so dramatically decreased and some pediatric residents needed to care for adult patients.
      In the face of lost experiences during crucial training years, PDs will have to carefully consider what constitutes competence and how to develop and support it going forward.
      • Ten Cate O
      • Schultz K
      • Frank JR
      • et al.
      Questioning medical competence: Should the Covid-19 crisis affect the goals of medical education?.
      The pandemic has highlighted the importance of frequent assessment and inclusion of specific activities in areas where residents are behind
      • Blankenburg R
      • Gonzalez Del Rey J
      • Aylor M
      • et al.
      The Impact of the COVID-19 Pandemic on Pediatric Graduate Medical Education: Lessons Learned and Pathways Forward.
      . The COVID-19 pandemic has accelerated the discussion of competency based medical education
      • Hall AK
      • Nousiainen MT
      • Campisi P
      • et al.
      Training disrupted: Practical tips for supporting competency-based medical education during the COVID-19 pandemic.
      ,
      • Ten Cate O
      • Schultz K
      • Frank JR
      • et al.
      Questioning medical competence: Should the Covid-19 crisis affect the goals of medical education?.
      ,
      • Hauer KE
      • Lockspeiser TM
      • Chen HC.
      The COVID-19 Pandemic as an Imperative to Advance Medical Student Assessment: Three Areas for Change.
      ,
      • Goldhamer MEJ
      • Pusic MV
      • Co JPT
      • Weinstein DF.
      Can Covid Catalyze an Educational Transformation? Competency-Based Advancement in a Crisis.
      . As the pandemic abates, PDs and other leaders in pediatric residency education have an opportunity to re-evaluate the essential components of effective residency education
      • Blankenburg R
      • Gonzalez Del Rey J
      • Aylor M
      • et al.
      The Impact of the COVID-19 Pandemic on Pediatric Graduate Medical Education: Lessons Learned and Pathways Forward.
      . While decisions about promotion have historically been based primarily on acceptable completion of a specific course of training, it will likely be essential to shift our focus to defining acceptable learner outcomes, frequent assessment of learners, and additional support or targeted learning experiences to ensure competency achievement
      • Goldhamer MEJ
      • Pusic MV
      • Co JPT
      • Weinstein DF.
      Can Covid Catalyze an Educational Transformation? Competency-Based Advancement in a Crisis.
      . Our qualitative data supports this shift. PDs appreciated the flexibility to declare residents competent for promotion or graduation despite failure to meet the traditionally accepted course of training. Several PDs suggested a shift to establishing specific benchmarks for competence that are not time-dependent. The COVID-19 pandemic has presented an opportunity to rethink and redesign education to meet the changing, individualized needs of learners and patients. This shift to competence based medical education may require more frequent, specific resident assessments to identify trainee gaps, tailored experiences to facilitate skill building in areas of identified gaps, and/ or one-on-one coaching
      • Ten Cate O
      • Schultz K
      • Frank JR
      • et al.
      Questioning medical competence: Should the Covid-19 crisis affect the goals of medical education?.
      ,
      • Hauer KE
      • Lockspeiser TM
      • Chen HC.
      The COVID-19 Pandemic as an Imperative to Advance Medical Student Assessment: Three Areas for Change.
      • Goldhamer MEJ
      • Pusic MV
      • Co JPT
      • Weinstein DF.
      Can Covid Catalyze an Educational Transformation? Competency-Based Advancement in a Crisis.
      • Deiorio NM
      • Carney PA
      • Kahl LE
      • Bonura EM
      • Juve AM.
      Coaching: a new model for academic and career achievement.
      .

      Limitations

      Our study has several limitations. Despite a reasonably robust response rate, we did not have any responses from military programs who may have unique concerns. Our survey was conducted early in the course of the pandemic and does not reflect its longitudinal impact, though does importantly report PD perceptions at that time and can give important insight about how to approach future significant, disruptive events and challenges us to think about the ways in which we need to support trainees from this period in an ongoing way. To that end, we plan to repeat our survey to assess ongoing impacts of the pandemic and the development of competency as it evolves. Another possible limitation is that our survey sought PD perceptions of educational components and resident preparation for more senior roles. While perceptions are subjective, it is important to remember that the ACGME and the ABP rely on PDs to assert residents are competent for advancement to the next level of training and for independent practice at graduation. Finally, while our study could not directly assess the impact of the changes, program directors clearly value direct communication and flexibility from accrediting bodies to address changes in the clinical environment and needs of learners.

      Conclusions

      The early phase of the COVID-19 pandemic caused significant disruption of both inpatient and outpatient clinical education that negatively impacted procedural competence and pediatric resident preparation for more senior roles. Maintaining high quality educational environments and assuring continued patient care interactions may help mitigate these negative effects during future disruptive events in healthcare. Tailoring resident assessment and training to assure competence prior to graduation will be important. It is possible that the loss of so many typical training experiences during the pandemic, catalyzed a shift towards competency based medical education. Finally, our data suggest a more robust discussion about what constitutes competence by pediatric educational leaders is in order.

      Funding Support

      No funding was secured for this study

      Declaration of Competing Interest

      The authors have no conflicts of interest relevant to this article to disclose

      Acknowledgement

      The authors would like to acknowledge Janis Campbell, Ph.D. for her help in gathering the COVID-19 disease burden data and Kendra Sikes, CCRP, CCRC for her editorial assistance. We would also like to acknowledge the APPD and Dennis West for their support in facilitating this project, APPD's Research and Scholarship Learning Community for their review of the survey and all of the Program Directors who participated by completing the survey.

      References

        • Clements JM
        • Burke JR
        • Hope C
        • et al.
        The quantitative impact of COVID-19 on surgical training in the United Kingdom.
        BJS Open. 2021; 5
        • Kim CS
        • Lynch JB
        • Cohen S
        • et al.
        One Academic Health System's Early (and Ongoing) Experience Responding to COVID-19: Recommendations From the Initial Epicenter of the Pandemic in the United States.
        Acad Med. 2020; 95: 1146-1148
        • Liang ZC
        • Ooi SBS
        • Wang W.
        Pandemics and Their Impact on Medical Training: Lessons From Singapore.
        Academic Medicine. 2020; 17: 17
        • Naifeh MM
        • Stevenson MD
        • Abramson EL
        • Aston CE
        • Li ST.
        Early Impact of the COVID-19 Pandemic on Pediatric Resident Workforce.
        Pediatrics. 2021; 148
      1. Boyle P. Hospitals Innovate Amid Dire Nursing Shortages. AAMC. https://www.aamc.org/news-insights/hospitals-innovate-amid-dire-nursing-shortages. Published 2021. Accessed.

        • Jumreornvong O
        • Yang E
        • Race J
        • Appel J
        Telemedicine and Medical Education in the Age of COVID-19.
        Acad Med. 2020; 95: 1838-1843
        • Chiel L
        • Winthrop Z
        • Winn AS.
        The COVID-19 Pandemic and Pediatric Graduate Medical Education.
        Pediatrics. 2020; 20: 20
      2. Hegland A, Kemp C, Miller D, Sulaski-Wykoff A. Pediatricians show their mettle as coronavirus sweeps the nation. https://www.aappublications.org/news/2020/04/21/covidvignettes042120. Published 2020. Accessed November 6, 2020.

        • Leary JC
        • Fox LA
        • Rakoczy K
        • Ross SLP.
        We Got This and We Don't: Pediatricians Going to Battle for the "Big Children" of COVID-19.
        Acad Pediatr. 2020; 20: 883-884
      3. Accreditation Council for Graduate Medical Education. ACGME Response to the Coronavirus (COVID-19). https://acgme.org/Newsroom/Newsroom-Details/ArticleID/10111/ACGME-Response-to-the-Coronavirus-COVID-19. Published 2020. Accessed June 1, 2020.

      4. Accreditation Council of Graduate Medical Education. Three Stages of GME During the COVID-19 Pandemic. https://acgme.org/COVID-19/Three-Stages-of-GME-During-the-COVID-19-Pandemic. Published 2020. Accessed June 2, 2020.

        • Rebecca Blankenburg MD, M
        • Patricia Poitevien MD, M
        • Javier Gonzalez del Rey MD, M
        • CAE LD
        Virtual Caf´e Study Team. Virtual Cafes: An Innovative Way for Rapidly Disseminating Educational Best Practices and Building Community During COVID-19.
        Academic Pediatrics. 2020;
        • Price DW
        • Campbell CM.
        Rapid Retooling, Acquiring New Skills, and Competencies in the Pandemic Era: Implications and Expectations for Physician Continuing Professional Development.
        Journal of Continuing Education in the Health Professions. 2020; 40: 74-75
        • Rogers A
        • Lynch K
        • Toth H
        • Weisgerber M.
        Patient and Family Centered (Tele)rounds: The use of video conferencing to maintain family and resident involvement in rounds.
        Academic Pediatrics. 2020;
      5. Accreditation Council for Graduate Medical Education. ACGME Program Requirements for Graduate Medical Education in Pediatrics. In: ACGME, ed2020.

        • Weiss PG
        • Li ST.
        Leading Change to Address the Needs and Well-Being of Trainees During the COVID-19 Pandemic.
        Acad Pediatr. 2020; 20: 735-741
      6. American Board of Pediatrics. Training Disruptions Due to COVID-19 for Residents and Fellows. https://www.abp.org/sites/abp/files/pdf/cic-training-disruptions-covid-2021.pdf. Published 2020. Accessed 2022.

      7. The New York Times. Coronavirus in the U.S.: Latest Map and Case Count. https://www.nytimes.com/interactive/2020/us/coronavirus-us-cases.html. Published 2020. Accessed May 28, 2020.

        • Anderson R.
        Intuitive Inquiry: A Transpersonal Approach.
        in: Braud W Anderson R. Transpersonal research methods for the social sciences: Honoring Human Experience. Sage Publications, Thousand Oaks, CA1998
        • Braun V
        • Clarke V.
        Qualitative Content Analysis: a guide to paths not taken.
        Qualitative Health Research. 2006; 3: 77-101
        • Erlandson DA
        • Harris E.L.
        • Skipper B.L.
        • Allen S.D.
        Doing Naturalistic Inquiry: a guide to methods.
        Sage, 1993
        • Chiel L
        • Winthrop Z
        • SW A
        The COVID-19 Pandemic and Pediatric Graduate Medical Education.
        Pediatrics. 2020;
        • Santoli JM
        • Lindley MC
        • DeSilva MB
        • et al.
        Effects of the COVID-19 Pandemic on Routine Pediatric Vaccine Ordering and Administration - United States, 2020.
        MMWR Morb Mortal Wkly Rep. 2020; 69: 591-593
        • Abuali M
        • Bonner R
        • Irigoyen M.
        Operationalizing an academic pediatric practice during the COVID-19 crisis.
        Am J Infect Control. 2020;
        • Columbus AB
        • Breen EM
        • Abelson JS
        • et al.
        What Just Happened to My Residency?" The Effect of the Early Coronavirus Disease 2019 Pandemic on Colorectal Surgical Training.
        Dis Colon Rectum. 2021; 64: 504-507
        • Van Heest A
        • Brandt AM
        • Dyer G
        • Homer CJ
        • Murray PM.
        COVID-19: Impact on Orthopaedic Graduate Medical Education in the U.S.: AOA Critical Issues Symposium.
        J Bone Joint Surg Am. 2021; 103: e65
        • Mazandi V
        • Gordon E.
        Resident Education and Redeployment During a Disaster.
        Anesthesiol Clin. 2021; 39: 353-361
        • Abdelsattar JM
        • Coleman JR
        • Nagler A
        • et al.
        Lived Experiences of Surgical Residents During the COVID-19 Pandemic: A Qualitative Assessment.
        J Surg Educ. 2021; 78: 1851-1862
        • Lucey CR
        • Johnston SC.
        The Transformational Effects of COVID-19 on Medical Education.
        JAMA. 2020; 324: 1033-1034
        • Hall AK
        • Nousiainen MT
        • Campisi P
        • et al.
        Training disrupted: Practical tips for supporting competency-based medical education during the COVID-19 pandemic.
        Medical Teacher. 2020; 42: 756-761
      8. American Board of Pediatrics. Exam Pass Rates Initial Certifying https://www.abp.org/sites/public/files/pdf/exam-pass-rates-init-cert.pdf. Published 2022. Accessed 4/14/2022, 2022.

        • Chertoff JD
        • Zarzour JG
        • Morgan DE
        • Lewis PJ
        • Canon CL
        • Harvey JA.
        The Early Influence and Effects of the Coronavirus Disease 2019 (COVID-19) Pandemic on Resident Education and Adaptations.
        J Am Coll Radiol. 2020; 17: 1322-1328
      9. American Board of Internal Medicine. First-Time Taker Pass Rates. https://www.abim.org/Media/yeqiumdc/certification-pass-rates.pdf. Published 2022. Accessed 4/14/2022, 2022.

        • Ten Cate O
        • Schultz K
        • Frank JR
        • et al.
        Questioning medical competence: Should the Covid-19 crisis affect the goals of medical education?.
        Med Teach. 2021; 43: 817-823
        • Blankenburg R
        • Gonzalez Del Rey J
        • Aylor M
        • et al.
        The Impact of the COVID-19 Pandemic on Pediatric Graduate Medical Education: Lessons Learned and Pathways Forward.
        Acad Med. 2022; 97: S35-S39
        • Hauer KE
        • Lockspeiser TM
        • Chen HC.
        The COVID-19 Pandemic as an Imperative to Advance Medical Student Assessment: Three Areas for Change.
        Acad Med. 2021; 96: 182-185
        • Goldhamer MEJ
        • Pusic MV
        • Co JPT
        • Weinstein DF.
        Can Covid Catalyze an Educational Transformation? Competency-Based Advancement in a Crisis.
        N Engl J Med. 2020; 383: 1003-1005
        • Deiorio NM
        • Carney PA
        • Kahl LE
        • Bonura EM
        • Juve AM.
        Coaching: a new model for academic and career achievement.
        Med Educ Online. 2016; 21: 33480