Advertisement

Flattening the Learning Curve: A Case-Based Video Orientation for a Subspeciality Residency Rotation

Published:August 25, 2020DOI:https://doi.org/10.1016/j.acap.2020.08.013

      Abstract

      Objective

      To determine whether a clinically focused rotation orientation delivered through e-learning videos would be an effective method to improve residents’ clinical knowledge and confidence.

      Methods

      A pre-post study design evaluated for change in knowledge and confidence between a control and intervention group of residents assigned to outpatient pediatric endocrinology rotations at 2 residency programs from July 2017 to March 2019. The intervention group utilized the first morning of the rotation to complete the video curriculum designed to rapidly orient residents to common clinical management tasks in outpatient pediatric endocrinology.

      Results

      A total of 35 of 41 residents (85%) completed the study (control group: 18/19 [95%]; intervention group: 17/22 [77%]). Score increase from pretest to post-test was significantly higher for intervention group compared to control group (+24.7% ± 12.1 vs +5.8% ± 7.9, P < .0001). Confidence increases were significantly higher in the intervention group compared to control group in 3 of 5 topics. Two themes illustrated residents’ perspectives of this e-learning curriculum: 1) increase in foundational clinical knowledge and 2) improvement in efficiency of learning and patient care.

      Conclusions

      This clinically focused rotation orientation delivered through e-learning videos was an effective method to improve residents’ clinical knowledge, without reliance on faculty to deliver this orientation throughout the academic year. Further studies should be pursued in various settings.

      Keywords

      What's New
      Pediatric subspecialty rotations can be challenging for residents given their limited prior knowledge and experience. We describe the development of a simple, effective, and well-received e-learning video curriculum which provides foundational clinical knowledge to residents starting an unfamiliar subspecialty rotation.
      It is well known that trainees can experience steep learning curves at the start of the traditional academic calendar, when interns begin their residencies and current residents advance in their roles and responsibilities.
      • Young JQ
      • Ranji SR
      • Wachter RM
      • et al.
      “July effect”: impact of the academic year-end changeover on patient outcomes: a systematic review.
      ,
      • Haller G
      • Myles PS
      • Taffé P
      • et al.
      Rate of undesirable events at beginning of academic year: retrospective cohort study.
      To mitigate these steep learning curves, specialty-specific orientations and “boot camps” were developed and have been shown to be effective.
      • Blackmore C
      • Austin J
      • Lopushinsky SR
      • et al.
      Effects of postgraduate medical education “boot camps” on clinical skills, knowledge, and confidence: a meta-analysis.
      ,
      • Nishisaki A
      • Hales R
      • Biagas K
      • et al.
      A multi-institutional high-fidelity simulation ‘‘boot camp’’ orientation and training program for first year pediatric critical care fellows.
      However, transitions to unfamiliar clinical environments can occur throughout the academic year, not just at the start.
      • Sobolewski B
      • Kerrey BT
      • Geis GL
      • et al.
      The April effect: a multimedia orientation approach to improve rotation transitions during pediatric residency.
      • Surcouf JW
      • Mumphrey CG
      • Barkemeyer BM
      • et al.
      Neonatal intensive care unit boot camp: a preparatory curriculum for pediatric residents.
      • Merrill A
      • Djuricich AM
      • Drazen JM
      • et al.
      NEJM resident 360.
      Residents starting a new subspecialty rotation may have limited knowledge, skills, and experience required to care for that patient population and will need to adjust to their new roles, responsibilities, and environment of that rotation.
      • Sobolewski B
      • Kerrey BT
      • Geis GL
      • et al.
      The April effect: a multimedia orientation approach to improve rotation transitions during pediatric residency.
      • Surcouf JW
      • Mumphrey CG
      • Barkemeyer BM
      • et al.
      Neonatal intensive care unit boot camp: a preparatory curriculum for pediatric residents.
      • Merrill A
      • Djuricich AM
      • Drazen JM
      • et al.
      NEJM resident 360.
      These difficulties can contribute to a stressful and ineffective learning environment that may have very real consequences for patient care.
      • Sobolewski B
      • Kerrey BT
      • Geis GL
      • et al.
      The April effect: a multimedia orientation approach to improve rotation transitions during pediatric residency.
      • Surcouf JW
      • Mumphrey CG
      • Barkemeyer BM
      • et al.
      Neonatal intensive care unit boot camp: a preparatory curriculum for pediatric residents.
      • Merrill A
      • Djuricich AM
      • Drazen JM
      • et al.
      NEJM resident 360.
      Despite these challenges, rotation orientations in graduate medical education have been understudied.
      The Accreditation Council for Graduate Medical Education requires rotations to explicitly state goals and objectives, but detailed recommendations to orient residents do not exist, and few have studied techniques to orient residents prior to a rotation.

      ACGME. Common program requirements (residency). 2019. Available at: https://acgme.org/Portals/0/PFAssets/ProgramRequirements/CPRResidency2019.pdf. Accessed February 6, 2020.

      Harel et al oriented residents to an adolescent medicine rotation by administering multiple choice questions and discussing answers while Surcouf et al developed a Neonatal Intensive Care Unit “boot camp” orientation consisting of live lectures and simulations.
      • Surcouf JW
      • Mumphrey CG
      • Barkemeyer BM
      • et al.
      Neonatal intensive care unit boot camp: a preparatory curriculum for pediatric residents.
      ,
      • Harel Z
      • Riggs S
      • Vaz R
      • et al.
      A session of multiple choice questions in the orientation process to the adolescent medicine rotation.
      While both interventions were well received by learners, faculty time was required to continuously deliver the information throughout the academic year prior to the start of each rotating resident. To reduce the burdensome nature of faculty-directed learning, Sobolewski et al utilized e-learning videos to orient pediatric residents to an emergency medicine rotation.
      • Sobolewski B
      • Kerrey BT
      • Geis GL
      • et al.
      The April effect: a multimedia orientation approach to improve rotation transitions during pediatric residency.
      This e-learning approach showed promise in providing a uniform, effective delivery tool that did not place significant burden on faculty and allowed time flexibility for residents. However, these videos primarily focused on an administrative orientation, which provides schedules, contact information, expectations, and other logistical information. In contrast, a clinically focused orientation, which provides clinical knowledge needed to manage patients encountered during the rotation, has been shown to be preferred by residents.
      • Grover M
      • Puczynski S
      Residency orientation: what we present and its effect on our residents.
      • Antonoff MB
      • Swanson JA
      • Acton RD
      • et al.
      Improving surgery intern confidence through the implementation of expanded orientation sessions.
      • Fein JA
      • Lavelle J
      • Giardino AP
      Teaching emergency medicine to pediatric residents: a national survey and proposed model.
      To our knowledge, a clinically focused rotation orientation delivered through e-learning has not been formally studied.
      Prior to the study, our approach to resident rotation orientation was to provide an administrative orientation through a syllabus via e-mail. However, over the years, we noted limited improvement in residents’ clinical performance over the course of the rotation. Since residents often have limited clinical knowledge to manage patients at the start of a pediatric subspecialty rotation, we hypothesize that a clinically focused orientation delivered through e-learning would be an effective method to enhance resident learning and confidence. This may allow residents to focus on application of learned material throughout the rotation and allow for more in-depth learning. In contrast, typical rotations require residents to independently pursue this foundational information by navigating complex subspecialty textbooks and resources, which may be inefficient in novice learners. The cognitive load theory supports this hypothesis, suggesting that when learners encounter complex, unfamiliar material, learning can be enhanced by providing “worked” or “partially completed” authentic cases, utilizing multiple modalities (audio and visual), and progressing from lower to higher physical fidelity (eg, progressing from paper-based cases to real patient encounters).
      • Young JQ
      • Van Merrienboer J
      • Durning S
      • et al.
      Cognitive load theory: implications for medical education: AMEE Guide No. 86.
      All of these strategies can be implemented through an e-learning-based clinically focused rotation orientation.
      We describe the design of a video-supported, e-learning orientation curriculum that provides foundational clinical knowledge to residents entering an outpatient pediatric endocrinology rotation, and report its effectiveness in resident learning from qualitative and quantitative perspectives.

      Methods

      We utilized Kern's well-established, 6-step approach to develop this e-learning, video-supported curriculum.
      • Kern DE
      • Thomas PA
      • Howard DM
      • et al.
      Curriculum Development for Medical Education: A Six-Step Approach.
      Step 1: Problem Identification and General Needs Assessment
      Described in the previous section.
      Step 2: Needs Assessment of Targeted Learners
      Our targeted learners are pediatric residents who typically have little to no experience in outpatient pediatric endocrinology. Based on informal discussions among faculty, residents frequently demonstrate difficulties obtaining pertinent history and determining appropriate management for patients throughout the rotation. Residents’ limited knowledge and experience, and difficulty of self-directed learning in an unfamiliar subspecialty were noted by faculty as possible barriers contributing to residents’ minimal improvement. Residents similarly noted that their limited experience and clinical knowledge made learning during subspecialty rotations more difficult. Some residents mentioned that clinical demands limited both self-directed learning and teaching from faculty.
      Step 3: Goals/Objectives
      Our goal was to orient residents to the most common clinical management tasks they would encounter during an outpatient pediatric endocrinology rotation. Consequently, faculty at both institutions unanimously identified the 5 most commonly encountered cases based on historic frequencies (Table 2) and 4 to 8 common management tasks for each case. For each management task, 1 learning objective was created (Appendix 1), utilizing Bloom's taxonomy.
      • Bloom BS
      Taxonomy of Educational Objectives: The Classification of Educational Goals.
      Step 4: Educational Strategies
      For each learning objective, case-based content in a question and answer format was developed. Residents were instructed to pause after each case-based question and record their responses in the accompanying worksheet (Appendix 2) to enhance learning and retention. They were encouraged to utilize the completed worksheet in clinic to assist with patient management. This e-learning curriculum resulted in 5 videos created by the first author (R.S.), employing PowerPoint with audio recording (Microsoft Corp, Redmond, Wash), and was accessible from UCLA's (University of California, Los Angeles) e-learning portal. The average video length was approximately 13 minutes, based on evidence-based recommendations.

      Hazlett C. Optimal video length for student engagement. 2013. Available at: http://blog.edx.org/optimal-video-length-student-engagement. Accessed January 5, 2020

      The videos are available at https://dgsom.hosted.panopto.com/Panopto/Pages/Sessions/List.aspx#folderID=%22760dafb6-aa2d-4ad2-9d32-a87a018ad3cc%22&sortColumn=1&sortAscending=false.
      Steps 5 & 6: Implementation and Evaluation

      Study Design

      A pretest-post-test study design was employed to detect effectiveness of this e-learning orientation curriculum between 2 groups of residents: intervention and control.

      Settings and Participants

      The study was conducted at 2 tertiary care children's hospital systems (UCLA Mattel Children's Hospital and Children's Hospital of Orange County at UC Irvine). All UCLA or UC Irvine pediatric residents assigned to a pediatric endocrinology outpatient rotation during the study period were invited to participate. To ensure adequate exposure, attendance was monitored by rotation directors, and residents who did not attend at least 10 half-day clinics during their 2-week rotation were excluded from the study. This study was certified exempt by the UCLA and UC Irvine Institutional Review Boards.

      Outcomes Measurement

      We administered identical instruments prior to, and immediately following residents’ 2-week rotation in pediatric endocrinology, assessing medical knowledge and self-rated confidence on disease management. The knowledge assessment consisted of 29 multiple choice questions (5–7 questions per topic, Appendix 3). Each question's content corresponded to a specific learning objective of the curriculum and was developed following principles outlined in the National Board of Medical Examiners test-question writing guide.
      • Paniagua MA
      • Swygert KA
      Constructing Written Test Questions for the Basic and Clinical Sciences.
      The order of questions and answer choices was randomized on the post-test to decrease recall bias. Residents’ self-rated confidence on all 5 topics was elicited on a 5-point Likert scale, with 1 or 2 questions per topic, depending on the depth of the topic (Appendix 3). Both knowledge and confidence assessments were pilot tested by residents and refined to establish clarity of questions, ease of administration, etc.
      Three free response questions were included to gain insight into residents’ perspectives of the e-learning curriculum on helpfulness in acquiring useful clinical knowledge, desirability of a similar curriculum in other subspecialty rotations, and areas of improvement/general comments (Table 4). Self-rated confidence and free response questions are outcome measures on Level 1 (Reaction) of the Kirkpatrick Model of Evaluation, whereas the knowledge assessment is on Level 2 (Learning). Prior to launch, all content and assessments were reviewed by 2 pediatric endocrinology faculty, 1 general pediatric faculty, and 2 senior pediatric endocrinology fellows.

      Study Protocol

      Residents were recruited via e-mail, with those recruited between July 2017 and February 2018 assigned to the control group (N = 18), and those recruited between March 2018 and March 2019 assigned to the intervention group (N = 17). All residents who agreed to participate were administered the pretest on the morning of the first day of the rotation. Residents in the control group participated in the standard orientation, which consisted of an e-mail, containing the syllabus. The syllabus reviewed rotation personnel, contact information, resident responsibilities, and the schedule. Residents in the intervention group were given a 3-hour time block immediately after their pretest to complete the 5 e-learning videos, as well as a video that reviewed the same syllabus. The videos were completed on a computer with headphones and were proctored by administrative assistants. Residents were instructed to complete the accompanying 15-page worksheet on paper while watching the videos. A fully completed worksheet was used to verify completion of the video curriculum. After the 3-hour time block, residents continued with the rotation, similar to the control group. All residents were expected to perform and document a history and physical exam for each patient seen and staff with a faculty member. Access to e-learning videos was available to the intervention group throughout the rotation and was not monitored. The control group did not have access to the videos, which were in development during that time. During the rotation, no other endocrine lectures were provided to residents in either group. Teaching by faculty was based on patient encounters. Pretest and results were not released to either group. The post-test and confidence assessments were administered on the last afternoon of the residents’ rotation to both control and intervention groups. Both pretest and post-tests were closed-book, proctored, timed (1 hour), and completed on paper.

      Data Analysis

      Demographic data were first analyzed using descriptive statistics. Fisher exact test was utilized to determine differences in demographic variables between the control and intervention groups. Confidence ratings, one for each topic, were computed and analyzed after averaging multiple-associated questions. We employed the Wilcoxon signed rank test to compare the percent score change in medical knowledge and confidence ratings from pretest to post-test within each group, and Mann-Whitney U test to detect between-group differences in these variables. All analyses were conducted using SPSS version 24 (IBM Corp, Armonk, NY). Statistical significance was established at P < .05.
      We used qualitative, thematic analysis to analyze written comments generated by the intervention group on effectiveness of the e-learning videos. Two of the study authors (R.S. and A.R.) individually reviewed responses to free response questions, generating an inductive list of codes to create categories of responses. These were refined and combined into a mutually accepted coding structure.

      Results

      All 41 eligible residents, recruited from the pediatric endocrinology outpatient rotation from the 2 selected institutions volunteered to participate (Table 1). Fisher exact test showed equal representation of demographic variables across control and intervention groups. Residents who completed all components of the study were included in the analysis, resulting in 18 of 19 (94.7%) in control group, and 17 of 22 (77.2%) in intervention group. Failure to complete post-test due to various factors, primarily scheduling conflicts, was the reason for resident noncompletion in the 6 subjects.
      Table 1Demographics of Participating Residents
      DemographicsControl (N = 18)Intervention (N = 17)Fisher Exact Test
      Institution
       UCLA11120.73
       CHOC75
      Year
       PGY 112
       PGY 2690.40
       PGY 3116
      Prior pediatric endocrinology rotations (total time)
       None1112
       1–2 weeks540.99
       3–4 weeks21
      Interest in pediatric endocrinology as a career?
       Yes211
       No1616
      UCLA indicates University of California, Los Angeles; CHOC, Children's Hospital of Orange County; and PGY, post-graduate year.
      Overall pretest scores (% correct) on medical knowledge were statistically equivalent for control and intervention groups (40.4 ± 6.6 vs 42.1 ± 10.3, P = .73), and increased for both groups on the post-test (46.2 ± 8.8, P = .01 vs 66.8 ± 13.6, P < .0001; Table 2). However, increase in scores from pretest to post-test were significantly higher for intervention group compared to control group (+24.7 ± 12.1 vs +5.8 ± 7.9, P < .0001). Two of 5 topics showed a statistically significant increase in scores from pretest to post-test for intervention group compared to control group: congenital hypothyroidism (28.3 ± 15.9 vs −6.7 ± 19.4, P < .0001), and short stature (32.4 ± 22.4 vs 8.3 ± 16.4, P = .002).
      Table 2Change in Medical Knowledge From Pretest to Post-Test
      Control Group (N = 18)Intervention Group (N =17)P Value
      Mann-Whitney U test applied for comparisons of pre-post change in scores between the control and intervention groups.
      Pre Mean (SD) [Range]Post Mean (SD) `[Range]Change

      Mean (SD) P Value
      Wilcoxon signed rank test employed for comparisons within the control and intervention groups, respectively.
      Pre Mean (SD) [Range]Post Mean (SD) [Range]Change Mean (SD) P Value
      Wilcoxon signed rank test employed for comparisons within the control and intervention groups, respectively.
      Total score, % correct
      Total score and score on subtopics reflect mean percent correct.
      40.4 (6.6)

      [28.9–52.8]
      46.2 (8.8)

      [27.4–60.6]
      5.8 (7.9)

      P = .01
      42.1 (10.3)

      [26.8–67.8]
      66.8 (13.6)

      [40.8–88.6]
      24.7 (12.1)

      P < .0001
      <.0001
      Congenital hypothyroidism57.8 (19.3)

      [20.0–80.0.]
      51.1 (17.1)

      [20.0–80.0]
      −6.7 (19.4)

      P = .18
      49.4 (16.0)

      [20.0–80.0]
      77.7 (12.0)

      [60.0–100.0]
      28.3 (15.9)

      P < .0001
      <.0001
      Precocious puberty11.1 (11.4)

      [0.0–33.3]
      29.6 (22.6)

      [0.0–66.7]
      18.5 (20.5)

      P < .01
      18.6 (17.6)

      [0.0–66.7]
      52.0 (33.3)

      [0.0–100.0]
      33.4 (31.2)

      P < .01
      .16
      Type I diabetes54.4 (22.6)

      [20.0–100.0]
      57.8 (16.7)

      [20.0–80.0]
      3.4 (17.2)

      P = .41
      54.1 (22.1)

      [20.0–100.0]
      69.4 (21.4)

      [20.0–100.0]
      15.3 (30.4)

      P = .01
      .06
      Short stature46.3 (17.7)

      [16.7–83.3]
      54.6 (19.6)

      [0.0–83.3]
      8.3 (16.4)

      P = .05
      41.2 (18.7)

      [16.7–66.7]
      73.5 (18.7)

      [50.0–100.0]
      32.4 (22.4)

      P < .01
      .002
      Obesity/PCOS32.5 (22.4)

      [0.0–71.4]
      38.1 (15.5)

      [14.3–57.1]
      5.6 (17.1)

      P = .15
      47.1 (20.7)

      [14.3–85.7]
      61.3 (17.3)

      [28.6–85.7]
      14.2 (25.3)

      P = .04
      .23
      SD indicates standard deviation; PCOS, polycystic ovary syndrome.
      low asterisk Total score and score on subtopics reflect mean percent correct.
      Wilcoxon signed rank test employed for comparisons within the control and intervention groups, respectively.
      Mann-Whitney U test applied for comparisons of pre-post change in scores between the control and intervention groups.
      Confidence ratings increased from pre- to postmeasures for all 5 topics in both groups, with average increases ranging between 0.4 to 1.1 for control group and 1.1 to 1.9 for intervention group (Table 3). These increases in confidence ratings were statistically different between control and intervention groups for precocious puberty (0.9 vs 1.4, P = .04), short stature (1.0 vs 1.9, P = .001), and Type 1 diabetes (0.4 vs 1.6, P < .0001).
      Table 3Change in Confidence Ratings From Prerotation to Postrotation
      Confidence ratings are means of original responses, measured on a 5-point Likert scale (1, not confident to 5, extremely confident) to the statement: “Without attending supervision, how confident do you feel managing a patient with…”).
      Control Group (N = 18)Intervention Group (N = 17)P Value
      Mann-Whitney U test applied for comparisons of pre-post change in confidence ratings between the control and intervention groups.
      Pre Mean (SD) [Range]Post Mean (SD) [Range]Change Mean (SD) P Value
      Wilcoxon signed rank test employed for comparisons within the control and intervention groups, respectively.
      Pre Mean (SD) [Range]Post Mean (SD) [Range]Change Mean (SD) P Value
      Wilcoxon signed rank test employed for comparisons within the control and intervention groups, respectively.
      Congenital hypothyroidism1.2 (0.4)

      [1.0–2.0]
      2.3 (0.8)

      [1.0–3.0]
      1.1 (1.0)

      P < .01
      1.4 (0.6)

      [1.0–3.0]
      2.7 (0.7)

      [2.0–4.0]
      1.3 (0.7)

      P < .0001
      .57
      Precocious puberty1.3 (0.4)

      [1.0–2.0]
      2.2 (0.6)

      [1.0–3.0]
      0.9 (0.6)

      P < .0001
      1.3 (0.5)

      [1.0–2.5]
      2.7 (0.6)

      [1.5–4.0]
      1.4 (0.5)

      P < .0001
      .04
      Type I diabetes2.2 (0.7)

      [1.0–4.0]
      2.6 (0.7)

      [2.0–4.0]
      0.4 (0.9)

      P = .07
      1.5 (0.7)

      [1.0–3.0]
      3.1 (0.7)

      [2.0–4.0]
      1.6 (0.7)

      P < .0001
      <.0001
      Short stature1.5 (0.5)

      [1.0–3.0]
      2.5 (0.7)

      [1.0–4.0]
      1.0 (0.9)

      P < .01
      1.2 (0.5)

      [1.0–2.5]
      3.1 (0.6)

      [2.0–4.0]
      1.9 (0.5)

      P < .0001
      .001
      Obesity/PCOS2.2 (0.7)

      [1.0–3.5]
      2.9 (0.6)

      [2.0–4.0]
      0.7 (0.7)

      P < .01
      2.0 (0.7)

      [1.0–3.0]
      3.1 (0.7)

      [1.5–4.0]
      1.1 (0.8)

      P < .01
      .07
      SD indicates standard deviation; PCOS, polycystic ovary syndrome.
      low asterisk Confidence ratings are means of original responses, measured on a 5-point Likert scale (1, not confident to 5, extremely confident) to the statement: “Without attending supervision, how confident do you feel managing a patient with…”).
      Wilcoxon signed rank test employed for comparisons within the control and intervention groups, respectively.
      Mann-Whitney U test applied for comparisons of pre-post change in confidence ratings between the control and intervention groups.
      Two themes emerged from residents’ responses to free response questions, which addressed impact of e-learning videos: 1) increase in foundational clinical knowledge and 2) improvement in efficiency of learning and patient care (Table 4).
      Table 4Themes Illustrating Residents’ Perspectives of Clinically Focused Rotation Orientations Delivered Through E-Learning
      ThemeRepresentative Quotations From Residents
      Responses were based on the following questions: Question 1: “Were the videos helpful in acquiring knowledge needed to care for pediatric endocrinology patients appropriately? Can you explain?” Question 2: “Would it be desirable to have similarly structured videos in other rotations in residency? Can you explain?” Question 3: “Any other comments or areas for improvement?”
      1. Increases Foundational Clinical Knowledge (N = 11)“The videos gave me a very important baseline so I started out ahead instead of trying to keep up.”

      “Awesome high yield pearls to help me establish a strong foundation of understanding my patients in clinic.”

      “Videos provided very helpful frameworks to know the goals for each visit and to improve baseline knowledge regarding initial evaluations, work up, etc. for various endo topics.”
      2. Improves efficiency (N = 37)
      2a. Improves efficiency of learning (N = 17)“They allow you to spend more time in clinic learning and less time figuring out which questions to ask and what labs to order”

      “It is often difficult for residents to know what is relevant to ask or follow in pediatric subspecialties because we typically have limited exposure to the field prior to the rotation. Similar videos would be helpful to bridge that gap in knowledge and allow for more learning during clinical encounters.”

      “Structured curriculum is desirable because it allows the learner to have focused teaching that is efficient and board-specific, instead of having to read through a book or article to gain the knowledge.”
      2b. Improves efficiency of patient care (N = 14)“It was really helpful to be shown examples of how to manage, for example, diabetes. I was able to apply it in practice and make it stick in my brain.”

      “It taught the most important information a physician needs to gather in history and physical exam so that I was more prepared as I walked in the room.”

      “To the point teaching that allowed me to get a more focused and appropriate history and provide anticipatory guidance.”
      2c: Consistent, comprehensive education (N = 6)“They would be a very helpful starting point for other subspecialties so as to make sure that all the “must” and “should” know conditions for that field are covered.”

      “I would definitely like other outpatient rotations to have similar system for teaching because clinic gest busy and there isn't always time for teaching”

      “Oftentimes residents learn by doing in clinic, however a more structured teaching course at the beginning of any subspecialty rotation would always be helpful because patient experiences/attending teaching is always variable.”
      N indicates number of coded responses.
      low asterisk Responses were based on the following questions: Question 1: “Were the videos helpful in acquiring knowledge needed to care for pediatric endocrinology patients appropriately? Can you explain?” Question 2: “Would it be desirable to have similarly structured videos in other rotations in residency? Can you explain?” Question 3: “Any other comments or areas for improvement?”

      Discussion

      To our knowledge, this is the first study to demonstrate efficacy of a clinically focused rotation orientation to significantly increase medical knowledge in residents, from pretest to post-test, compared to a control group. The minimal knowledge increase in the control group was expected based on our needs assessment and may reflect learning inefficiencies in rotations without such an orientation, as described in the aforementioned sections. Our qualitative assessment of residents (Table 4) suggests that learning inefficiencies may diminish through implementation of such an orientation while improving the learning experience and educational yield of the rotation.
      These qualitative findings are in alignment with prior studies of clinically focused rotation orientations.
      • Surcouf JW
      • Mumphrey CG
      • Barkemeyer BM
      • et al.
      Neonatal intensive care unit boot camp: a preparatory curriculum for pediatric residents.
      ,
      • Harel Z
      • Riggs S
      • Vaz R
      • et al.
      A session of multiple choice questions in the orientation process to the adolescent medicine rotation.
      However, we describe a more feasible delivery method (e-learning) to address this problem as prior studies required substantial faculty participation to deliver content to each rotating resident throughout the academic year. Furthermore, at the time of this publication, an ongoing pandemic and mandated social distancing reminds us of the need to harness available technology as an alternative to classical, in-person or bedside teaching that may be interrupted by various factors.
      In the past, technical challenges and high cost were barriers to creating e-learning curricula. However, newer, freely available online platforms ease the cost and technical skills needed to create e-learning curricula, while adding features to increase engagement with peers and supervisors. We developed our videos by simply using Microsoft PowerPoint with its audio recording feature, which can be created and administered without an Internet connection. We found that the initial time investment to create our e-learning curriculum (approximately 5–7 hours per video) far exceeded the time required to update it with evidence-based material on an annual basis.
      Limitations of this study include its small sample size, immediate testing at the end of the rotation, and evaluation that is limited to the lower tiers of the Kirkpatrick Model (learner reactions and knowledge acquisition). Future studies could be larger and formally assess reactions of key rotation stakeholders, longer term knowledge retention, and higher tiers of the Kirkpatrick Model, such as learner behavior and patient care. Finally, conducting the study in a variety of subspecialties, in both the inpatient and outpatient setting, would be important.
      Since the conclusion of this study, all residents rotating through endocrinology at both institutions are required to complete this e-learning orientation and it is being used to on-board new fellows. We have also been approached by other subspecialties to help them create a similar orientation for their residents.

      Conclusions

      We described the development of a simple, effective, and well-received e-learning video curriculum to orient residents to the foundational clinical knowledge needed for an outpatient pediatric subspecialty rotation, without persistent reliance of faculty time and availability to deliver this content to each new rotating resident. Further studies should be pursued to provide more knowledge of the applicability and efficacy of this method in various training settings.

      Appendix. SUPPLEMENTARY DATA

      References

        • Young JQ
        • Ranji SR
        • Wachter RM
        • et al.
        “July effect”: impact of the academic year-end changeover on patient outcomes: a systematic review.
        Ann Intern Med. 2011; 155: 309-315
        • Haller G
        • Myles PS
        • Taffé P
        • et al.
        Rate of undesirable events at beginning of academic year: retrospective cohort study.
        BMJ. 2009; 339: b3974
        • Blackmore C
        • Austin J
        • Lopushinsky SR
        • et al.
        Effects of postgraduate medical education “boot camps” on clinical skills, knowledge, and confidence: a meta-analysis.
        J Grad Med Educ. 2014; 6: 643-652
        • Nishisaki A
        • Hales R
        • Biagas K
        • et al.
        A multi-institutional high-fidelity simulation ‘‘boot camp’’ orientation and training program for first year pediatric critical care fellows.
        Ped Crit Care Med. 2009; 10: 157-162
        • Sobolewski B
        • Kerrey BT
        • Geis GL
        • et al.
        The April effect: a multimedia orientation approach to improve rotation transitions during pediatric residency.
        Acad Pediatr. 2016; 16: 220-223
        • Surcouf JW
        • Mumphrey CG
        • Barkemeyer BM
        • et al.
        Neonatal intensive care unit boot camp: a preparatory curriculum for pediatric residents.
        MedEdPORTAL. 2018; 14: 10720
        • Merrill A
        • Djuricich AM
        • Drazen JM
        • et al.
        NEJM resident 360.
        N Engl J Med. 2016; 374: 2595-2596
      1. ACGME. Common program requirements (residency). 2019. Available at: https://acgme.org/Portals/0/PFAssets/ProgramRequirements/CPRResidency2019.pdf. Accessed February 6, 2020.

        • Harel Z
        • Riggs S
        • Vaz R
        • et al.
        A session of multiple choice questions in the orientation process to the adolescent medicine rotation.
        Teach Learn Med. 2004; 16: 365-367
        • Grover M
        • Puczynski S
        Residency orientation: what we present and its effect on our residents.
        Fam Med. 1999; 31: 697-702
        • Antonoff MB
        • Swanson JA
        • Acton RD
        • et al.
        Improving surgery intern confidence through the implementation of expanded orientation sessions.
        Surgery. 2010; 148: 181-186
        • Fein JA
        • Lavelle J
        • Giardino AP
        Teaching emergency medicine to pediatric residents: a national survey and proposed model.
        Pediatr Emerg Care. 1995; 11: 208-211
        • Young JQ
        • Van Merrienboer J
        • Durning S
        • et al.
        Cognitive load theory: implications for medical education: AMEE Guide No. 86.
        Med Teach. 2014; 36: 371-384
        • Kern DE
        • Thomas PA
        • Howard DM
        • et al.
        Curriculum Development for Medical Education: A Six-Step Approach.
        Johns Hopkins University Press, Baltimore, Md1998
        • Bloom BS
        Taxonomy of Educational Objectives: The Classification of Educational Goals.
        Longmans, Green, New York, NY1956
      2. Hazlett C. Optimal video length for student engagement. 2013. Available at: http://blog.edx.org/optimal-video-length-student-engagement. Accessed January 5, 2020

        • Paniagua MA
        • Swygert KA
        Constructing Written Test Questions for the Basic and Clinical Sciences.
        National Board of Medical Examiners, Philadelphia Pa2016