Advertisement

Exploring the Value of Just-in-Time Teaching as a Supplemental Tool to Traditional Resident Education on a Busy Inpatient Pediatrics Rotation

  • Ross Mangum
    Affiliations
    Department of Pediatrics, Ohio State University College of Medicine/Nationwide Children's Hospital, Columbus, Ohio
    Search for articles by this author
  • John Lazar
    Affiliations
    Department of Pediatrics, Ohio State University College of Medicine/Nationwide Children's Hospital, Columbus, Ohio
    Search for articles by this author
  • Melissa J. Rose
    Affiliations
    Department of Pediatrics, Ohio State University College of Medicine/Nationwide Children's Hospital, Columbus, Ohio

    Division of Hematology, Oncology & BMT, Nationwide Children's Hospital, Columbus, Ohio
    Search for articles by this author
  • John D. Mahan
    Affiliations
    Department of Pediatrics, Ohio State University College of Medicine/Nationwide Children's Hospital, Columbus, Ohio

    Division of Nephrology, Nationwide Children's Hospital, Columbus, Ohio
    Search for articles by this author
  • Suzanne Reed
    Correspondence
    Address correspondence to Suzanne Reed, MD, Nationwide Children's Hospital, 700 Children's Drive – Room ED2A, Columbus, OH 43205.
    Affiliations
    Department of Pediatrics, Ohio State University College of Medicine/Nationwide Children's Hospital, Columbus, Ohio

    Division of Hematology, Oncology & BMT, Nationwide Children's Hospital, Columbus, Ohio
    Search for articles by this author
Published:April 26, 2017DOI:https://doi.org/10.1016/j.acap.2017.04.021
      Because of changes in modern medicine
      • Mahan J.D.
      • Clinchot D.
      Why medical education is being (inexorably) re-imagined and re-designed.
      and evolving requirements mandated by the Accreditation Council for Graduate Medical Education,
      • Nasca T.J.
      • Day S.H.
      • Amis Jr., E.S.
      The new recommendations on duty hours from the ACGME task force.
      residency programs have sought to develop innovative curricular programs and teaching modalities to ensure high-quality resident education in an ever-changing environment.
      • Reed S.
      • Shell R.
      • Kassis K.
      • et al.
      Applying adult learning practices in medical education.
      More stringent work hour restrictions, busy inpatient censuses, and increased demands of the electronic health record are a few of the obstacles limiting opportunities for direct resident teaching, lectures, and/or seminars.
      • Mahan J.D.
      • Clinchot D.
      Why medical education is being (inexorably) re-imagined and re-designed.
      These factors increase demands on attending and resident physicians, leading to an overall less formalized educational experience—less face to face interaction with attending physicians, decreased bedside teaching, and limited direct observation of patient care.
      • Golbus J.R.
      • Manly D.A.
      • Wonneberger K.A.
      • et al.
      Implementation of a novel, resident-led, nocturnal curriculum.
      Just-in-Time Teaching (JiTT) is an andragogic strategy used primarily in classroom-based settings to promote improved opportunities for active learning. JiTT uses Web-based and other teaching modalities to optimize and diversify out-of-class preparation to establish a strong foundation upon which meaningful in-class experiences can be structured.
      • Novak G.M.
      • Patterson E.T.
      • Gavrin A.D.
      • et al.
      Just-in-Time Teaching: Blending Active Learning With Web Technology.
      This teaching/learning strategy has had limited implementation in residency program curricula with 1 report that detailed improvements in learner participation, time spent learning, and overall retention.
      • Schuller M.C.
      • DaRosa D.A.
      • Crandall M.L.
      Using just-in-time teaching and peer instruction in a residency program's core curriculum: enhancing satisfaction, engagement, and retention.
      The limited literature investigating the effect of JiTT in clinical settings focuses on improving technical skill in infrequently performed procedures (lumbar punctures,
      • Kessler D.
      • Pusic M.
      • Chang T.P.
      • et al.
      Impact of just-in-time and just-in-place simulation on intern success with infant lumbar puncture.
      tracheal intubation,
      • Nishisaki A.
      • Donoghue A.J.
      • Colborn S.
      • et al.
      Effect of just-in-time simulation training on tracheal intubation procedure safety in the pediatric intensive care unit.
      volar splinting,
      • Cheng Y.
      • Liu D.R.
      • Wang V.J.
      Teaching splinting techniques using a just-in-time training instructional video.
      mock code scenarios
      • Sam J.
      • Pierse M.
      • Al-Qahtani A.
      • et al.
      Implementation and evaluation of a simulation curriculum for paediatric residency programs including just-in-time in situ mock codes.
      ) and has shown mixed results. Using Kern's 6-step approach to curricular development
      • Kern D.E.
      • Thomas P.A.
      • Hughes M.T.
      Curriculum Development for Medical Education: A Six-Step Approach.
      and a logic model framework, we sought to create and implement JiTT in a busy inpatient pediatrics clinical setting. A logic model is a graphical/visual representation of a program, illustrating the relationship among available resources, activities necessary for the process, and outcomes desired.
      W.K. Kellogg Foundation
      Using Logic Models to Bring Together Planning, Evaluation, and Action: Logic Model Development Guide.
      We identified an educational gap in the multifactorial reduction in direct face-to-face teaching and learning opportunities between faculty and resident learners, and the need for alternative learning opportunities. Our logic model is illustrated in the Figure. We aimed to investigate whether a JiTT educational approach could be effectively used as a supplemental tool to help residents better prepare for, and formally learn within, a busy clinical rotation and whether this approach would be perceived as a worthwhile educational resource by residents.
      Figure thumbnail gr1
      FigureLogic model for just-in-time teaching (JiTT) supplemental curriculum.

      Methods

      Setting and Participants

      Our educational intervention was implemented in a large (464-bed) pediatric tertiary care hospital. The pediatric residency program is comprised of 147 residents—107 Pediatrics and 40 Internal Medicine-Pediatrics residents. Because of its high census and complex and significantly ill patients, our inpatient pediatric hematology/oncology ward is generally recognized as one of the most demanding resident services, with 1644 admissions in 2014, and an average daily census of 24 patients. The significant patient care demands on attending and resident physicians often limit opportunities for formal, face to face resident education, making this an ideal service to investigate the utility of JiTT educational modules. Because our study was an evaluation of a supplemental education tool, it was deemed exempt from Institutional Review Board application.

      Intervention

      By identifying an educational problem—a multifactorial lack of formal instruction in our busy clinical settings—we completed the first step in Kern's 6-step approach to curriculum development (step 1). In our targeted needs assessment (step 2), we obtained feedback from residents with previous experience on our inpatient pediatric hematology/oncology service on educational gaps and common clinical scenarios lacking formal instruction from attending physicians. Using this feedback, 4 e-learning modules were created using Articulate Presenter ‘09 (version 6.3.1103.112 Pro; Articulate Global Inc, New York, NY), a multidimensional e-learning program, allowing users to create modules incorporating audio, visual, and interactive learning using 11 different learning templates.
      This variety of interactive templates allows many options for conceptual representation of different learning topics. For example, the stepwise process template was used to teach steps in assessing and treating evolving septic shock, with a labeled graphic template to visualize and describe the set-up of a push-pull system for fluid resuscitation. Our modules focused on high-yield, clinically relevant topics likely to be encountered on the inpatient pediatric hematology/oncology service, including: 1) septic shock, 2) acute chest syndrome, 3) anemia, and 4) management of newly diagnosed acute leukemia. Modules were limited to 15 minutes in length. The goals of our curriculum (step 3) were to enhance residents' knowledge and proficiency in these commonly encountered clinical scenarios and to prepare residents for initial management of these patients. To address this educational goal, we created a supplemental learning resource that could be used by residents in a controlled, consequence-free environment. We used the JiTT strategy (step 4) by creating concise, easily accessible, Web-based modules highlighting various high-yield topics in hematology/oncology to focus resident clinical time on experiential learning instead of depending on formal lectures or small-group teaching sessions. For implementation of this supplemental curriculum (step 5), interns and supervisory residents were notified of and given access to the modules 1 to 2 weeks before the start of their 4-week service block. They were informed that these modules served strictly as a supplementary learning tool and their participation would be voluntary. There was neither reward for completion nor consequence for not completing the modules. The online modules were available in an online learning management system.

      Analysis

      After completion of the 4-week inpatient pediatric hematology/oncology rotation, all residents who completed at least 1 module were sent electronic surveys evaluating their perceptions and experiences (step 6). Survey questions were a combination of open- and closed-ended questions, with all questions having space for additional narrative feedback/expansion on an answer. Questions were directed toward our objectives of determining utility and feasibility, including ease and accessibility of the modules, preference for the Articulate (Articulate Global Inc) format, usefulness of the information, feasibility as a supplemental learning tool, barriers to use, and potential value of this approach for other inpatient services or medical specialties. Survey completion was also voluntary. Because our primary aim was to evaluate the usefulness of the modules after actually experiencing them, we did not solicit feedback from those who had not completed any modules.

      Results

      Over the 36 weeks of our study period (9 total service blocks), 56 residents completed the 4-week hematology/oncology service block. Of these residents, 36 (64%) were first-year residents (postgraduate year [PGY]-1) and 20 (36%) were supervisory residents (PGY-2/3). Thirty-three residents (59%) completed at least 1 JiTT training module (25 PGY-1, 8 PGY-2/3), and of these, 29 completed more than 1 module. Sixteen residents completed all modules. Twenty-one of the 56 residents who had completed at least 1 module responded to our survey (15 PGY-1, 6 PGY-2/3). Most of the responding residents (17 of 21; 81%) found the modules easy-to-use, whereas 4 of 21 (19%) had technical difficulties primarily related to incompatibility with home Apple computers. Additionally, most (18 of 21; 86%) favored the Articulate (Articulate Global Inc) e-learning format, whereas 3 residents identified other alternative learning methods as preferable to e-learning modules.
      Specific comments regarding the format included, “Everyone has a shorter and shorter attention span, so actually engaging with the presentation instead of being lectured to keep my attention,” and, “I liked the format with slide and narration. It helped to keep my attention and focus on the most important things.” A few residents also commented that although they enjoyed the e-module format, they did not want face to face learning to be completely replaced by electronic learning. Twenty of the 21 responding residents found the modules useful and worthy of recommendation to others, with feedback such as, “These presentations are high-yield, in that we saw these concerns in patients every day.” Further positive perception was shown by 17 of 21 (81%) residents finding modules feasible, with comments like, “This is a great resource to have available since we often don't get much teaching during the rotation due to high patient load.” Although these modules were available for home use on personal computers away from the hectic hospital environment, 4 residents had concerns about time for completion during working hours. Finally, almost all (20 of 21; 95%) of the residents found this learning strategy worthy of expansion to other clinical rotations.

      Discussion

      In our pilot study we found that a JiTT-based supplemental curriculum using Web-based, interactive Articulate (Articulate Global Inc) e-learning modules was a feasible and useful learning resource for residents on a busy inpatient clinical rotation. As numerous factors in current graduate medical training have led to a less formalized educational experience by trainees,
      • Mahan J.D.
      • Clinchot D.
      Why medical education is being (inexorably) re-imagined and re-designed.
      this learning resource is a potential way to meet these evolving needs of trainees in the current clinical educational environment. The combined effect of more stringent work limitations and extensive patient care demands often results in decreased time for direct teaching opportunities between attending and resident physicians. As such, exploration of more innovative approaches is necessary to ensure high-quality resident training. In addition, our increasingly digital world provides opportunities for technology-based learning approaches to meet some of the needs of our trainees.
      Resident perception of JiTT as an effective supplemental learning resource is essential to the success of our educational innovation because this curriculum was purposefully not made a required element of this rotation. We believed that true perception of the usefulness and educational benefit of the curriculum would be best assessed without making it “required.” Because of the unpredictable nature of a resident work schedule, implementation of a routine and regimented lecture and/or personal study plan can be extremely difficult. Therefore, on busy inpatient services that might lack regular protected teaching infrastructure, residents are at risk for rotating through an entire service month with an unfortunate deficiency of formal resident education. A supplemental JiTT e-learning curriculum, with brief, high-yield clinical topics relevant to a rotation, can ensure residents do not miss out on education topics defined as being particularly important on highly specialized services in which residents might lack significant experience.
      Although we did not measure resident preparedness for clinical management or clinical performance surrounding JiTT topics, residents' significantly favorable perception of this supplemental curriculum is encouraging. Resident narrative feedback consistently commented on the user-friendly, interactive nature of the modules and how their short duration was conducive with their frequently demanding schedules. This feedback underscores key components of adult learning theory, including the notion that adult learners value learning that integrates with the demands of their everyday life and that they are more interested in immediate, problem-centered approaches than in subject-centered ones.
      • Kaufman D.M.
      Applying educational theory in practice.
      This feedback is also reflective of educational preferences of the millennial generation, which include prioritizing personalized and interactive learning, self-directed teaching methods, and incorporation of technology.
      • Desy J.R.
      • Reed D.A.
      • Wolanskyj A.P.
      Milestones and millennials: a perfect pairing-competency-based medical education and the learning preferences of generation Y.
      • Twenge J.M.
      Generational changes and their impact in the classroom: teaching Generation Me.
      • Roberts D.H.
      • Newman L.R.
      • Schwartzstein R.M.
      Twelve tips for facilitating Millennials' learning.
      Millennial learners also desire direct, focused, and high-yield learning experiences,
      • Roberts D.H.
      • Newman L.R.
      • Schwartzstein R.M.
      Twelve tips for facilitating Millennials' learning.
      a specific objective of our JiTT pilot curriculum. We believe that the characteristics of this JiTT educational experience match with millennial learning preferences, and this contributed significantly to the overall high satisfaction with the modules. In meeting the evolving needs of millennial learners, JiTT might have a useful role in other busy inpatient rotations, and also within outpatient clinical rotations as well.
      From a programmatic practical implementation standpoint, this interactivity and succinctness is key to the JiTT curriculum. It is important to note that residents also expressed a desire for continued face to face teaching and that JiTT should not replace direct educational interaction with attending physicians, consistent with the tenet of adult learning theory that adult learners desire diverse educational methods.
      • Knowles M.
      The Adult Learner: A Neglected Species.
      Therefore, residents were aware of the value of seeking out more independent learning opportunities. Programs can support this by offering a variety of these opportunities, including JiTT e-learning modules.
      In our study, JiTT modules were more frequently used by first-year residents compared with senior residents, which might reflect the lack of experience and knowledge in this group of trainees as a motivator for voluntarily completing these supplementary e-modules. Supervisory residents have more experience with pediatric hematology/oncology patients, and might feel more comfortable managing this difficult patient population. These factors might lead to less motivation to use the JiTT modules. These findings are supported by Knowles' assumption in adult learning theory that learners become increasingly self-directed and that motivation moves increasingly from external to internal as a learner develops.
      • Knowles M.
      The Adult Learner: A Neglected Species.
      • Hopstock L.A.
      Motivation and adult learning: a survey among hospital personnel attending a CPR course.
      This suggests first-year residents, inexperienced supervisory residents, and/or supervisory residents for whom it has been an extended period of time since a particular rotation might benefit most from use of these supplementary resources. This concept might explain that although the feedback on the modules was overwhelmingly positive, only 16 residents completed all 4 modules. This could be because certain residents might be familiar with some topics covered by the modules and not view these topics as learning needs.
      Although upfront faculty investment might be significant—particularly with regard to the time commitment to construct the modules—little is required after the development of the curriculum. For our curriculum, initial modules, depending on length and complexity, took up to 20 hours each to prepare. However, with increased familiarity with Articulate software (Articulate Global Inc), later modules took less time. Additionally, if key content in particular areas could be taught using this format, this might save faculty time in direct teaching and allow faculty to focus on higher-level content during the rotation. In programs with very busy clinical faculty, it is most reasonable for multiple people to participate in module creation. When modules are housed online or in a learning management system, tracking of participation and regular yearly review for needed updates and/or content changes is the only ongoing maintenance required.
      There are some limitations to our educational intervention and analysis. In our pilot program, only a small number of residents participated in the rotation and had the option to complete the modules during the study period. Only a subset of those residents completed our survey after module use. Feedback was not sought from those who did not complete modules, so we lack potentially valuable understanding and feedback from this nonparticipation group. This might suggest that, for the group not completing any modules, e-learning modules are not preferable learning modalities. An additional challenge was unpredictable technology that made it difficult for some residents to gain access to the JiTT modules. Finally, the effect of our modules on objective preparedness for high-quality patient care is unknown.
      Further research is needed to evaluate whether JiTT leads to improvement in objective preparedness for a clinical rotation. A carefully designed study, which incorporates faculty feedback and rotation evaluations, might be able to measure potential effect of JiTT on resident performance and/or overall rotation educational experience.
      • Iobst W.F.
      • Sherbino J.
      • Cate O.T.
      • et al.
      Competency-based medical education in postgraduate medical education.
      • Holmboe E.S.
      • Sherbino J.
      • Long D.M.
      • et al.
      International CBME Collaborators
      The role of assessment in competency-based medical education.

      Conclusion

      These pilot data support the feasibility and subjective utility of a supplemental JiTT curriculum in a busy inpatient clinical environment. In our program, they also support the exploration of expansion of this supplemental resource to include additional modules on clinically relevant topics within pediatric hematology/oncology and potentially other resident rotations or disciplines. Most of the responding residents perceived the JiTT modules as easy to use, helpful supplementary resources. Furthermore, the creation of these training modules required relatively few resources, thus making this endeavor reproducible across other medical specialties. On the basis of our results, there is utility in further studying the application of the JiTT strategy as a resource in graduate medical education.

      References

        • Mahan J.D.
        • Clinchot D.
        Why medical education is being (inexorably) re-imagined and re-designed.
        Curr Probl Pediatr Adolesc Health Care. 2014; 44: 137-140
        • Nasca T.J.
        • Day S.H.
        • Amis Jr., E.S.
        The new recommendations on duty hours from the ACGME task force.
        N Engl J Med. 2010; 363: e3
        • Reed S.
        • Shell R.
        • Kassis K.
        • et al.
        Applying adult learning practices in medical education.
        Curr Probl Pediatr Adolesc Health Care. 2014; 44: 170-181
        • Golbus J.R.
        • Manly D.A.
        • Wonneberger K.A.
        • et al.
        Implementation of a novel, resident-led, nocturnal curriculum.
        J Grad Med Educ. 2015; 7: 417-421
        • Novak G.M.
        • Patterson E.T.
        • Gavrin A.D.
        • et al.
        Just-in-Time Teaching: Blending Active Learning With Web Technology.
        Prentice-Hall, Upper Saddle River, NJ1999
        • Schuller M.C.
        • DaRosa D.A.
        • Crandall M.L.
        Using just-in-time teaching and peer instruction in a residency program's core curriculum: enhancing satisfaction, engagement, and retention.
        Acad Med. 2015; 90: 384-391
        • Kessler D.
        • Pusic M.
        • Chang T.P.
        • et al.
        Impact of just-in-time and just-in-place simulation on intern success with infant lumbar puncture.
        Pediatrics. 2015; 135: e1237-e1246
        • Nishisaki A.
        • Donoghue A.J.
        • Colborn S.
        • et al.
        Effect of just-in-time simulation training on tracheal intubation procedure safety in the pediatric intensive care unit.
        Anesthesiology. 2010; 113: 214-223
        • Cheng Y.
        • Liu D.R.
        • Wang V.J.
        Teaching splinting techniques using a just-in-time training instructional video.
        Pediatr Emerg Care. 2017; 33: 166-170
        • Sam J.
        • Pierse M.
        • Al-Qahtani A.
        • et al.
        Implementation and evaluation of a simulation curriculum for paediatric residency programs including just-in-time in situ mock codes.
        Paediatr Child Health. 2012; 17: e16-e20
        • Kern D.E.
        • Thomas P.A.
        • Hughes M.T.
        Curriculum Development for Medical Education: A Six-Step Approach.
        2nd ed. The John's Hopkins University Press, Baltimore, MD2009
        • W.K. Kellogg Foundation
        Using Logic Models to Bring Together Planning, Evaluation, and Action: Logic Model Development Guide.
        W.K. Kellogg Foundation, Battle Creek, Mich2004
        • Kaufman D.M.
        Applying educational theory in practice.
        BMJ. 2003; 326: 213-216
        • Desy J.R.
        • Reed D.A.
        • Wolanskyj A.P.
        Milestones and millennials: a perfect pairing-competency-based medical education and the learning preferences of generation Y.
        Mayo Clin Proc. 2017; 92: 243-250
        • Twenge J.M.
        Generational changes and their impact in the classroom: teaching Generation Me.
        Med Educ. 2009; 43: 398-405
        • Roberts D.H.
        • Newman L.R.
        • Schwartzstein R.M.
        Twelve tips for facilitating Millennials' learning.
        Med Teach. 2012; 34: 274-278
        • Knowles M.
        The Adult Learner: A Neglected Species.
        Gulf, Houston, Texas1988
        • Hopstock L.A.
        Motivation and adult learning: a survey among hospital personnel attending a CPR course.
        Resuscitation. 2008; 76: 425-430
        • Iobst W.F.
        • Sherbino J.
        • Cate O.T.
        • et al.
        Competency-based medical education in postgraduate medical education.
        Med Teach. 2010; 32: 651-656
        • Holmboe E.S.
        • Sherbino J.
        • Long D.M.
        • et al.
        • International CBME Collaborators
        The role of assessment in competency-based medical education.
        Med Teach. 2010; 32: 676-682