Advertisement

Medical Education Journal Club for the Millennial Resident: An Interactive, No-Prep Approach

      Abstract

      Objective

      The traditional journal club (JC) format of reviewing an article followed by group discussion may be misaligned with millennial learners’ needs and may not rely on best principles of adult learning. Our objective was to deliver an interactive JC allowing pediatric residents to critically engage with medical education research without previous preparation.

      Methods

      We conducted 4 one-hour “interactive, no-prep” medical education JCs for pediatric residents in a medium-sized program in 2018. Without previous reading, participants developed methods to answer the study question, compared that with actual methods, analyzed the results, and extrapolated the findings. We developed a simple, anonymous evaluation tool to determine perceived educational impact, analyzed using mixed methods.

      Results

      A total of 52 of 59 participants (88% response rate) indicated on a 7-point scale that the JC helped them think about how to analyze a paper (mean = 5.32), use a paper to inform further study questions (mean = 5.42), and understand medical education research (mean = 6.00). Four qualitative themes indicated that, although improvement was possible, it provided a strong interactive learning experience.

      Conclusions

      Our JC approach using active learning principles and requiring no advance preparation is proof of concept that faculty's objectives to teach critical literature evaluation and millennial needs for engagement can be simultaneously met.

      Keywords

      What's New
      We describe the creation and implementation of an innovative, interactive journal club model which requires no prior preparation, relies on adult learning principles, and is adapted to the needs of the millennial learner.
      Journal clubs (JCs) are a versatile graduate medical education tool to develop critical appraisal skills, encourage evidence-based practice, support research training, and disseminate new findings.
      • Deenadayalan Y
      • Grimmer-Somers K
      • Prior M
      • et al.
      How to run an effective journal club: a systematic review.
      • Moharari RS
      • Rahimi E
      • Najafi A
      • et al.
      Teaching critical appraisal and statistics in anesthesia journal club.
      • Topf JM
      • Sparks MA
      • Phelan PJ
      • et al.
      The evolution of the journal club: from Osler to Twitter.
      • Alguire PC
      A review of journal clubs in postgraduate medical education.
      • Dzara K
      • Jain G
      • Soltys SM
      The self-directed, structured summary as a teaching tool in a psychiatry journal club.
      • McLeod P
      • Steinert Y
      • Boudreau D
      • et al.
      Twelve tips for conducting a medical education journal club.
      A traditional JC consists of a group of engaged physicians discussing 1 or more research articles.
      • Topf JM
      • Sparks MA
      • Phelan PJ
      • et al.
      The evolution of the journal club: from Osler to Twitter.
      • McLeod P
      • Steinert Y
      • Boudreau D
      • et al.
      Twelve tips for conducting a medical education journal club.
      Participants are expected to pre-read the article, bring questions and comments, and analyze the methods and results with faculty and peers.
      • Topf JM
      • Sparks MA
      • Phelan PJ
      • et al.
      The evolution of the journal club: from Osler to Twitter.
      Anecdotally, faculty at our institution have noted that residents are increasingly not reading articles for JC and are either coming unprepared or not attending at all. This aligns with what is known about millennial, as well as Generation Z, learners; they appreciate engaging in active, team-based learning, desire convenience, have short attention spans, desire to use technology when learning, and prefer multitasking.
      • Hopkins L
      • Hampton BS
      • Abbott JF
      • et al.
      To the point: medical education, technology, and the millennial learner.
      • Shatto B
      • Erwin K
      Teaching Millennials and Generation Z: bridging the generational divide.
      Given these preferences, they may choose not to critically read articles before JCs.
      • Rosenthal J
      • Rosenthal KS
      Interactive journal club: teaching an old dog new tricks.
      • Crichlow R
      Trans-formative journal club experience as a basis for a longitudinal EBM curriculum.
      Recognizing the problem of waning trainee engagement in JC, there have been innovative methods to improve them, for example, focusing on peer mentorship,
      • MacMillan TE
      • Rawal S
      • Cram P
      • et al.
      A journal club for peer mentorship: helping to navigate the transition to independent practice.
      pairing trainees to present articles,

      Johns Hopkins Medicine. Osler Journal Club. 2019. Available at:https://www.hopkinsmedicine.org/gim/training/Osler_Journal_Club/index.html. Accessed April 22, 2019.

      having participants write multiple-choice questions to encourage greater engagement,
      • Franquet E
      • Parker J
      • Donohoe K
      Question-writing Journal-Club is an alternative learning method for the residents.
      using structured summaries,
      • Dzara K
      • Jain G
      • Soltys SM
      The self-directed, structured summary as a teaching tool in a psychiatry journal club.
      or framing JCs as debates.
      • Antonoff M
      • Nguyen T
      • Luc J
      • et al.
      A structured, debate-style cardiothoracic surgery journal club for trainee acquisition and application of seminal literature.
      There also have been previous attempts to eliminate the need for advanced preparation.
      • Rosenthal J
      • Rosenthal KS
      Interactive journal club: teaching an old dog new tricks.
      • Crichlow R
      Trans-formative journal club experience as a basis for a longitudinal EBM curriculum.
      In one approach, faculty facilitators provided participants with the study title, tables, and figures, and prompted participants to offer their own interpretation.
      • Rosenthal J
      • Rosenthal KS
      Interactive journal club: teaching an old dog new tricks.
      Another developed an “Abstract Attack” model, which was successful with 85% to 100% of participants not reading in advance.
      • Crichlow R
      Trans-formative journal club experience as a basis for a longitudinal EBM curriculum.
      Anecdotally, participants reported having fun, which encouraged participation.
      • Crichlow R
      Trans-formative journal club experience as a basis for a longitudinal EBM curriculum.
      Neither method was formally evaluated. Overall, very little literature addresses the challenge of unprepared attendees. Furthermore, limited literature addresses the use of JCs to expose learners to medical education research.
      • Simpson D
      • Flynn C
      • Wendelberger K
      An evidence-based education journal club.
      • Centeno AM
      • Blanco A
      • Arce M
      Journal club devoted to educational issues.
      We transformed the traditional JC format into an “interactive, no-prep” series for the MassGeneral Hospital for Children pediatric residents. Our aim was to deliver a time-efficient JC using active learning techniques to allow pediatric residents to critically engage with medical education research without previous preparation. The aim of our educational evaluation was to determine participants’ perceived learning impact of the JCs using a mixed methods analysis.

      Methods

      Setting and Participants

      We conducted 4 one-hour “interactive, no-prep” medical education JCs for pediatrics residents in a medium-sized program between February and September 2018. Sessions were held during the noon conference hour in a conference room with a whiteboard and projector. Medical students, pediatrics residents of all levels including chiefs, and a small number of faculty attended. Residents were expected to attend if clinical demands permitted.

      Implementation

      Learning objectives were to have participants develop a plan for answering the article's study question, compare that with the actual study methodology, analyze the results of the study, and extrapolate the study findings to their own experiences. Participants were expected to analyze medical education research methodology to determine strengths and weaknesses of the design and analysis, interpret the practical significance of results, and recognize further questions the study raised.
      Each JC explored a medical education research article chosen by the session leaders (K.D., A.F.V.). Purposefully selected articles fit 4 criteria aligning with adult learning principles
      • Knowles MS
      • Holton E
      • Swanson R
      The Adult Learner: The Definitive Classic in Adult Education and Human Resource Development.
      :
      • 1)
        relevant to pediatrics residents’ real-world training experience (fits with learners’ previous experience)
      • 2)
        focused on graduate medical education (aligns with learners’ self-concept)
      • 3)
        well-described research methodologies (encourages learners’ need to know about research methods for research projects)
      • 4)
        published within the past 3 months (supports learners’ motivation to review new literature)
      Articles collectively had quantitative, qualitative, and mixed-methodologies to expose participants to varied medical education research methods.
      We introduced participants to medical education research, provided the article title, discussed the background to the current study, and provided the research question. Participants were divided into small groups to brainstorm how to design study methodology for the research question specifically focusing on what they could measure to answer the question and how they would collect the data.
      After 15 minutes of discussion, each group reported their proposed methods and perceived challenges. In the large group, we revealed and scrutinized the study methodology, results, and conclusions. We focused on interpreting medical education terminology and techniques in distilled, approachable language. We then discussed results and implications. Post-session, the article was e-mailed to the residents. The Figure displays an implementation flowchart.

      Data Collection and Analysis

      We developed a simple, anonymous evaluation tool to determine participants’ perceived educational impact, which was not pre-tested before use. Hand-written responses were collected anonymously at the end of each session; responses were voluntary. Participants rated 3 critical appraisal outcome measures anchored from 1 (not at all helpful) to 7 (extremely helpful) (Table 1) and were asked 2 qualitative, open-ended questions about what went well and what could have been done differently.
      Table 1Quantitative and Qualitative Educational Evaluation Results
      Quantitative—How Helpful Was This Journal Club in:Low ScoreHigh ScoreMean ScoreModal ScoreStandard DeviationInterquartile Range
      Helping you to think about how to analyze a paper375.326.001.111.25
      Helping you to think about how to use a paper to inform further study questions375.426.001.241.00
      Helping you to understand what medical education research is and how it can be conducted476.007.001.011.25
      Qualitative - Theme #1: Strong Interactive Learning Experience
      “Asking the group how they would approach investigating the questions allowed for a good discussion.”
      “Excellent brief review of specific and relevant statistical tools.”
      “I loved developing our own methods before learning paper method.”
      Qualitative - Theme #2: Preparation Not Required
      “Great participation despite not reading paper.”
      “I appreciated being able to go through the paper on the spot.”
      Qualitative - Theme #3: Improvement Is Possible
      “Offer a little more of the session leaders' thoughts during discussion.”
      “Try to more evenly distribute medical students and residents in the small groups.”
      “We got off topic on the study question in the small groups.”
      Qualitative - Theme #4: Desire for Further Learning
      “I would have liked going deeper into results.”
      “Provide access to take article home.”
      “I enjoyed focusing less on specifics of this paper but allowing us the chance to think about how we would approach it.”
      Quantitative data were entered into Microsoft Excel (Office 365; Microsoft, Redmond, Wash) and analyzed for descriptive statistics including mean, mode, and interquartile range.
      To analyze open-ended questions, the authors conducted a thematic analysis following the “five stages to qualitative research” framework.
      • Ritchie J
      • Spencer L
      Qualitative data analysis for applied policy research.
      Dedoose (version 8.1; SocioCultural Research Consultants, LLC, Los Angeles, Calif) was used to facilitate data management. First, 2 coders (K.D. and A.F.V.) familiarized themselves with the data, each independently reading the responses to the 2 qualitative questions and creating 2 independent preliminary code lists with 40 (A.F.V.) and 26 (K.D.) codes, respectively. The coders then compared codes, collapsed codes, and refined definitions to develop a codebook containing 29 final codes that were then applied to all data. After all codes were applied, the coders discussed recurring data patterns and combined codes into categories (n = 11), followed by themes (n = 4). The coders independently reread the coded data within each theme to ensure coding consistency.
      The educational evaluation was designated “clinical quality improvement/measurement” by the Partners Human Research Committee, requiring no additional institutional review board review.

      Results

      In total, 59 participants attended at least 1 of the JCs. Fifty-two evaluations were returned (88% response rate). Overall, the JC was well-received, with mean scores for the 3 objective outcome measures falling between 5.32 and 6.00 on a 7-point scale (Table 1).
      A total of 45 respondents (86.5%) answered at least one qualitative question. Thematic analysis resulted in 4 themes:
      • Theme #1: Strong Interactive Learning Experience
      • Respondents felt that developing methods via active learning was a strength, the small group experience was valuable, and discussion enabled a positive learning experience. Although the JC was not perceived as easy, the session leaders were viewed as strong teachers providing an innovative and engaging learning experience (Table 1).
      • Theme #2: Preparation Not Required
      • That preparation was not required before the JC session was perceived as a welcome change to their training and a strength by participants (Table 1).
      • Theme #3: Improvement Is Possible
      • Potential improvements included a desire for additional input from the session leaders and an adequate mix of different training levels within small groups. Respondents wanted group facilitation to keep small group discussion on track. Some suggestions included preferences that were outside the scope of the JC, such as including case studies or vignettes (Table 1).
      • Theme #4: Desire for Further Learning
      • Respondents indicated an interest in continued learning. They would have preferred that printed copies of the article were available at the JCs. They also suggested that the large group discussions led by the session leaders might offer a deeper dive into the results. Post-session, respondents desired additional research terms and methods resources (Table 1).

      Discussion

      We designed an “interactive, no-prep” approach to JC targeted toward millennial learners. The JCs were well received; participants indicated having a strong overall learning experience. This approach adds to the literature on innovative JCs by combining interactive techniques
      • Dzara K
      • Jain G
      • Soltys SM
      The self-directed, structured summary as a teaching tool in a psychiatry journal club.
      • MacMillan TE
      • Rawal S
      • Cram P
      • et al.
      A journal club for peer mentorship: helping to navigate the transition to independent practice.

      Johns Hopkins Medicine. Osler Journal Club. 2019. Available at:https://www.hopkinsmedicine.org/gim/training/Osler_Journal_Club/index.html. Accessed April 22, 2019.

      • Franquet E
      • Parker J
      • Donohoe K
      Question-writing Journal-Club is an alternative learning method for the residents.
      • Antonoff M
      • Nguyen T
      • Luc J
      • et al.
      A structured, debate-style cardiothoracic surgery journal club for trainee acquisition and application of seminal literature.
      with a no-preparation approach,
      • Rosenthal J
      • Rosenthal KS
      Interactive journal club: teaching an old dog new tricks.
      • Crichlow R
      Trans-formative journal club experience as a basis for a longitudinal EBM curriculum.
      both reported in other papers, and assessing not only residents’ quantitative assessment of their learning, but their qualitative feedback. This mixed methods approach allows for a better understanding of what JC aspects were helpful to participants. Based on their qualitative feedback, we attribute our success, in part, to the session leaders recognizing the learning preferences of clinically busy millennial learners in 3 key ways: we did not require previous preparation, we allowed for active engagement with the material, and we made the information relevant. To allow for participants not to prepare in advance, we needed to ensure that we explained educational research terminology and methodology in real time. As participants developed ideas for study methods, we explained how their ideas fit into research methodology and offered terminology for the concepts they described. We applied active learning techniques by having participants engage in study design, analysis of results, and application of findings to their own experiences; by having participants create their own study methodology, we used the highest level of learning objectives in Bloom's taxonomy, creating.
      • Adams NE
      Bloom's taxonomy of cognitive learning objectives.
      We used articles that aligned with adult learning principles and made the information relevant by choosing articles on familiar graduate medical education topics such as patient feedback, milestones-based ratings, organizational culture and feedback, and remote supervision. The discussions offered opportunity for participants to demonstrate autonomy, share expertise, and work in teams. Participants desired additional learning resources and more engagement with material, such as additional in-depth discussion, which suggests they found the material meaningful. Because we formally evaluated the JCs, we add to the limited literature regarding the utility of both interactive and no-preparation approaches to JCs.

      Limitations

      The JCs took place at a single pediatric residency program over a short time period. Resident schedules limited participation. Because we conducted a routine educational evaluation, our tool was not validated and only assessed participants’ perception of learning, which lacks objectivity; there is evidence that physicians have a limited ability to accurately self-assess.
      • Davis DA
      • Mazmanian PE
      • Fordis M
      • et al.
      Accuracy of physician self-assessment compared with observed measures of competence: a systematic review.
      We also did not conduct a pre-survey to demonstrate change in participants’ perception of learning. To mitigate this, we used thematic analysis to understand participants’ qualitative reactions to their learning experience. Finally, using 4 different articles limits generalizability but demonstrates wide application potential.

      Next Steps

      This JC strategy could be implemented in other medical fields and for other research types. It is reproducible; the overall time commitment to develop each session was 2 to 3 hours. The implementation flowchart aids educators in adapting this model for their own institutions (Figure). Faculty also could consider encouraging this model for self-study. Participants could read the article title and introduction, stop and detail a possible methodology for the study, and then read the methods and critique them. Participants could then compare their proposed methodology with the actual methodology and consider why the authors made their methodologic decisions. They could then read the results and conclusions and determine, for themselves, the paper's implications and whether they agreed with the authors’ conclusions. By practicing this in a large group setting and then trying it out individually, participants might learn a literature review technique which forces them to actively analyze while reading.
      We plan to improve our JC by introducing a faculty moderator in each group, ensuring the groups have a mix of learners, leaving more time for detailed discussion of results and implications, and providing resources for additional self-directed learning. Future studies could assess the impact on residents’ approach to reading literature and their knowledge of medical education research methodologies by giving residents a medical education article and having them individually analyze the methodology, results, conclusions, and implications to determine their ability to apply the process individually.

      Conclusions

      We designed an “interactive, no-prep” approach to JC targeted toward millennial learners using the principles of adult learning and combating the challenge of engaging unprepared learners. We focused on exposing pediatric residents to medical education research using active learning principles, making the experience relevant to them by choosing articles focusing on graduate medical education. Our approach was well-received by participants, who noted having a better understanding of what medical education research is and how it is conducted and left the sessions better equipped to analyze a medical education research paper and use that information to inform further study questions. Our experience is proof of concept that the need to teach critical evaluation of the literature and millennial residents’ needs can be met simultaneously in an “interactive, no-prep” JC.

      Acknowledgments

      We acknowledge Benjamin Nelson, MD, for his helpful comments on an early draft of this paper.

      References

        • Deenadayalan Y
        • Grimmer-Somers K
        • Prior M
        • et al.
        How to run an effective journal club: a systematic review.
        J Eval Clin Pract. 2008; 14: 898-911
        • Moharari RS
        • Rahimi E
        • Najafi A
        • et al.
        Teaching critical appraisal and statistics in anesthesia journal club.
        QJM. 2009; 102: 139-141
        • Topf JM
        • Sparks MA
        • Phelan PJ
        • et al.
        The evolution of the journal club: from Osler to Twitter.
        Am J Kidney Dis. Jun 2017; 69: 827-836
        • Alguire PC
        A review of journal clubs in postgraduate medical education.
        J Gen Intern Med. 1998; 13: 347-353
        • Dzara K
        • Jain G
        • Soltys SM
        The self-directed, structured summary as a teaching tool in a psychiatry journal club.
        Acad Psychiatry. 2012; 36: 490-492
        • McLeod P
        • Steinert Y
        • Boudreau D
        • et al.
        Twelve tips for conducting a medical education journal club.
        Med Teach. 2010; 32: 368-370
        • Hopkins L
        • Hampton BS
        • Abbott JF
        • et al.
        To the point: medical education, technology, and the millennial learner.
        Am J Obstet Gynecol. 2018; 218: 188-192
        • Shatto B
        • Erwin K
        Teaching Millennials and Generation Z: bridging the generational divide.
        Creat Nurs. 2017; 23: 24-28
        • Rosenthal J
        • Rosenthal KS
        Interactive journal club: teaching an old dog new tricks.
        J Med Educ Curric Dev. 2017; 42382120517719710
        • Crichlow R
        Trans-formative journal club experience as a basis for a longitudinal EBM curriculum.
        in: Paper Presented at: Society of Teachers of Family Medicine Conference. San Antonio, Texas, May 3–7, 2014
        • MacMillan TE
        • Rawal S
        • Cram P
        • et al.
        A journal club for peer mentorship: helping to navigate the transition to independent practice.
        Perspect Med Educ. 2016; 5: 312-315
      1. Johns Hopkins Medicine. Osler Journal Club. 2019. Available at:https://www.hopkinsmedicine.org/gim/training/Osler_Journal_Club/index.html. Accessed April 22, 2019.

        • Franquet E
        • Parker J
        • Donohoe K
        Question-writing Journal-Club is an alternative learning method for the residents.
        J Nucl Med. 2016; 57 (1252–1252)
        • Antonoff M
        • Nguyen T
        • Luc J
        • et al.
        A structured, debate-style cardiothoracic surgery journal club for trainee acquisition and application of seminal literature.
        MedEdPORTAL. 2016; 12: 10521
        • Simpson D
        • Flynn C
        • Wendelberger K
        An evidence-based education journal club.
        Acad Med. 1997; 72: 464
        • Centeno AM
        • Blanco A
        • Arce M
        Journal club devoted to educational issues.
        Acad Med. May 1999; 74: 464
        • Knowles MS
        • Holton E
        • Swanson R
        The Adult Learner: The Definitive Classic in Adult Education and Human Resource Development.
        6th ed. Elsevier, Burlington, MA2005
        • Ritchie J
        • Spencer L
        Qualitative data analysis for applied policy research.
        in: Bryman A Burgess B Analyzing Qualitative Data. Routledge, London1994: 175-194
        • Adams NE
        Bloom's taxonomy of cognitive learning objectives.
        J Med Libr Assoc. 2015; 103: 152-153
        • Davis DA
        • Mazmanian PE
        • Fordis M
        • et al.
        Accuracy of physician self-assessment compared with observed measures of competence: a systematic review.
        JAMA. 2006; 296: 1094-1102