Advertisement

Not Throwing Away My Shot: Leveraging a Peer Vaccination Workshop to Increase Residents’ Immunization Skills

      Abstract

      Objective

      The Accreditation Council for Graduate Medical Education (ACGME) requires that pediatric residents demonstrate competence administering immunizations. Despite mandatory simulation training, less than half our residents reported immunization competence. All residents need to receive their influenza vaccination, but schedule restrictions present logistical challenges. We implemented a peer influenza immunization workshop and assessed the impact on resident immunization competence compared to simulation-only training.

      Methods

      Prospective cohort study at a pediatric residency program in a tertiary academic center. We implemented an annual influenza immunization workshop including immunization education, simulated practice, and peer influenza immunization. We compared workshop participation to simulation-only training on resident confidence immunizing, number of immunizations logged, and ACGME survey results for immunization skills.

      Results

      In 2019, 80% (N = 59) of residents participated in the workshop. Participants were more likely to report confidence in immunization skills than nonparticipants (P = .001). Resident-administered immunizations increased from 1 in the 3 years preceding workshop implementation to 74 during the 2019 to 2020 academic year. Significantly, more ACGME survey respondents reported preparedness to immunize after workshop implementation (P = .02).

      Conclusions

      Implementation of an influenza immunization workshop provides an innovative opportunity to increase resident preparedness performing an ACGME-required procedure while also helping ensure programs remain compliant with influenza requirements.

      Keywords

      What's New
      Implementation of an influenza immunization workshop is associated with improved resident self-confidence in their immunization skills and procedural experience. This provides an opportunity to increase resident preparedness performing an Accreditation Council for Graduate Medical Education-required procedure while also helping programs remain compliant with influenza requirements.
      The Accreditation Council for Graduate Medical Education (ACGME) requires that pediatric residents demonstrate competence administering immunizations.

      Accreditation Council on Graduate Medical Education. ACGME program requirements for graduate medical education in pediatrics. 2020. Available at:https://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/320_Pediatrics_2020.pdf?ver=2020-06-29-162726-647. Accessed February 9, 2020.

      A national survey of pediatric residents found that only 66% received formal education on this skill and over 20% did not feel comfortable immunizing at the end of their training.
      • Schumacher DJ
      • Frintner MP
      • Cull W
      Graduating pediatric resident reports on procedural training and preparation.
      A majority of surveyed program directors felt immunization administration is a highly important skill that residents are not well prepared to perform.
      • Bismilla Z
      • Dubrowski A
      • Amin HJ
      Program directors’ perceptions of importance of pediatric procedural skills and resident preparedness.
      Local ACGME survey results indicated similar trends. Despite having mandatory simulation training, less than half our pediatric residents felt prepared to immunize. Graduate survey results indicated a desire to be more prepared. Local logs demonstrated that almost none of our graduates administered a nonsimulated immunization. Residents identify a lack of opportunities to immunize in a nonsimulation environment as a contributor to their lack of confidence. Although immunizations are frequently administered in clinical environments, the fast pace and increasing presence of nonphysician clinicians performing procedures traditionally performed by residents can preclude residents from practicing this skill.
      • Gaies MG
      • Landrigan CP
      • Hafler JP
      • et al.
      Assessing procedural skills training in pediatric residency programs.
      ,
      • Gozzo YF
      • Cummings CL
      • Chapman RL
      • et al.
      Who is performing medical procedures in the neonatal intensive care unit?.
      In addition, certain local sites restrict residents from immunizing. There is a paucity of literature on what approaches are most effective at improving resident immunization skills.
      According to Center for Disease Control guidelines and hospital policy, residents must receive annual influenza vaccinations unless medically contraindicated.

      Centers for Disease Control and Prevention. Everyone 6 months of age and older should get a flu vaccine. Available at: https://www.cdc.gov/flu/prevent/vaccinations.htm. 2019. Accessed February 9, 2020.

      This can be logistically challenging for residents due to schedule restrictions. An internal medicine survey found that over a third of trainees did not have primary care physicians and over half indicated that residency makes access to health care difficult.
      • Rosen IM
      • Christie JD
      • Bellini LM
      • et al.
      Health and health care among housestaff in four U.S. internal medicine residency programs.
      Historically, our program's Occupational Health attended a conference to immunize trainees. Starting in 2017, this was no longer available due to institutional limitations. We therefore needed a new approach to ensure our residents were efficiently immunized.
      To address the need for additional immunization training and a process to ensure our residents received influenza vaccines, we developed a hands-on peer influenza immunization workshop. Our hypothesis was that workshop participation would increase resident confidence in immunization skills and allow them to efficiently receive their influenza vaccination. The conceptual framework of deliberate practice supports an educational approach of practicing skills with immediate feedback and repetition to develop skills.
      • Ericsson KA
      Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains.
      Simulation education has been shown to play an important role in this process for procedure training.
      • Nestel D
      • Groom J
      • Eikeland-Husebø S
      • et al.
      Simulation for learning and teaching procedural skills: the state of the science.
      ,
      • Augustine EM
      • Kahana M
      Effect of procedure simulation workshops on resident procedural confidence and competence.
      The limitations of simulation have also been documented, however, with prior work suggesting nonsimulation exposure may also be required in some instances to achieve procedural competence. The objective of this study was to compare the impact of a peer influenza immunization workshop to baseline simulation-only training on 1) resident confidence in immunization skills, 2) number of immunizations residents logged, and 3) program ACGME survey results related to immunization skills.

      Methods

      Setting

      We conducted a prospective cohort study at a medium-sized (∼26 resident per class) pediatric residency program at a tertiary academic medical center. At baseline, all residents took part in a workshop about immunizations using a simulation task trainer. There was no other formal immunization curriculum prior to this project.

      Intervention

      We developed a 15-minute voluntary workshop including didactic education, simulated practice, and peer immunization. Residents watched a brief didactic overview by the session facilitator on indications, types of immunizations, required equipment, and proper technique. They then practiced using a low-fidelity simulation injection task trainer. Residents then administered the influenza vaccination in pairs to co-residents under direct supervision of a trained observer, an associate program director trained in the procedure by nursing educators. The observer provided feedback during the simulation and observed during peer administration to ensure immunizations were administered correctly. Three annual workshops were conducted starting in 2017.
      We partnered with our Occupational Health who provided supplies and assisted with supply disposal. The workshop was offered immediately prior to/during/after previously established conferences at the same location to promote attendance. Three sessions were offered annually on separate weeks to support attendance. Workshop participation and survey completion were voluntary. Residents were provided alternative options of receiving their immunization from a nurse except if contraindicated in accordance with hospital policies.

      Outcomes

      We used Kirkpatrick's framework
      • Kirkpatrick DL
      Evaluating Training Programs: The Four Levels.
      for curriculum evaluation and identified outcomes of resident reaction, behavior change, and high stakes outcomes of ACGME survey results as described below.

      Reaction

      One month following the inaugural workshop in 2017, we surveyed all residents to assess their confidence administering immunizations by a 5-point Likert scale (1 = low, 5 = high). They were also asked to indicate whether they participated in the workshop. Participating residents were also asked if the experience was conducive to learning (5-point Likert scale, 1 = low, 5 = high).

      Behavior Change

      Resident logs were reviewed to compare the number of residents who logged immunizations and the number of immunizations logged prior to and after workshop implementation.

      High Stakes Survey Results

      Annual ACGME surveys for pediatric residency programs ask residents to rate their agreement with the following statement on a 5-point Likert scale: I feel well prepared to administer immunizations without supervision. Survey results were compared prior to and after workshop implementation. Similar comparisons were performed for 2 procedures not targeted with educational initiatives (incision and drainage and bladder catheterization) to serve as a baseline control for potential improvements in overall procedural confidence of our trainees over this timeframe unrelated to the targeted immunization education. Survey results were compared for the program as a whole prior to and after workshop implementation regardless of workshop participation. The anonymous data collection precluded us from looking specifically at those who participated in the workshop.
      We also tracked resident receipt of their influenza immunization prior to institutional deadlines.

      Data Analysis

      Analysis was generated with SPSS version 24.0 (IBM Corp, Armonk, NY). Descriptive statistics were used to report workshop participation, immunization logs counts, and ACGME survey results. Mann-Whitney U test was used to assess differences in resident immunization confidence between those who did or did not participate. Fisher's exact tests were used to compare residents’ self-reported procedural preparedness across 3 procedures before and after workshops began.

      Ethical Considerations

      This project was determined to meet the criteria for registration in accordance with our institution's Institutional Review Board.

      Results

      The voluntary workshop has been in place for 3 years. During the initial year, 36% of categorical pediatric residents participated (N = 27). During the third year, 80% (N = 59) participated. Participants included all 3 years of training.

      Reaction

      Sixty-one percent of our program completed the survey 1 month post the initial workshop in 2017 assessing their confidence in immunization skills. Residents who participated in the workshop were more likely to report feeling confident (n = 19, median rating 4, interquartile range 1) in their immunization skills than those who did not (n = 26, median rating 3, interquartile range 1; P = .001).
      Hundred percent of participants reported workshop participation was conducive to their learning.

      Behavior Change

      In the 3 years preceding workshop implementation, 1 resident logged 1 nonsimulated immunization. This increased following workshop implementation, with 59 distinct residents logging a total of 74 nonsimulated immunizations during the 2019 to 2020 academic year (Fig. 1). The majority were administered as part of the workshop to peers or faculty (n = 69) and 5 to patients in clinical settings.
      Figure 1
      Figure 1Number of residents who logged nonsimulation immunization procedures and number of immunizations logged before and after workshop implementation. Total number of residents in program ∼74.

      High Stakes Survey Results

      There was a significant increase in ACGME survey respondents who reported they agreed/strongly agreed with the statement “I feel well prepared to perform [immunizations] without supervision” after workshop implementation (38%–64%, P < .02, Fig. 2). No change was seen regarding bladder catheterization or incision and drainage, procedures not targeted for educational intervention (Fig. 2). Although our program's self-reported preparedness to immunize remains below the national average, the gap is closing as improvements seen locally were not mirrored by national trends which remained consistent over the timeframe (Fig. 2).
      Figure 2
      Figure 2Comparison ACGME survey respondents’ level of agreed with the statement “I feel well prepared to perform [procedures] without supervision” before and after workshop implementation. ACGME indicates Accreditation Council for Graduate Medical Education.

      Program Compliance With Influenza Guidelines

      Hundred percent of our residents received influenza immunizations prior to hospital deadline. No adverse events were reported as a result of immunizations administered during the workshop.

      Discussion

      The implementation of an influenza immunization workshop was associated with improved resident self-confidence in immunization skills, increased number of immunizations residents’ log, and improved ACGME survey results related to immunization preparation.
      We saw an increase in resident confidence in immunization skills, both in our 1-month post workshop survey and our annual ACGME surveys. Although confidence does not necessarily indicate competence, prior work suggests a correlation between resident self-reported confidence in procedural skills and how they perform,
      • Resch J
      • Smolcich A
      • Rogers A
      • et al.
      Pediatric resident self-confidence, not past experience, is correlated with their ability to perform bag-mask ventilation and endotracheal intubation.
      supporting the value of including this metric in assessing procedural educational interventions. It is also notable that no changes were seen for procedures not targeted by our workshop, suggesting an impact from the workshop itself rather than unrelated general improvements in resident procedure confidence over the study timeframe.
      We also saw an increase in immunizations logged following the workshop. The majority were administered within the workshop. It is possible there are additional barriers preventing residents from immunizing in clinical settings apart from a lack of procedural skills including time restrictions and other providers performing procedures.
      • Gozzo YF
      • Cummings CL
      • Chapman RL
      • et al.
      Who is performing medical procedures in the neonatal intensive care unit?.
      An additional benefit of this workshop was that it provided a feasible opportunity to ensure our program remained compliant with institutional requirements and national recommendations to receive influenza vaccinations. Although we historically achieved 100% compliance, this was previously reliant on on-site Occupational Health support. There was a concern that with the loss of that resource in 2017, we were at risk of being out of compliance. The workshop therefore provided a dual benefit of creating a novel educational opportunity while also ensuring we maintained compliance with these requirements.
      Identifying ways to effectively teach immunization skills is valuable given the omnipresent role immunizations play in multiple specialties and that it is an ACGME-mandated skill in programs including pediatrics.

      Accreditation Council on Graduate Medical Education. ACGME program requirements for graduate medical education in pediatrics. 2020. Available at:https://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/320_Pediatrics_2020.pdf?ver=2020-06-29-162726-647. Accessed February 9, 2020.

      Although a variety of studies assess vaccine-related education including knowledge about vaccine safety and schedules and communication skills related to vaccine hesitancy,
      • Pelly LP
      • Pierrynowski MacDougall DM
      • Halperin BA
      • et al.
      The vaxed project: an assessment of immunization education in Canadian health professional programs.
      ,
      • Zimmerman RK
      • Barker WH
      • Strikas RA
      • et al.
      Developing curricula to promote preventive medicine skills. The teaching immunization for medical education (TIME) project. TIME Development Committee.
      there is a paucity of literature on how to best teach the procedure itself. Simulation has been shown to have a valuable role within procedural education although its limitations are also well documented. Simulation alone may promote poor practices including missing key safety steps or improper use of personal protective equipment.
      • Weller JM
      • Nestel D
      • Marshall SD
      • et al.
      Simulation in clinical teaching and learning.
      Therefore there is potential benefit in pairing simulations with in vivo procedural experiences. Our findings support the implementation of a peer influenza immunization workshop for programs that are aiming to enhance their immunization training as well as those looking for ways to ensure their trainees receive their influenza immunization.
      This educational innovation was acceptable to our residents, as evidenced by all participants endorsing it was conducive to learning and the incremental increased proportion of residents attending annually. It was feasible because supplies are provided by Occupational Health at no cost to the program. The workshop is reproducible due to the annual need for influenza immunizations. Similar workshops could be implemented in specialties given the role of immunizations in other fields including family practice and internal medicine and the universal recommendation to receive the immunization is not unique to pediatrics.
      There were limitations to this study. The voluntary nature of the workshop precluded us from randomizing groups. There was a small sample size at a single institution. Results were primarily limited to self-reported confidence and may not correlate with actual improved performance. Behavior results obtained from resident logs may not accurately reflect the number of immunizations due to underreporting. In addition, although didactic and simulation components of the workshop reviewed multiple types of immunizations (intramuscular, subcutaneous, intradermal), the peer immunization element was isolated to intramuscular vaccinations to adults, and there are currently limited opportunities to reliably continue to practice these skills in clinical settings.
      Next steps include identifying opportunities to practice other forms of vaccinations and gain experience with pediatric patients as part of this ongoing training. We also plan to incorporate validated observation tools to objectively assess immunization skills and continue to monitor resident logs to assess potential long-term impacts of the workshop on resident performance of immunizations in the clinical setting.

      Conclusions

      Implementations of an influenza immunization workshop provides an innovative opportunity to increase resident confidence and experience performing an ACGME-required procedural skill while also helping to ensure program compliance with institutional and national guidelines pertaining to influenza immunization requirements.

      References

      1. Accreditation Council on Graduate Medical Education. ACGME program requirements for graduate medical education in pediatrics. 2020. Available at:https://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/320_Pediatrics_2020.pdf?ver=2020-06-29-162726-647. Accessed February 9, 2020.

        • Schumacher DJ
        • Frintner MP
        • Cull W
        Graduating pediatric resident reports on procedural training and preparation.
        Acad Pediatr. 2018; 18: 73-78https://doi.org/10.1016/j.acap.2017.08.001
        • Bismilla Z
        • Dubrowski A
        • Amin HJ
        Program directors’ perceptions of importance of pediatric procedural skills and resident preparedness.
        BMC Res Notes. 2015; 8: 550https://doi.org/10.1186/s13104-015-1499-8
        • Gaies MG
        • Landrigan CP
        • Hafler JP
        • et al.
        Assessing procedural skills training in pediatric residency programs.
        Pediatrics. 2007; 120: 715-722https://doi.org/10.1542/peds.2007-0325
        • Gozzo YF
        • Cummings CL
        • Chapman RL
        • et al.
        Who is performing medical procedures in the neonatal intensive care unit?.
        J Perinatol. 2011; 31: 206-211https://doi.org/10.1038/jp.2010.121
      2. Centers for Disease Control and Prevention. Everyone 6 months of age and older should get a flu vaccine. Available at: https://www.cdc.gov/flu/prevent/vaccinations.htm. 2019. Accessed February 9, 2020.

        • Rosen IM
        • Christie JD
        • Bellini LM
        • et al.
        Health and health care among housestaff in four U.S. internal medicine residency programs.
        J Gen Intern Med. 2000; 15: 116-121https://doi.org/10.1046/j.1525-1497.2000.11218.x
        • Ericsson KA
        Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains.
        Acad Med J Assoc Am Med Coll. 2004; 79: S70-S81https://doi.org/10.1097/00001888-200410001-00022
        • Nestel D
        • Groom J
        • Eikeland-Husebø S
        • et al.
        Simulation for learning and teaching procedural skills: the state of the science.
        Simul Healthc. 2011; 6: S10-S13https://doi.org/10.1097/SIH.0b013e318227ce96
        • Augustine EM
        • Kahana M
        Effect of procedure simulation workshops on resident procedural confidence and competence.
        J Grad Med Educ. 2012; 4: 479-485https://doi.org/10.4300/JGME-D-12-00019.1
        • Kirkpatrick DL
        Evaluating Training Programs: The Four Levels.
        1st ed. Berrett-Koehler, San Francisco, CA1994
        • Resch J
        • Smolcich A
        • Rogers A
        • et al.
        Pediatric resident self-confidence, not past experience, is correlated with their ability to perform bag-mask ventilation and endotracheal intubation.
        Acad Pediatr. 2018; 18: e5https://doi.org/10.1016/j.acap.2018.04.025
        • Pelly LP
        • Pierrynowski MacDougall DM
        • Halperin BA
        • et al.
        The vaxed project: an assessment of immunization education in Canadian health professional programs.
        BMC Med Educ. 2010; 10: 86https://doi.org/10.1186/1472-6920-10-86
        • Zimmerman RK
        • Barker WH
        • Strikas RA
        • et al.
        Developing curricula to promote preventive medicine skills. The teaching immunization for medical education (TIME) project. TIME Development Committee.
        JAMA. 1997; 278: 705-711
        • Weller JM
        • Nestel D
        • Marshall SD
        • et al.
        Simulation in clinical teaching and learning.
        Med J Aust. 2012; 196: 594https://doi.org/10.5694/mja10.11474