If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Address correspondence to Amanda Rogers, MD, Department of Pediatrics, Section of Hospital Medicine, Children's Corporate Center, Suite 560, PO Box 1997, Milwaukee, WI 53201-1197
The Accreditation Council for Graduate Medical Education (ACGME) requires that pediatric residents demonstrate competence administering immunizations. Despite mandatory simulation training, less than half our residents reported immunization competence. All residents need to receive their influenza vaccination, but schedule restrictions present logistical challenges. We implemented a peer influenza immunization workshop and assessed the impact on resident immunization competence compared to simulation-only training.
Methods
Prospective cohort study at a pediatric residency program in a tertiary academic center. We implemented an annual influenza immunization workshop including immunization education, simulated practice, and peer influenza immunization. We compared workshop participation to simulation-only training on resident confidence immunizing, number of immunizations logged, and ACGME survey results for immunization skills.
Results
In 2019, 80% (N = 59) of residents participated in the workshop. Participants were more likely to report confidence in immunization skills than nonparticipants (P = .001). Resident-administered immunizations increased from 1 in the 3 years preceding workshop implementation to 74 during the 2019 to 2020 academic year. Significantly, more ACGME survey respondents reported preparedness to immunize after workshop implementation (P = .02).
Conclusions
Implementation of an influenza immunization workshop provides an innovative opportunity to increase resident preparedness performing an ACGME-required procedure while also helping ensure programs remain compliant with influenza requirements.
Implementation of an influenza immunization workshop is associated with improved resident self-confidence in their immunization skills and procedural experience. This provides an opportunity to increase resident preparedness performing an Accreditation Council for Graduate Medical Education-required procedure while also helping programs remain compliant with influenza requirements.
The Accreditation Council for Graduate Medical Education (ACGME) requires that pediatric residents demonstrate competence administering immunizations.
A national survey of pediatric residents found that only 66% received formal education on this skill and over 20% did not feel comfortable immunizing at the end of their training.
Local ACGME survey results indicated similar trends. Despite having mandatory simulation training, less than half our pediatric residents felt prepared to immunize. Graduate survey results indicated a desire to be more prepared. Local logs demonstrated that almost none of our graduates administered a nonsimulated immunization. Residents identify a lack of opportunities to immunize in a nonsimulation environment as a contributor to their lack of confidence. Although immunizations are frequently administered in clinical environments, the fast pace and increasing presence of nonphysician clinicians performing procedures traditionally performed by residents can preclude residents from practicing this skill.
In addition, certain local sites restrict residents from immunizing. There is a paucity of literature on what approaches are most effective at improving resident immunization skills.
According to Center for Disease Control guidelines and hospital policy, residents must receive annual influenza vaccinations unless medically contraindicated.
Centers for Disease Control and Prevention. Everyone 6 months of age and older should get a flu vaccine. Available at: https://www.cdc.gov/flu/prevent/vaccinations.htm. 2019. Accessed February 9, 2020.
This can be logistically challenging for residents due to schedule restrictions. An internal medicine survey found that over a third of trainees did not have primary care physicians and over half indicated that residency makes access to health care difficult.
Historically, our program's Occupational Health attended a conference to immunize trainees. Starting in 2017, this was no longer available due to institutional limitations. We therefore needed a new approach to ensure our residents were efficiently immunized.
To address the need for additional immunization training and a process to ensure our residents received influenza vaccines, we developed a hands-on peer influenza immunization workshop. Our hypothesis was that workshop participation would increase resident confidence in immunization skills and allow them to efficiently receive their influenza vaccination. The conceptual framework of deliberate practice supports an educational approach of practicing skills with immediate feedback and repetition to develop skills.
The limitations of simulation have also been documented, however, with prior work suggesting nonsimulation exposure may also be required in some instances to achieve procedural competence. The objective of this study was to compare the impact of a peer influenza immunization workshop to baseline simulation-only training on 1) resident confidence in immunization skills, 2) number of immunizations residents logged, and 3) program ACGME survey results related to immunization skills.
Methods
Setting
We conducted a prospective cohort study at a medium-sized (∼26 resident per class) pediatric residency program at a tertiary academic medical center. At baseline, all residents took part in a workshop about immunizations using a simulation task trainer. There was no other formal immunization curriculum prior to this project.
Intervention
We developed a 15-minute voluntary workshop including didactic education, simulated practice, and peer immunization. Residents watched a brief didactic overview by the session facilitator on indications, types of immunizations, required equipment, and proper technique. They then practiced using a low-fidelity simulation injection task trainer. Residents then administered the influenza vaccination in pairs to co-residents under direct supervision of a trained observer, an associate program director trained in the procedure by nursing educators. The observer provided feedback during the simulation and observed during peer administration to ensure immunizations were administered correctly. Three annual workshops were conducted starting in 2017.
We partnered with our Occupational Health who provided supplies and assisted with supply disposal. The workshop was offered immediately prior to/during/after previously established conferences at the same location to promote attendance. Three sessions were offered annually on separate weeks to support attendance. Workshop participation and survey completion were voluntary. Residents were provided alternative options of receiving their immunization from a nurse except if contraindicated in accordance with hospital policies.
for curriculum evaluation and identified outcomes of resident reaction, behavior change, and high stakes outcomes of ACGME survey results as described below.
Reaction
One month following the inaugural workshop in 2017, we surveyed all residents to assess their confidence administering immunizations by a 5-point Likert scale (1 = low, 5 = high). They were also asked to indicate whether they participated in the workshop. Participating residents were also asked if the experience was conducive to learning (5-point Likert scale, 1 = low, 5 = high).
Behavior Change
Resident logs were reviewed to compare the number of residents who logged immunizations and the number of immunizations logged prior to and after workshop implementation.
High Stakes Survey Results
Annual ACGME surveys for pediatric residency programs ask residents to rate their agreement with the following statement on a 5-point Likert scale: I feel well prepared to administer immunizations without supervision. Survey results were compared prior to and after workshop implementation. Similar comparisons were performed for 2 procedures not targeted with educational initiatives (incision and drainage and bladder catheterization) to serve as a baseline control for potential improvements in overall procedural confidence of our trainees over this timeframe unrelated to the targeted immunization education. Survey results were compared for the program as a whole prior to and after workshop implementation regardless of workshop participation. The anonymous data collection precluded us from looking specifically at those who participated in the workshop.
We also tracked resident receipt of their influenza immunization prior to institutional deadlines.
Data Analysis
Analysis was generated with SPSS version 24.0 (IBM Corp, Armonk, NY). Descriptive statistics were used to report workshop participation, immunization logs counts, and ACGME survey results. Mann-Whitney U test was used to assess differences in resident immunization confidence between those who did or did not participate. Fisher's exact tests were used to compare residents’ self-reported procedural preparedness across 3 procedures before and after workshops began.
Ethical Considerations
This project was determined to meet the criteria for registration in accordance with our institution's Institutional Review Board.
Results
The voluntary workshop has been in place for 3 years. During the initial year, 36% of categorical pediatric residents participated (N = 27). During the third year, 80% (N = 59) participated. Participants included all 3 years of training.
Reaction
Sixty-one percent of our program completed the survey 1 month post the initial workshop in 2017 assessing their confidence in immunization skills. Residents who participated in the workshop were more likely to report feeling confident (n = 19, median rating 4, interquartile range 1) in their immunization skills than those who did not (n = 26, median rating 3, interquartile range 1; P = .001).
Hundred percent of participants reported workshop participation was conducive to their learning.
Behavior Change
In the 3 years preceding workshop implementation, 1 resident logged 1 nonsimulated immunization. This increased following workshop implementation, with 59 distinct residents logging a total of 74 nonsimulated immunizations during the 2019 to 2020 academic year (Fig. 1). The majority were administered as part of the workshop to peers or faculty (n = 69) and 5 to patients in clinical settings.
Figure 1Number of residents who logged nonsimulation immunization procedures and number of immunizations logged before and after workshop implementation. Total number of residents in program ∼74.
There was a significant increase in ACGME survey respondents who reported they agreed/strongly agreed with the statement “I feel well prepared to perform [immunizations] without supervision” after workshop implementation (38%–64%, P < .02, Fig. 2). No change was seen regarding bladder catheterization or incision and drainage, procedures not targeted for educational intervention (Fig. 2). Although our program's self-reported preparedness to immunize remains below the national average, the gap is closing as improvements seen locally were not mirrored by national trends which remained consistent over the timeframe (Fig. 2).
Figure 2Comparison ACGME survey respondents’ level of agreed with the statement “I feel well prepared to perform [procedures] without supervision” before and after workshop implementation. ACGME indicates Accreditation Council for Graduate Medical Education.
Hundred percent of our residents received influenza immunizations prior to hospital deadline. No adverse events were reported as a result of immunizations administered during the workshop.
Discussion
The implementation of an influenza immunization workshop was associated with improved resident self-confidence in immunization skills, increased number of immunizations residents’ log, and improved ACGME survey results related to immunization preparation.
We saw an increase in resident confidence in immunization skills, both in our 1-month post workshop survey and our annual ACGME surveys. Although confidence does not necessarily indicate competence, prior work suggests a correlation between resident self-reported confidence in procedural skills and how they perform,
supporting the value of including this metric in assessing procedural educational interventions. It is also notable that no changes were seen for procedures not targeted by our workshop, suggesting an impact from the workshop itself rather than unrelated general improvements in resident procedure confidence over the study timeframe.
We also saw an increase in immunizations logged following the workshop. The majority were administered within the workshop. It is possible there are additional barriers preventing residents from immunizing in clinical settings apart from a lack of procedural skills including time restrictions and other providers performing procedures.
An additional benefit of this workshop was that it provided a feasible opportunity to ensure our program remained compliant with institutional requirements and national recommendations to receive influenza vaccinations. Although we historically achieved 100% compliance, this was previously reliant on on-site Occupational Health support. There was a concern that with the loss of that resource in 2017, we were at risk of being out of compliance. The workshop therefore provided a dual benefit of creating a novel educational opportunity while also ensuring we maintained compliance with these requirements.
Identifying ways to effectively teach immunization skills is valuable given the omnipresent role immunizations play in multiple specialties and that it is an ACGME-mandated skill in programs including pediatrics.
Although a variety of studies assess vaccine-related education including knowledge about vaccine safety and schedules and communication skills related to vaccine hesitancy,
Developing curricula to promote preventive medicine skills. The teaching immunization for medical education (TIME) project. TIME Development Committee.
there is a paucity of literature on how to best teach the procedure itself. Simulation has been shown to have a valuable role within procedural education although its limitations are also well documented. Simulation alone may promote poor practices including missing key safety steps or improper use of personal protective equipment.
Therefore there is potential benefit in pairing simulations with in vivo procedural experiences. Our findings support the implementation of a peer influenza immunization workshop for programs that are aiming to enhance their immunization training as well as those looking for ways to ensure their trainees receive their influenza immunization.
This educational innovation was acceptable to our residents, as evidenced by all participants endorsing it was conducive to learning and the incremental increased proportion of residents attending annually. It was feasible because supplies are provided by Occupational Health at no cost to the program. The workshop is reproducible due to the annual need for influenza immunizations. Similar workshops could be implemented in specialties given the role of immunizations in other fields including family practice and internal medicine and the universal recommendation to receive the immunization is not unique to pediatrics.
There were limitations to this study. The voluntary nature of the workshop precluded us from randomizing groups. There was a small sample size at a single institution. Results were primarily limited to self-reported confidence and may not correlate with actual improved performance. Behavior results obtained from resident logs may not accurately reflect the number of immunizations due to underreporting. In addition, although didactic and simulation components of the workshop reviewed multiple types of immunizations (intramuscular, subcutaneous, intradermal), the peer immunization element was isolated to intramuscular vaccinations to adults, and there are currently limited opportunities to reliably continue to practice these skills in clinical settings.
Next steps include identifying opportunities to practice other forms of vaccinations and gain experience with pediatric patients as part of this ongoing training. We also plan to incorporate validated observation tools to objectively assess immunization skills and continue to monitor resident logs to assess potential long-term impacts of the workshop on resident performance of immunizations in the clinical setting.
Conclusions
Implementations of an influenza immunization workshop provides an innovative opportunity to increase resident confidence and experience performing an ACGME-required procedural skill while also helping to ensure program compliance with institutional and national guidelines pertaining to influenza immunization requirements.
Centers for Disease Control and Prevention. Everyone 6 months of age and older should get a flu vaccine. Available at: https://www.cdc.gov/flu/prevent/vaccinations.htm. 2019. Accessed February 9, 2020.
Developing curricula to promote preventive medicine skills. The teaching immunization for medical education (TIME) project. TIME Development Committee.
30441-1&id=gr1.jpg){kind=link}
30441-1&id=gr2.jpg){kind=link}