If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Address correspondence to Brad Sobolewski, MD, MEd, Division of Emergency Medicine, Department of Pediatrics, University of Cincinnati, Cincinnati Children's Hospital Medical Center, 3333 Burnet Ave, Cincinnati, OH 45229.
In the lay press, concerns about the transition from medical school to residency have been embodied in the so-called July effect, with reports of increased medical errors in the first month of the traditional academic calendar.
These transitions between rotations present many challenges, including different care environments, collaboration with a new set of providers, unique patient populations, varied severity of acute and chronic diseases, and differing learning objectives and clinical expectations. Pediatric residency programs must adhere to the Accreditation Council for Graduate Medical Education specifications for the quantity and specific type of rotation experiences, with each rotation including a formal orientation with accompanying explicit discussion of goals and objectives.
The rotation orientation is intended to mitigate the impact of transitions on patient care and resident learning. A well-structured orientation should explain resident roles and responsibilities, introduce them to critical personnel, and highlight the unique features of the rotation.
From the perspective of residency program and rotation directors, the heterogeneity of clinical rotations and work-hour regulations have made effectively orienting and transitioning residents to and from rotations more challenging.
While educators agree that it is important to ensure residents are prepared to transition into new roles, the development, implementation, and effectiveness of resident orientations are understudied aspects of training.
Our objective was to apply the full process of curriculum development to create and evaluate a Web-based multimedia orientation to the pediatric emergency department (PED) that would supplement and reinforce the basic rotation introduction covered during the first week of the PL1 year. We present our experience with the development of this orientation, beginning with a clear identification of the problems we faced, followed by a targeted needs assessment, design, implementation, and finally impact assessment. The study was approved by the institutional review board at Cincinnati Children's Hospital Medical Center (CCHMC).
Problem Identification and Needs Assessment
CCHMC is an urban academic pediatric institution that is the major regional provider of emergency care to children, with approximately 85,000 annual visits to the emergency department (ED). The pediatric residency program has nearly 200 total residents, and approximately 120 are categorical pediatrics. Pediatric residents at CCHMC spend up to 4 nonconsecutive months in the ED, with individual rotations lasting either 2 weeks or 1 month. First-year pediatric residents are required to spend 1 month total in the ED; second and higher year residents rotate for a total of 3 months during their remaining years of training. Residents at all levels of training work 10-hour shifts. During a given month, there may be as many as 40 residents in the ED across all levels of training (categorical pediatrics, combined pediatric programs, and rotating residents from emergency medicine and family medicine). As is common within training institutions, clinical roles and responsibilities of residents grow in complexity over the course of training.
At our institution, some faculty and residents in the ED shared concerns that our orientation process was suboptimal because it did not specifically address the unique personnel and patient care issues involved in caring for acutely ill and injured children. Before our work, the orientation to the ED rotation consisted of brief, informal introductions before the PL1 and PL2 years, as well as materials hosted on the hospital Intranet. Residents voiced concerns that they felt underprepared for the rotation, especially in the resuscitation area, an environment unique to the ED and likely not reproduced in any other area of residency. These concerns prompted an internally derived survey-based, anonymous needs assessment, which was voluntarily completed during a residents-only meeting in the spring of 2012 by a limited number (55 of the nearly 200 residents) of trainees, all of whom were rising PL2 and PL3 residents. Key ED faculty and the chief residents developed the survey, which consisted of Likert-response items and open-ended answer questions. Almost three-fourths (69%, 34 of 55) of respondents indicated that they had an inadequate orientation to the resuscitation area. Only 14% (8 of 55) were aware of the criteria on which they were being evaluated while rotating in the ED. Overall, 71% (39 of 55) indicated that the existing ED orientation was ineffective. On the basis of the needs assessment, we identified numerous challenges to orienting residents to the ED, including the following:
A large number of resident trainees each month, working a variable shift schedule.
An unfamiliar clinical environment.
A wide spectrum of illness and injury.
An expectation of frequent patient turnover within each shift, as well as receiving and giving sign-outs on patients whose disposition is undetermined at the time of sign-out.
Highly variable care teams, with more than 40 faculty, 11 ED fellows, and numerous allied health professionals.
Unique features of the ED rotation, including the evaluation and management of patients in the resuscitation area.
Development and Implementation of New Orientation
Given the aforementioned challenges, we determined that a Web-based orientation would likely best serve our trainees' needs. Online sources of education can be leveraged in order to supplement the traditional curriculum, engage residents, and embed new learning before their arrival on an upcoming rotation. Online tools can provide rich learning experiences that allow learners to review materials on their own time, with straightforward methods to record access.
Thus, in anticipation of the 2012–2013 academic year, we used existing ED rotation goals and objectives and information from the survey to develop and implement an online orientation program to the ED rotation for residents. This program included:
A general video introduction to the ED.
A specific video introduction for the ED resuscitation area that included information on care team composition, resident role and responsibilities, and the correct performance of the rapid cardiopulmonary assessment.
A detailed presentation on the Pediatric Milestones–based ED rotation evaluation.
The video-based components were filmed in the ED, with faculty and staff in various roles. Video footage was edited on Final Cut Pro X (Apple, Cupertino, Calif) and the final general and resuscitation area videos uploaded to Vimeo (White Plains, NY). The orientation videos were password protected and embedded on the orientation Web site, hosted via WordPress (WordPress, San Francisco, Calif) on a domain owned by the principal investigator (http://www.pemcincinnati.com/orientation). The entire orientation took 25 to 30 minutes to review. It was critical to assure that residents were held accountable for accessing the orientation. As such, we decreed that residents who did not attest to viewing the orientation would not be permitted to participate in care in the resuscitation area of the ED.
Beginning with the July 2012 rotation block, an e-mail was sent to all PL1 and PL2 categorical pediatric residents scheduled to rotate through the ED, both 7 days and 48 hours before the start of their rotation. We embedded an e-mail form on the orientation Web site that allowed residents to attest to viewing of the orientation materials. We chose only the PL1 and PL2 residents because PL1s were entering the PED for the first time, and the PL2s would participate in the care of patients in the resuscitation area. PL3 and above residents were not included in the e-mail reminders because they had already rotated through the PED several times and had experience in the resuscitation area. Because of differing goals and objectives, visiting residents (family medicine) and emergency medicine residents were also excluded.
Measuring Impact of Orientation
Record of e-mail attestation from the Web site served as a proxy measure for resident access of the orientation materials. We assessed the impact of the orientation in 2 ways. First, research coordinators approached PL1s and PL2s in the ED and had them answer 10 single-answer questions related to principal concepts featured in the orientation, such as personnel roles and responsibilities, trauma team activation criteria, and ED-specific patient care processes. The 10 questions were randomly selected and sequenced from a bank of 25, which were developed by the study authors and piloted by the chief residents. The research coordinators approached residents in private during the first or second ED shift of their rotation, performed the knowledge testing, and provided answers for any incorrect responses. These pop quizzes were conducted between January and June of 2013. Using analysis of variance, we assessed for an association between the number of prior PED rotations and total score, suspecting that new PL1 residents who viewed the orientation would have similar knowledge to PL2 residents who had rotated previously. Second, after year 1 of our orientation implementation, we again surveyed the residents. We compared survey responses by chi-square test.
Results of Intervention
During the 2012–2013 academic year, 94% of eligible residents (93 of 99) attested via e-mail that they viewed the orientation Web site before at least one ED rotation, including 96% of PL1s (50 of 52) and 91.5% of PL2s (43 of 47). Eighty-eight percent (87 of 99) of the residents rotated through the ED more than once; 31% (30 of 99) attested that they viewed the orientation before all of their ED rotations, including 67% (35 of 52) of PL1s and 17% (8 of 47) of PL2s.
More than half (64%, 63 of 99) of the residents were approached for the pop quizzes (25 of 52 PL1s and 38 of 47 PL2s). The mean number of correct answers to knowledge questions by all residents was 7.2 of 10 (SD 2.1). The mean number of correct answers by PL2s (8.1 of 10, SD 1.7) was greater than that of PL1s (5.8 of 10, SD 2.1); P < .0001. The mean number of correct responses for residents in their first ED rotation was lower compared to those who had previously spent 1 or more months on the rotation (P = .001; Fig).
Fifty-four (58%) of 93 residents who attested to viewing the orientation materials responded to our postsurvey, which asked the same questions as the presurvey. The postsurvey participants (PL1 and PL2 trainees) again provided their responses in an anonymous fashion. Furthermore, the pre- and postsurvey respondent populations were not identical. Sixty-five percent (35 of 54) indicated that they were aware of the specific criteria on which they were being evaluated, compared to 14% historically (8 of 56), (P < .0001). All (54 of 54) survey respondents felt that they had received an appropriate orientation to the resuscitation area compared to 32% (18 of 56) historically (P < .0001).
In its first year, a Web-based and multimedia orientation to our PED was associated with improvements in resident exposure to orientation materials, resident knowledge of rotation expectations/evaluation criteria, and resident perception of the adequacy of the ED orientation. Since its inception, the orientation has been updated in an iterative fashion in response to changes at the institution.
As noted earlier, most studies of residency rotation orientation effectiveness have been limited to survey-based methodology and do not address the process from start to finish, in contrast to our orientation program development process
Multiple orientation curricula can be found online through repositories such as MedEdPORTAL, but they require modifications for local adaptation. Although many such curricula may include evaluation tools, data allowing an interpretation of the effects of implementation are lacking.
There are limitations to our study and the interpretation of our findings. First, we were limited in our ability to assure that all residents watched the entirety of the videos. Content hosted on Vimeo or other online services lack detailed tracking beyond rudimentary view counts. Thus, future modifications will include content stratified by resident level and designed with adaptive modular platforms (such as Adobe Captivate) in order to better engage learners and track compliance. Second, we were not able to capture all of the available trainees in the knowledge assessment. However, we were reassured by the pop quiz scores of the PL1s that had never before rotated in the ED. Their results suggest that rotation-specific knowledge was acquired via exposure to the online orientation. It is important to note that residents with greater prior ED experience answered more questions correctly on average. We suspect that cumulative experience in the ED accounted for this measured difference. Third, we did not utilize a pretest–posttest methodology to establish the baseline knowledge of PL1s before the online orientation. Future studies should consider incorporating a pretest or use of a control group (ie, outside rotators such as family practice residents) to better assess the effectiveness of the orientation. Finally, it is important to note that our experience is an investigation of the impact of a pediatric residency rotation orientation at a single institution and these results may not be generalizable to all pediatric institutions or ED rotations. Though the content of our orientation was highly specific to our institution, the methodology is easily replicable and the process to create a rotation specific video can be accomplished affordably via commercially available software and equipment.
Despite these limitations, several important lessons were learned that will inform our future endeavors and be of value to fellow educators who wish to develop similar orientation programs:
Involve key stakeholders in the process, including faculty (the rotation director), program leadership, ED care team personnel, chief residents, and the residents themselves.
Conduct a thorough needs assessment in order to identify your orientation's deficiencies, and plan to assess the impact of your new orientation.
If all of the trainees can't be in the same room at the same time, leverage the power of online tools and multimedia solutions.
Implement a system of accountability by assuring that residents know that the orientation is valuable and meaningful, and their participation is part of the professional expectations for the rotation.
It is unlikely that residency work hours will increase in the foreseeable future. However, trainee concerns about transitions to new rotations will almost certainly persist. Developing methodologically sound plans for orientation to clinical rotations, in formats that are easily available and comprehensive, may mitigate some of the negative impact of having new residents each month in different areas of the hospital. Future study is needed to assess whether our, or similar online orientations, can reduce the impact of the July effect—and for that matter the September effect or even the April effect.
Fewer hours for doctors in training leading to more mistakes.