Advertisement

Performance of International Medical Graduates in Pediatric Residency: A Study of Peer and Faculty Perceptions

      Abstract

      Background

      International medical graduates (IMGs) constitute approximately 25% of the US pediatric workforce. Their recruitment into US residency training raises concerns regarding their competence, although this has not been formally studied. Cincinnati Children's Hospital has systematically recruited IMGs over the past 16 years. This study evaluates perceptions of IMG performance by faculty and US graduate (USG) peers.

      Methods

      We surveyed IMG, USG, and faculty groups, including current and former trainees, assessing perceived IMG performance compared with that of USGs in terms of clinical knowledge/skills, resource utilization, communication, public health knowledge and efficiency, and overall impact on the program.

      Results

      Overall perceived performance was within 1 standard deviation of expected USG performance. IMGs outperformed USGs in clinical knowledge/skills and resource utilization but underperformed in communication, public health knowledge, and efficiency. Significant differences were noted in communication with patients and public health knowledge; IMGs ranked their performance significantly lower than USGs/faculty ranked their performance. Overall impact was perceived positively, including an increased interest in global health in among USGs.

      Conclusions

      Carefully recruited IMGs are perceived to perform nearly equal to their USG peers, and their presence is perceived as positive to a major pediatric residency program. Specific domains for educational interventions are identified for programs wishing to expand IMG recruitment.

      Keywords

      What's New
      As an important proportion of the pediatric workforce, recruitment of international medical graduates into residency training requires better understanding of their potential, and shortcomings, beyond standardized test scores. Domains for targeted improvement and education for interested programs are identified.
      International medical graduates (IMGs) represent approximately 25% of all pediatricians and exceed 27% of pediatric subspecialists in the United States,
      • Ahmed AA
      • Hwang WT
      • Thomas Jr, CR
      • et al
      International medical graduates in the US physician workforce and graduate medical education: current and historical trends.
      • Freed GL
      • Moran LM
      • Van KD
      • et al
      Research Advisory Committee of the American Board of Pediatrics. Current workforce of general pediatricians in the United States.
      and they play an essential role in meeting the needs of underserved pediatric populations.
      • Umoren R
      • Rybas N
      • Frintner MP
      The contribution of childhood and medical school location to the career paths of graduating pediatrics residents.
      Because IMGs are trained in medical schools not accredited by the Liaison Committee on Medical Education, their recruitment into US residency programs may raise concerns about their competence, as well as the effect on programs’ reputations.
      • Go PH
      • Klaassen Z
      • Chamberlain RS
      An ERAS-based survey evaluating demographics, United States Medical Licensing Examination performance, and research experience between American medical graduates and United States citizen international medical graduates: is the bar higher on the continent?.
      Although some literature suggests that performance in residency and career outcomes are linked to United States Medical Licensing Examination (USMLE) scores,
      • Norcini JJ
      • Boulet JR
      • Opalek A
      • et al
      The relationship between licensing examination performance and the outcomes of care by international medical school graduates.
      concerns exist regarding the overall knowledge and skill set of IMGs on admission to US residency programs.
      • Osta AD
      • Barnes MM
      • Pessagno R
      • et al
      Acculturation needs of pediatric international medical graduates: a qualitative study.
      • Zulla R
      • Baerlocher MO
      • Verma S
      International medical graduates (IMGs) needs assessment study: comparison between current IMG trainees and program directors.
      Careful IMG selection may result in well-adapted and functional trainees, but little information is available regarding the perceived performance of selected IMG cohorts in pediatric residency training. Despite recommendations by the American Academy of Pediatrics for favorable migratory conditions, recruitment among large academic programs remains low, and IMGs often work in non–university-based programs.
      • Basco WT
      • Rimsza ME,
      Committee on Pediatric Workforce et al
      Pediatrician workforce policy statement.
      In 2001, the pediatric residency training program at Cincinnati Children's Hospital Medical Center (CCHMC) began intentionally recruiting top IMGs. This was done to increase diversity in our workplace and solidify our global health opportunities (educational and overseas rotations). This practice also laid a foundation for our future Global Health Residency Track. At the time, global health was often unidirectional (ie, physicians from high-income countries going to low- and middle-income countries) with limited long-term impact on infrastructure or practice policies. Therefore, a part of our initial plan was to develop a bidirectional aspect to our global health work by introducing top IMGs with diverse backgrounds to our program, with the idea of maintaining connections to their countries of origin and thereby develop ongoing relationships with those programs. It was also hoped that US graduate (USG) residents would benefit from sustained interactions with individuals of different backgrounds and nationalities, experiencing different approaches to patient care and training. Currently, CCHMC annually recruits 40 individuals into categorical pediatrics and 16 to 18 individuals into combined programs; as many as 12% (per class) are carefully selected IMG applicants deemed capable of delivering medical care at the level of USGs. To evaluate our institutional experience of IMG inclusion, we solicited the perceptions of USGs and faculty regarding IMGs’ performance during training and their overall impact on the residency program. Domains assessed included elements of the Accreditation Council for Graduate Medical Education milestones and proxies to these constructs, which included concerns raised in the medical literature regarding IMGs’ performance often encountered in the authors’ experience.

      Methods

      This cross-sectional descriptive study used a custom-designed survey approved by CCHMC's institutional review board. Three participant groups were included. Group 1 included all 40 IMGs either enrolled in 1 of CCHMC's residency programs during the study (n = 11) or who graduated between 2001 and 2014 (n = 29). Group 2 comprised all 133 enrolled postgraduate year 2 (PGY-2) through PGY-5 USGs and all 56 of the 2014 USG graduates. Interns were excluded to ensure that study participants had at least 1 full year of exposure to their IMG peers. Group 3 included all program faculty from 3 hospital divisions (34 from Emergency Medicine, 36 from Hospital Medicine, and 22 from Neonatology). These were selected because all trainees rotate for at least 1 month in each division during each of their initial 3 years of residency, which provided more frequent interactions with these faculty.

      IMG Recruitment

      IMG applicants are screened by CCHMC using the following criteria: 1) a USMLE score above the 75th percentile, 2) a curriculum vitae demonstrating sustained research/scholarly and/or community advocacy activities, and 3) awards for academic performance or humanism. These rigorous standards are similar to those used to select USGs for interviews. Excelling individuals are extended interview invitations. A process including at least 5 interviews with program directors, chief residents, and department chairs provides the opportunity to gauge the IMG’s fit for the program and an overview of his or her language proficiency. Approximately 25 of 800 IMG applicants are selected annually for interviews.

      Survey Design

      The investigators (A.J.G. and C.J.S.) developed the survey and pretested it for clarity and ease of completion with faculty and interns outside of the participant groups. The web-based survey was designed and administered using Qualtrics (Provo, Utah), and it was composed primarily of multiple-choice questions using ordinal scales. Open-ended questions that allowed for elaboration and clarification were included as well. All groups were asked to gauge the perceived performance of IMGs compared with USGs in various domains: clinical knowledge (understanding of disease processes and physiology), clinical skills (ability to integrate knowledge into practice during examination and procedures), resource utilization, communication with patients and staff, public health knowledge, and overall efficiency (ability to integrate knowledge and skills into a systems-based practice with minimal disruption). Additional information requested from IMGs included USMLE scores, fellowship status and employment, and clinical/research collaborations with the home country. Self-assessment—an important part of the program's trainee evaluation—was included and provided additional viewpoints in data analysis. Additional information requested from USGs and faculty included assessing the overall positive or negative impact of IMGs on the residency program.
      Respondents received an e-mail invitation with a project description, assurance of anonymity, a statement on voluntary participation, and a link to the online survey. Completion of the survey was considered to represent consent to participate. Periodic reminders for survey completion were sent to participants. The survey was administered over a 3-week period.

      Data Analysis

      Descriptive statistics were calculated for all variables. Mean ratings were stratified, with possible scores ranging between –2 (“IMGs perform much less effectively”) and 2 (“IMGs perform much more effectively”). In addition, aggregate results for each category from USGs and faculty were analyzed to represent a global perspective of IMG performance and a comparative rating (vs USGs), considered the benchmark. Mean differences in perceptions among participant groups on the performance of IMGs versus USGs were examined with 1-way between-subjects ANOVA. A significant F value for a given category indicated a statistically significant difference in performance perceptions among groups. Pairwise tests were then conducted using the Scheffé post hoc criterion for significance to identify which groups differed. Two investigators (A.J.G. and C.J.S.) analyzed the content of open-ended responses to identify the main theme(s). If a consensus on a specific theme was not reached, a third investigator (C.L.A. or M.R.F.) reviewed the disputed question to reach a final consensus.

      Results

      The survey response rate was 60% (193 of 321). Overall, 83% (33 of 40) of IMGs, 52% (98 of 189) of USGs, and 60% (55 of 92) of faculty responded. All responding IMGs were non–US citizens. Demographic data for all groups and career outcomes for IMGs are displayed in the Table.
      TableDemographic Information for Respondent Groups
      Total% Response
      a. Respondent Groups
      IMGs
       Current1091
       Graduate2379
      USGs
       Current7556
       Graduate2341
      Faculty
       EM2574
       HM1644
       NICU1464
      b. IMG Demographic Data
      Region of origin
       Africa13
       Asia
      Excludes the Middle East.
      1030
       Europe515
       Americas1545
       Middle East26
      Mean USMLE scores
       Step 1243.3
       Step 2253.8
      c. Career Outcomes
      ABP initial certification pass rate (%)
       CCHMC IMGs100
       CCHMC USGs95
      Career paths for CCHMC graduates (%)
      USGsIMGs
       Primary care4717
       Fellowship5383
      IMGs indicates international medical graduates; USGs, US graduates; EM, emergency medicine; HM, hospital medicine; NICU, neonatology; USMLE, United States Medical Licensing Examination; APB, American Board of Pediatrics; and CCHMC, Cincinnati Children's Hospital Medical Center.
      low asterisk Excludes the Middle East.

      Performance of IMGs Compared With USGs

      Mean ratings among IMGs, USGs, and faculty regarding the average performance of IMGs compared with USGs in different areas are shown in the Figure. ANOVA results indicated a significant difference in perceptions among the participant groups in “communication with patients” (F(2,178) = 3.86; P = .023). Post hoc analyses revealed that IMGs’ ratings of their own performance versus USG performance (mean, –0.73 ± 0.88) were lower than USGs’ ratings of IMG performance (mean, –0.37 ± 0.53). Results also indicated a significant difference in perceptions among the participant groups in “public health knowledge” (F(2,178) = 13.41; P < .001). Here, IMGs’ ratings of their own comparative performance (mean, –0.58 ± 0.87) were statistically different from those of USGs, who rated IMGs’ performance as more effective than their own (mean, 0.13 ± 0.53). Faculty rated the performance between groups in this area as equal (mean, 0.00 ± 0.81). No additional statistical differences were uncovered in other categories.
      Fig 1
      FigureMean ratings of international medical graduate (IMG) performance compared with US graduate (USG) performance per category and evaluating group. Here, 2 represents that IMGs perform much more effectively; 1, IMGs perform somewhat more effectively; 0, equal USG/IMG performance; –1, IMGs perform somewhat less effectively; and –2, IMGs perform much less effectively.

      Overall Impact of IMGs

      In total, 73% of faculty and 89% of USGs rated the overall presence of IMGs as “mostly positive” or “somewhat positive.” Some 24% of faculty and 12% of USGs were neutral, and 4% of faculty perceived the impact to be “somewhat negative.” Qualitative detail regarding IMGs’ impact was also obtained. Major positive themes included “[cultural] diversity” (49% of responses) and “[different/new] experience” (37%). Negative themes included “[difficulties with] communication” (18%) and “[difficulties with] adjustment” (12%). Some 61% of USGs and faculty responded “none” (or similar) regarding a negative impact on the program, even when not required to provide a response to this question.
      Finally, 32% of USGs responded that the presence of IMGs “increased” their interest in global health, 68% responded that it had “no impact,” and none responded that it “decreased” their interest.

      Discussion

      IMGs’ overall performance was perceived by USGs and faculty as equal, or nearly equal, to USGs’ performance throughout residency (mean, –0.05 ± 0.66). This is supported by consistent success in the American Board of Pediatrics examination (Table).
      American Board of Pediatrics
      Initial certifying examination first-time taker passing rates.
      In addition, IMGs overwhelmingly seek and successfully enter subspecialty training. We believe that this may be owing to an intent to train beyond what is available in their home countries whether or not they intend to eventually return or contribute there.
      Some domains showed subtle perceived variations in IMG contributions and shortcomings. Clinical knowledge (ie, recognition of etiology and pathophysiology, and empirical aspects of care) and skills (ie, recognition of signs/symptoms and procedural dexterity) were perceived by USGs and faculty to be superior in the IMG group (mean, 0.20 ± 0.66). This may be related to more extensive clinical exposure preceding their residency admission, which many IMGs gain in medical school (4–5 clinical years), or to previous postgraduate training in pediatrics in their home countries. In our cohort, 24% of IMGs had trained or worked clinically in pediatrics before their residency training at CCHMC. These environments often have fewer staff and ancillary services, exposing IMGs to more opportunities for independent clinical decision making and performance of procedures. These contributions are recognized by peers and faculty as “added experience” to the training program. These observations also match perceptions noted in surgical specialties with extensive research on IMG performance.
      • Horvath K
      • Coluccio G
      • Foy H
      • Pellegrini C
      A program for successful integration of international medical graduates (IMGs) into US surgical residency training.
      Communication remained the major theme of perceived IMG underperformance (mean, –0.36 ± 0.57), also recognized as a disadvantage by IMGs (mean, –0.61 ± 0.78). We suspect that this is owing to the rapid acculturation expected of IMGs at the start of training. The literature suggests that this is a major concern when programs consider recruiting IMGs,
      • Zulla R
      • Baerlocher MO
      • Verma S
      International medical graduates (IMGs) needs assessment study: comparison between current IMG trainees and program directors.
      • Jain P
      • Krieger JL
      Moving beyond the language barrier: the communication strategies used by international medical graduates in intercultural medical encounters.
      and multiple studies recommend initial cultural and linguistic orientation.
      • Osta AD
      • Barnes MM
      • Pessagno R
      • et al
      Acculturation needs of pediatric international medical graduates: a qualitative study.
      • Lineberry M
      • Osta A
      • Barnes M
      • et al
      Educational interventions for international medical graduates: a review and agenda.
      ,
      • Dorgan KA
      • Lang F
      • Floyd M
      • et al
      International medical graduate–patient communication: a qualitative analysis of perceived barriers.
      In addition, IMGs’ own perceived underperformance in public health knowledge may suggest the need for more extensive linguistic and cultural orientation, as well as socioeconomic and health systems orientation, which may enhance IMGs’ confidence.
      • Rao A
      • Freed CR
      • Trimm RF
      International and American medical graduates in a US pediatric residency program: a qualitative study about challenges during post-graduate year 1.
      Our experience suggests that IMGs have extremely heterogeneous cultural backgrounds, meaning that such orientation initiatives must be carefully tailored. Despite a rigorous interview process that also assesses language fluency, these results demonstrate the need for improvement in cultural acclimation.
      Underperformance of IMGs in efficiency (mean, –0.19 ± 0.61) was also identified. Our experience suggests that this was owing to differences between the systems-based practices in the United States and countries in which IMGs initially trained (mainly low- and middle-income countries). We believe that IMGs adapt more gradually to protocolized environments with evidence-based resources available, but these differences disappear through their training.
      Finally, the results highlight the overall perceived value and contribution of IMGs in diversifying the training environment and the institutional investment in global health. CCHMC expects IMGs to contribute locally and globally: 24% had clinical rotations and 15% had research collaborations in their home countries during residency. Among alumni, 35% continued to be involved in research and 26% worked clinically in their home countries. We believe that these activities help palliate the concerns of “brain drain” from low- and middle-income countries and also enhance CCHMC’s global presence.
      This study has multiple limitations. It is a single-institution study with a small cohort of carefully selected IMGs. There is an inherent selection bias owing to our institutional selection process. In addition, exposure to the included IMGs over time, and their skill sets, may have varied among faculty across our institution. This in turn risks a recall bias among respondents. We attempted to diminish this by surveying divisions in which many staff were exposed to IMGs across the years of training, with numerous opportunities for clinical decision making and performance of procedures for residents, allowing for better assessment of performance. In addition, we were unable to obtain information from contemporary cohorts of USGs (2001–2014), which may affect variability in USG perceptions over time.
      Despite our experience-based recognition of progression during residency in skills and communication among IMGs, we did not comprehensively investigate the differences among IMGs at different stages of training. If done in the future, this could allow for a better understanding of how the gaps between IMGs and USGs narrow over the course of training.

      Conclusions

      The overall performance of a select cohort of IMGs with outstanding candidate profiles and interviews is longitudinally comparable to that of USGs, as perceived by USGs’ peers and supervisors. To our knowledge, this is the first evaluation of peer/faculty perceptions and outcomes of a cohort of IMGs trained in a US residency program.
      Despite obstacles to training and varying immigration circumstances, IMGs are important for a diversified workforce to meet expanding health care needs. Previous American Academy of Pediatrics statements have recognized the need for IMGs
      • Basco WT
      • Rimsza ME,
      Committee on Pediatric Workforce et al
      Pediatrician workforce policy statement.
      American Academy of Pediatrics Committee on Pediatric Workforce
      Pediatrician workforce statement.
      and the services they provide to underserved populations.
      • Umoren R
      • Rybas N
      • Frintner MP
      The contribution of childhood and medical school location to the career paths of graduating pediatrics residents.
      ,
      • Richards MR
      • Chou CF
      • Lo Sasso AT
      Importing medicine: a look at citizenship and immigration status for graduating residents in New York state from 1998 to 2007.
      • Thompson MJ
      • Hagopian A
      • Fordyce M
      • et al
      Do international medical graduates (IMGs) “fill the gap” in rural primary care in the United States? A national study.
      • Howard DL
      • Bunch CD
      • Mundia WO
      • et al.
      Comparing United States versus international medical school graduate physicians who serve African American and white elderly.
      By providing outstanding IMGs the opportunity to train in highly academic settings, these competent professionals will contribute to a diversified pediatric workforce, providing care to the underserved at home and abroad.

      References

        • Ahmed AA
        • Hwang WT
        • Thomas Jr, CR
        • et al
        International medical graduates in the US physician workforce and graduate medical education: current and historical trends.
        J Grad Med Educ. 2018; 10: 214-218
        • Freed GL
        • Moran LM
        • Van KD
        • et al
        Research Advisory Committee of the American Board of Pediatrics. Current workforce of general pediatricians in the United States.
        Pediatrics. 2016; 137:e20154242
        • Umoren R
        • Rybas N
        • Frintner MP
        The contribution of childhood and medical school location to the career paths of graduating pediatrics residents.
        Acad Pediatr. 2015; 15: 557-564
        • Go PH
        • Klaassen Z
        • Chamberlain RS
        An ERAS-based survey evaluating demographics, United States Medical Licensing Examination performance, and research experience between American medical graduates and United States citizen international medical graduates: is the bar higher on the continent?.
        J Surg Educ. 2012; 69: 143-148
        • Norcini JJ
        • Boulet JR
        • Opalek A
        • et al
        The relationship between licensing examination performance and the outcomes of care by international medical school graduates.
        Acad Med. 2014; 89: 1157-1162
        • Osta AD
        • Barnes MM
        • Pessagno R
        • et al
        Acculturation needs of pediatric international medical graduates: a qualitative study.
        Teach Learn Med. 2017; 29: 143-152
        • Rao A
        • Freed CR
        • Trimm RF
        International and American medical graduates in a US pediatric residency program: a qualitative study about challenges during post-graduate year 1.
        Med Teach. 2013; 35: 815-819
        • Zulla R
        • Baerlocher MO
        • Verma S
        International medical graduates (IMGs) needs assessment study: comparison between current IMG trainees and program directors.
        BMC Med Educ. 2008; 8: (42)
        • Basco WT
        • Rimsza ME,
        • Committee on Pediatric Workforce et al
        Pediatrician workforce policy statement.
        Pediatrics. 2013; 132: 390-397
        • American Board of Pediatrics
        Initial certifying examination first-time taker passing rates.
        (Available at:) (Accessed July 1, 2017)
        • Horvath K
        • Coluccio G
        • Foy H
        • Pellegrini C
        A program for successful integration of international medical graduates (IMGs) into US surgical residency training.
        Curr Surg. 2004; 61: 492-498
        • Jain P
        • Krieger JL
        Moving beyond the language barrier: the communication strategies used by international medical graduates in intercultural medical encounters.
        Patient Educ Couns. 2011; 84: 98-104
        • Lineberry M
        • Osta A
        • Barnes M
        • et al
        Educational interventions for international medical graduates: a review and agenda.
        Med Educ. 2015; 49: 863-879
        • Dorgan KA
        • Lang F
        • Floyd M
        • et al
        International medical graduate–patient communication: a qualitative analysis of perceived barriers.
        Acad Med. 2009; 84: 1567-1575
        • American Academy of Pediatrics Committee on Pediatric Workforce
        Pediatrician workforce statement.
        Pediatrics. 2005; 116: 263-269
        • Richards MR
        • Chou CF
        • Lo Sasso AT
        Importing medicine: a look at citizenship and immigration status for graduating residents in New York state from 1998 to 2007.
        Med Care Res Rev. 2009; 66: 472-485
        • Thompson MJ
        • Hagopian A
        • Fordyce M
        • et al
        Do international medical graduates (IMGs) “fill the gap” in rural primary care in the United States? A national study.
        J Rural Health. 2009; 25: 124-134
        • Howard DL
        • Bunch CD
        • Mundia WO
        • et al.
        Comparing United States versus international medical school graduate physicians who serve African American and white elderly.
        Health Serv Res. 2006; 41: 2155-2181