Advertisement

Parent Involvement in the Pediatric Resident Applicant Interview

Published:March 19, 2018DOI:https://doi.org/10.1016/j.acap.2018.02.017

      Abstract

      Background

      Parents and patients are actively involved in the clinical learning environment, yet scant literature exists about their involvement in the residency interview process. We aimed to pilot a process of including parents in resident interviews and to determine its value.

      Methods

      During the 2016-17 residency interview cycle, 22 parent volunteers, blinded to applicant credentials, conducted brief structured interviews with 118 applicants. We then surveyed all parents and applicants with the use of mixed methods: descriptive statistics to analyze 5-point Likert-type-scale responses, and content analysis to identify themes from open-ended questions. Although parent interviews were not used in ranking, we later compared final composite parent interview scores (1–10, with 10 being high) among the final rank and match list candidates.

      Results

      Response rates were high for both groups (parents 100%; resident applicants 98.3%). Parents felt strongly positive about meeting applicants (mean ± SD, 5.00 ± 0.00), the value of parent participation (4.90 ± 0.30), and their own experience (4.95 ± 0.22). Applicants felt positive about meeting parents (4.45 ± 0.70), the value of parent participation (3.92 ± 0.84), and their own experience (4.51 ± 0.67). Several themes emerged from both groups, with the most salient parent themes including the value of patient-centered perspectives and appreciation and joy of meeting applicants. Parent interview scores correlated with the final match list, with matched applicants scoring higher (9.08 vs 8.51; P = .05).

      Conclusions

      Involvement of parents in the pediatric residency interview process is achievable, is perceived positively by parents and applicants, and may provide valuable perspectives for consideration in residency selection.
      Residency programs use faculty and residents to interview medical student applicants to assess their future performance as resident trainees. An annual survey from the National Resident Matching Program (NRMP) lists qualities that pediatric applicants cite as important factors in selecting residency programs for application and ranking. Impressions of the interview day experience and curricular quality are ranked highly, yet impressions of the emphasis on patient-centered care are unknown.
      National Resident Matching Program, Data Release, and Research Committee.
      Parent input and feedback are commonly included in several assessments
      • Schwartz A.
      • Margolis M.J.
      • Multerer S.
      • et al.
      A multi-source feedback tool for measuring a subset of Pediatrics Milestones.
      (eg, simulations, global multisource evaluations, and Observed Structured Clinical Exam exercises) for pediatric learners and remain important for ongoing professional development.
      • Sargeant J.
      • Bruce D.
      • Campbell C.M.
      Practicing physicians' needs for assessment and feedback as part of professional development.
      Acquiring clinical knowledge, skills, and attitudes to provide patient and family-centered care is essential for all learners, and this development involves active resident engagement with patients and their families.
      • Levinson W.
      • Lesser C.S.
      • Epstein R.M.
      Developing physician communication skills for patient-centered care.
      Interview interactions are used to assess applicant fit and preparation for the culture and clinical learning environment.
      • Stephenson-Famy A.
      • Houmard B.S.
      • Oberoi S.
      • et al.
      Use of the interview in resident candidate selection: a review of the literature.
      Although faculty and resident involvement in this process is critical, that may not completely account for the input of patients' families, who also play an important role in the assessment and development of learners.
      • Brinkman W.B.
      • Geraghty S.R.
      • Lanphear B.P.
      • et al.
      Evaluation of resident communication skills and professionalism: a matter of perspective?.
      • O'Keefe M.
      Should parents assess the interpersonal skills of doctors who treat their children? A literature review.
      Given this backdrop, interactions between applicants and parents may offer insight into learner fit and preparation for the learning environment. Unlike other disciplines, pediatric residents interact with parents of patients at nearly every clinical encounter. The interactions also present an inclusion opportunity for these important stakeholders. To our knowledge, parents have not commonly participated in the residency interview and selection process. The present pilot study aimed to evaluate the impact of including parents in the applicant interview process, to describe parent and applicant perceptions of the value of this involvement, and to compare the parent rating to the programmatic rank.

      Methods

      This mixed-methods prospective study was conducted at the Kaiser Permanente Oakland Medical Center, an academic community hospital.

       Interview Tool

      We aimed to create an easy-to-use set of questions for parents to gauge an applicant's joy in practice, communication skills, and ability to be trusted. Drawing from existing literature,
      • Hicks P.J.
      • Englander R.
      • Schumacher D.J.
      • et al.
      Pediatrics milestone project: next steps toward meaningful outcomes assessment.
      • Makoul G.
      • Krupat E.
      • Chang C.H.
      Measuring patient views of physician communication skills: development and testing of the Communication Assessment Tool.
      • Min A.A.
      • Leetch A.
      • Nuño T.
      • et al.
      How well will you FIT? Use of a modified MMI to assess applicants' compatibility with an emergency medicine residency program.
      • Association of American Medical Colleges
      Interviewing for residency: frequently asked interview questions.
      we chose 3 questions for the structured portion of the interview: 1) “Why are you excited about pediatrics?” 2) “What makes you an effective listener in the patient care environment?” and 3) “What makes you a trustworthy team member?

       Survey Development

      When they were finished interviewing the applicant, parents were asked to answer 2 10-point Likert-type-scale questions, assessing their own comfort in developing a relationship with the prospective resident (1 = very uncomfortable; 10 = very comfortable) and their confidence in the prospective resident's ability as an effective care team member for their child (1 = not confident; 10 = very confident). Parent assessments were not used during the evaluation or ranking of applicants. In addition, both parents and applicants were given an anonymous voluntary paper-based survey of 5-point Likert-type-scale questions (1 = strongly negative; 5 = strongly positive) to rate their general opinion, perception of value, and personal experience regarding parents meeting applicants.

       Parent Selection

      Based on staff and faculty referrals of known parents, a convenience sample of English-speaking racially and ethnically diverse parents with children of varying ages were recruited via telephone and e-mail. Once recruited, each parent participated in a 10-minute telephone meeting with the same faculty member who oriented them to the residency application, interview, and matching process, as well as the 3-question interview tool.

       Interview Logistics

      All applicants were interviewed by a parent for 15 minutes on interview days for the Kaiser Permanente Northern California Pediatric Residency Training Program during the fall/winter of 2016–2017. Parents were aware of the applicant's name and their status as a medical student learner applying for a residency position and blinded to other information, including medical school, academic performance, and previous experiences. Applicants were aware that they would meet parents but were uninformed of interview questions or format.

       Analysis

      We used descriptive statistics to analyze parent and applicant survey responses to close-ended categoric questions and content analysis to identify themes in the responses to open-ended questions. Initial themes were identified by 1 author (A.D.); 2 additional authors (E.M. and R.C.) then reviewed the open-ended responses with the suggested themes. There was complete agreement about the thematic categories and which responses fit into each theme, ensuring high intercoder reliability of the textual data. Analyses of variance (ANOVAs) were used to compare mean responses according to themes. To determine possible congruence with rank or match list, ANOVAs were also used to compare the parental mean composite scores according to rank list position, broken down into tertiles and dichotomously based on top 10 rank status, and according to matched status. This study was reviewed and considered to be exempt by the Kaiser Foundation's Institutional Review Board.

      Results

      Twenty-two of 25 parent volunteers agreed to participate. All 22 parents (100%) and 116 (98.3%) of the 118 applicants completed the survey. Parent perceptions were strongly positive about meeting applicants (mean ± SD; 5.00 ± 0.00), the value of parent participation (4.90 ± 0.30), and their own experience (4.95 ± 0.22). Applicants overall were strongly positive about the concept of parent involvement in the interview process (4.45 ± 0.70), positive about the value of parent participation in their selection of a residency program (3.92 ± 0.84), and strongly positive about their own experience meeting with parents on their interview day (4.51 ± 0.67).
      Applicants' views of meeting parents, parent participation, and their own interview day experience (questions 1–3 combined) were overall positive (4.30 ± 0.60). Similar significant variations in mean values were found for each question independently; these views were significantly less positive if they provided a neutral or negative comment (n = 18) or both a neutral or negative and positive comment (n = 14; 4.05 ± 0.64), versus only a positive comment (n = 59; 4.56 ± 0.44; P < .0001).
      Three overarching themes emerged from parent comments: 1) value of patient-centered perspectives (n = 10); 2) appreciation for parent involvement (n = 8); and 3) joy of meeting applicants (n = 8; Table). Eight themes emerged from applicant comments, of which 4 were deemed to have positive connotations and 4 to have neutral or negative connotations. The 4 positive themes were: 1) noting the innovative nature of the process (n = 25); 2) enjoyment (n = 28); 3) noting the patient-centered nature of the process (n = 37); and 4) perceiving the experience as having high value (n = 37). The 4 neutral or negative themes included: 1) wanting more parent perspective (n = 12); 2) wanting more time or questions (n = 9); 3) feeling unfamiliar with the questions and/or format (n = 11); and 4) perceiving the experience as having low value (n = 5). The Table summarizes these themes with representative quotes.
      TableParent and Applicant Open-Ended Survey Responses
      ThemenRepresentative Comments
      Parent themes (positive)
      Value of patient-centered perspectives10• “families are an integral piece”
      • “value in including parent perspective into providers that will be caring for our children”
      Parent involvement appreciation8• “really glad to participate”
      • “cutting edge to have us involved”
      Joy of meeting applicants8• “enjoyed meeting applicants”
      • “enlightening experience to meet young doctors and why they chose to do what they do”
      Applicant themes (positive)
      Patient-centered37• “added another important layer of insight”
      • “great to hear from patient families themselves”
      High-value37• “positively influenced my view of the program and its values”
      • “contributes well to the interview”
      Enjoyable28• “very positive experience”
      • “really great and fun experience”
      Innovative25• “unique idea”
      • “very innovative”
      Applicant themes (neutral/negative)
      Want more parent perspective12• “would have liked to hear more of their experience”
      • “hoping to get more parent perspective”
      Unfamiliar with questions/format11• “the questions asked were kind of awkward”
      • “most difficult and abstract questions of the day”
      More time or questions9• “wanted to spend more time with the parent”
      • “would have liked a few more questions to discuss”
      Low-value5• “not sure if serves any purpose”
      • “not sure if there is much utility in having this on the interview day”
      Averaging parent ratings of comfort in developing a relationship with the prospective resident and confidence in the prospective resident's ability to be a team member caring for their child showed that parents rated applicants favorably overall (8.57 ± 1.16). Matched applicants versus other applicants (9.28 ± 0.83 vs 8.51 ± 1.17; P = .05), as well as those in the top 10 versus other applicants (9.20 ± 0.75 vs 8.51 ± 1.18; P = .07), were rated higher but without reaching statistical significance. The top 10 applicants were also rated highly compared with other ranked applicants. There were no significant differences between rank tertiles, although higher ranking trended toward higher parent ratings (top 1/3, 8.84 ± 1.08; middle 1/3, 8.49 ± 1.28; bottom 1/3, 8.39 ± 1.10; P = .22).

      Discussion

      This study contributes to the limited literature on parent involvement in the pediatric resident applicant interview process. Because parents are involved in nearly all pediatric clinical encounters, inclusion in the interview process may supplement successful involvement in other similar learner assessments to complement faculty, peer, and interprofessional perceptions and feedback.
      • Eady K.
      • Moreau K.A.
      Using parent feedback: a qualitative study of residents' and physician-educators' perspectives.
      • Moreau K.A.
      • Eady K.
      • Tang K.
      • et al.
      The development of the PARENTS: a tool for parents to assess residents' non-technical skills in pediatric emergency departments.
      • Moreau K.A.
      • Pound C.M.
      • Eady K.
      Pediatric caregiver involvement in the assessment of physicians.
      We found that the unique experience of including parents was both achievable and viewed positively by parents and applicants. Several themes emerged from both groups, with the most salient themes including the value and appreciation of patient-centered perspectives, innovation, value, and joy of the meeting. Parent interview scores also correlated with the final match list, with matched and top applicants scoring highly.
      Nearly all parents favorably valued the process and rated their experience highly. Parents' perceptions were strongly positive about being included as stakeholders in interviewing prospective residents. As demonstrated in other venues, involvement and feedback may also provide an early valuable window into prospective residents' communication and family engagement, which are crucial skills in the clinical learning environment.
      • Brinkman W.B.
      • Geraghty S.R.
      • Lanphear B.P.
      • et al.
      Evaluation of resident communication skills and professionalism: a matter of perspective?.
      • Osorio S.N.
      • Ward M.J.
      • Siew L.
      • et al.
      Assessment of pediatric residents' communication and interpersonal skills during family-centered rounds.
      • Bogetz A.L.
      • Rassbach C.E.
      • Chan T.
      • et al.
      Exploring the educational value of patient feedback: a qualitative analysis of pediatric residents' perspectives.
      Comments and ratings from applicants suggested an overall positive experience. Applicants who commented on the enjoyable, high-valued, and patient-centered aspects of the process rated their experience more favorably than applicants who commented on their unfamiliarity with parent questions and the interview format and/or described the process as being of less value. Although applicants overall felt very positive about the process and experience of involving parents, they felt somewhat less positive about the perceived value of parent involvement informing their selection of a residency program. The unique experience, setting of an interview day, time limitations, and meeting parents outside of the context of a clinical encounter may also contribute to these findings. The unfamiliarity of this experience in the framework of expectations during a residency interview and possible perceptions of nonphysician feedback may lead to more ambivalence regarding perceived value.
      • Vesel T.P.
      • O'Brien B.C.
      • Henry D.M.
      • et al.
      Useful but different: resident physician perceptions of interprofessional feedback.
      These findings suggest that applicants overall enjoyed meeting parents during the interview, but perhaps did not know how it would inform the selection decision, creating uncertainty. Improving applicant knowledge about the impact of including parents and how it may affect decision making in resident selection will be an important next step.
      In reviewing parent assessments, the average parent rating of comfort and confidence significantly favored applicants who subsequently matched with our program and had some alignment with top applicant ranking positions. More parent familiarity and experience in interviewing, refinement of the assessment tool, and parent engagement in the ranking process may aid in studying the overall impact of this involvement. Parent comments may provide rich content for holistic review of applicants. Interestingly, even without academic information, parents could identify matched and top-ranked applicants, warranting more inquiry on the weight of academic achievements by selection committees. Future inclusion of this data in the review and ranking process may crystallize its value within ranking groups and clarify its impact on eventual rank positions. More study is needed to determine the impact on the accuracy of these ratings as it relates to future resident performance and subsequent patient and family interactions.
      Through this pilot study, we achieved our aim to complete an interview season with the involvement of parents. Although the aim was met successfully without using extra resources, considerations of coordination, funding, and time must be factored into scaling and implementing this at other institutions. Finally, the impact of parents volunteering their time and effort must also be considered carefully.
      This study had several limitations. It was conducted at a single institution with a 30-resident pediatric residency program. Because parents were volunteers, there is a likelihood of selection and social desirability bias influencing survey responses. In addition, the brief faculty-led orientation may have inadvertently influenced parents to view applicants similarly to faculty. While neither parent feedback nor participation in the questionnaire affected ranking, social desirability and perceived lack of anonymity for applicants in completing the survey may have biased their responses. Finally, our study did not longitudinally follow applicants into their residency training to measure correlation with actual performance.

      Conclusion

      Parents and patients are important stakeholders in health care delivery in all areas of the clinical learning environment, and their involvement in the recruitment of future learners and leaders offers a unique patient-centered inclusion opportunity. We demonstrated that involving parents in the pediatric residency applicant interview process was achievable, and because parental scoring mirrored our final resident ranking, this approach may support a program's ranking and selection process. Future inquiry may guide how this involvement affects applicants' decisions regarding their future training settings and how it may provide a further window into communication skills and relationship development related to ranking and selection.

      Acknowledgment

      We thank Bruce Blumberg, MD, for his support of this project.

      References

      1. National Resident Matching Program, Data Release, and Research Committee.
        (Results of the 2015 NRMP Applicant Survey by Preferred Specialty and Applicant Type. Washington, DC: National Resident Matching Program)2015
        • Schwartz A.
        • Margolis M.J.
        • Multerer S.
        • et al.
        A multi-source feedback tool for measuring a subset of Pediatrics Milestones.
        Med Teach. 2016; 38: 995-1002
        • Sargeant J.
        • Bruce D.
        • Campbell C.M.
        Practicing physicians' needs for assessment and feedback as part of professional development.
        J Contin Educ Health Prof. 2013; 33
        • Levinson W.
        • Lesser C.S.
        • Epstein R.M.
        Developing physician communication skills for patient-centered care.
        Health Aff. 2010; 29: 1310-1318
        • Stephenson-Famy A.
        • Houmard B.S.
        • Oberoi S.
        • et al.
        Use of the interview in resident candidate selection: a review of the literature.
        J Grad Med Educ. 2015; 7: 539-548
        • Brinkman W.B.
        • Geraghty S.R.
        • Lanphear B.P.
        • et al.
        Evaluation of resident communication skills and professionalism: a matter of perspective?.
        Pediatrics. 2006; 118: 1371-1379
        • O'Keefe M.
        Should parents assess the interpersonal skills of doctors who treat their children? A literature review.
        J Paediatr Child Health. 2001; 37: 531-538
        • Hicks P.J.
        • Englander R.
        • Schumacher D.J.
        • et al.
        Pediatrics milestone project: next steps toward meaningful outcomes assessment.
        J Grad Med Educ. 2010; 2: 577-584
        • Makoul G.
        • Krupat E.
        • Chang C.H.
        Measuring patient views of physician communication skills: development and testing of the Communication Assessment Tool.
        Patient Educ Couns. 2007; 67: 333-342
        • Min A.A.
        • Leetch A.
        • Nuño T.
        • et al.
        How well will you FIT? Use of a modified MMI to assess applicants' compatibility with an emergency medicine residency program.
        Med Educ Online. 2016; 21: 29587
        • Association of American Medical Colleges
        Interviewing for residency: frequently asked interview questions.
        (Available at:)
        • Eady K.
        • Moreau K.A.
        Using parent feedback: a qualitative study of residents' and physician-educators' perspectives.
        Perspect Med Educ. 2018; 7: 33-39
        • Moreau K.A.
        • Eady K.
        • Tang K.
        • et al.
        The development of the PARENTS: a tool for parents to assess residents' non-technical skills in pediatric emergency departments.
        BMC Med Educ. 2017; 17: 210
        • Moreau K.A.
        • Pound C.M.
        • Eady K.
        Pediatric caregiver involvement in the assessment of physicians.
        BMC Med Educ. 2015; 15: 123
        • Osorio S.N.
        • Ward M.J.
        • Siew L.
        • et al.
        Assessment of pediatric residents' communication and interpersonal skills during family-centered rounds.
        Hosp Pediat. 2012; 2: 85-92.
        • Bogetz A.L.
        • Rassbach C.E.
        • Chan T.
        • et al.
        Exploring the educational value of patient feedback: a qualitative analysis of pediatric residents' perspectives.
        Acad Pediat. 2017; 17: 4-8
        • Vesel T.P.
        • O'Brien B.C.
        • Henry D.M.
        • et al.
        Useful but different: resident physician perceptions of interprofessional feedback.
        Teach Learn Med. 2016; 28: 125-134