Advertisement

Look Who's Talking: A Survey of Pediatric Program Directors on Communication Skills Education in Pediatric Residency Programs

      Abstract

      Objective

      To determine current practices for communication skills curriculum and assessment in pediatric residency programs and to identify programs’ greatest needs regarding communication curricula and assessment.

      Methods

      We surveyed pediatric residency program directors about their programs’ approach to teaching and assessing residents’ communication skills and how satisfied they were with their curricula and assessment of competence. Respondents were asked about their programs’ greatest needs for teaching and assessing communication skills.

      Results

      Response rate was 41% (82/202). Most programs did teach communication skills to residents; only 14% provided no formal training. Programs identified various 1) educational formats for teaching communication skills, 2) curricular content, and 3) assessment methods for determining competence. Many programs were less than satisfied with their curriculum and the accuracy of their assessments. The greatest programmatic need regarding curricula was time, while the greatest need for assessment was a tool.

      Conclusions

      While teaching and assessment of communication skills is common in pediatric residency programs, it is inconsistent and variable, and many programs are not satisfied with their current communication training. There is need for development of and access to appropriate and useful curricula as well as a practical tool for assessment which has been evaluated for validity evidence.

      Keywords

      What's New
      This study describes current practices and identifies greatest needs for communication training and assessment in pediatric residency programs. These data highlight inconsistent communication teaching and assessment nationally and help lay groundwork to guide the development of resources for best practices.
      Effective communication in medicine is critical to both patient care and patient satisfaction, and it is crucial to building trust in the physician−patient relationship.
      • Dwamena F
      • Holmes-Rovner M
      • Gaulden CM
      • et al.
      Interventions for providers to promote a patient-centred approach in clinical consultations.
      • Boissy A
      • Windover AK
      • Bokar D
      • et al.
      Communication skills training for physicians improves patient satisfaction.
      In addition, both the Accreditation Council for Graduate Medical Education and the American Board of Pediatrics have placed an increasingly large emphasis on developing communication skills in physicians-in-training. Many physicians, from all levels of training, have reported that they lack confidence in their communication abilities.
      • Keir A
      • Wilkinson D.
      Communication skills training in paediatrics.
      • Ishikawa H
      • Son D
      • Eto M
      • et al.
      The information-giving skills of resident physicians: relationships with confidence and simulated patient satisfaction.
      In addition, studies examining patients’ perspectives report that physician communication has significant room for improvement.
      • Trudel JG
      • Leduc N
      • Dumont S
      Perceived communication between physicians and breast cancer patients as a predicting factor of patients' health-related quality of life: a longitudinal analysis.
      • Merel SE
      • McKinney CM
      • Ufkes P
      • et al.
      Sitting at patients' bedsides may improve patients' perceptions of physician communication skills.
      Communication in pediatrics can be even more challenging, as physicians must navigate not only both communication with parents or caregivers but also children at their individual developmental levels.
      Residents in pediatrics perceive competence in communication skills as important and an educational priority yet feel inadequately prepared, particularly for difficult conversations.
      • Rassbach CE
      • Bogetz AL
      • Orlov N
      • et al.
      The effect of faculty coaching on resident attitudes, confidence, and patient-rated communication: a multi-institutional randomized controlled trial.
      • Peterson EB
      • Boland KA
      • Bryant KA
      • et al.
      Development of a comprehensive communication skills curriculum for pediatrics residents.
      Although the Accreditation Council for Graduate Medical Education clearly prioritizes interpersonal communication skills as one of its core competencies and requires residents to achieve competence in this area,

      Accreditation Council for Graduate Medical Education. Common program requirements. http://www.acgme.org/What-We-Do/Accreditation/Common-Program-Requirements. Accessed December 1, 2018

      residency training programs may measure competence differently, creating the potential for inconsistent education and assessment in this critical skill. It is unknown the degree to which different pediatric residency programs teach and assess competence in communication skills and if programs have unmet needs in this area.
      We conducted a survey-based, mixed-methods study to assess national current practices regarding communication skills training and assessment of competence in pediatric residency programs. We also aimed to identify the greatest needs in pediatric residency programs for communication training to guide curriculum and assessment development.

      Methods

       Survey Development

      Initial survey content was developed by 3 investigators from different institutions, as members of the Association of Pediatric Program Directors (APPD) Assessment Task Force. The survey questions were developed using survey design best practices.
      • Artino Jr., AR
      • Gehlbach H
      • Durning SJ
      AM last page: avoiding five common pitfalls of survey design.
      • Magee C
      • Rickards G
      • A Byers L
      • et al.
      Tracing the steps of survey design: a graduate medical education research example.
      Survey questions were reviewed by the APPD Research and Scholarship Task Force and then revised by the investigators based on this consensus review. Finally, the survey was refined using cognitive interviewing.
      • Willis GB
      • Artino Jr., AR
      What do our respondents think we're asking? Using cognitive interviewing to improve medical education surveys.
      • Willis G.
      Cognitive interviewing as a tool for improving the informed consent process.
      Seven cognitive interviews were conducted with a combined approach using think-aloud and verbal probing techniques. The survey was further revised for clarity based on the cognitive interviews, resulting in the final survey. The institutional review boards at Nationwide Children's Hospital, the Partners Human Research Committee, and University of Texas Southwestern deemed this study exempt.

       Survey Content

      The survey contained 22 total items: 11 quantitative questions, 4 qualitative questions, and 7 questions with quantitative and qualitative components. Initial questions asked about size and setting of residency programs. Subsequent questions asked about communication curricula, including format, frequency, content, and resources at the respondent's program. Respondents were then asked about methods used to assess residents’ competence in communication and for their perspective on the effectiveness of these assessments using a 5-point Likert scale. Finally, respondents were asked open-ended, qualitative questions about their greatest programmatic needs for communication training and assessment at their residency programs (online Appendix).

       Survey Administration

      The final survey was distributed by the APPD using Survey Monkey. The survey was only sent to directors of pediatric residency programs to avoid multiple responses from the same institution. Program directors were instructed to forward the survey to another program representative (such as an associate program director) if appropriate. A total of 3 reminder e-mails were sent at 2-week intervals following initial survey distribution.

       Data Analysis

      A Chi-square test was used to compare demographics in the sample population compared with to the APPD membership at large. Membership data were obtained directly from the APPD. For quantitative questions, data were analyzed using descriptive statistics, summarized by percentages, calculated using Survey Monkey software. The Likert scale responses were treated as categorical variables in the analysis. For qualitative questions, content analysis was used. Using Microsoft Excel (Microsoft, Redmond, Wash), the investigators independently coded qualitative responses and grouped these into categories. Investigators then reviewed individual codes and categories together until consensus was reached on code(s) and category(s) assignment for each response. Some responses were assigned more than one code.

      Results

      The response rate was 41% (82/202). This response rate is similar to other APPD survey studies, with response rates from 20% to 53%.
      • Mallory LA
      • Calaman S
      • Lee White M
      • et al.
      Targeting simulation-based assessment for the pediatric milestones: a survey of simulation experts and program directors.
      • Mann KJ
      • Craig MS
      • Moses JM
      Quality improvement educational practices in pediatric residency programs: survey of pediatric program directors.
      Small, medium, and large programs were represented. There was no significant difference (P = .71) in the distribution of program size in our survey sample compared with the APPD at large. A variety of program settings (university, community, military) were represented. Our sample included slightly more university-affiliated programs (56% compared with 47%) and fewer university-affiliated, community based (28% compared with 44%) programs compared with the APPD at large, representing a significant difference (P = .015) between survey responders and the APPD overall.

       Quantitative Results

      Programs identified a variety of formats for teaching communication skills. The most common formats used were formal didactics (64%), role-playing (48%), workshops (44%), and simulated cases (40%). Other formats were less frequently reported, including standardized patients, online modules, required reading, and direct observation. Notably, 14% of programs reported no formal education for communication skills. Most programs delivered communication training 1 to 3 times per year for each postgraduate year level. A few programs reported more frequent communication training. Communication training was specific to the level of the learner in only 30% of programs.
      The content of communication curricula varied. The most frequent topics included: handovers (87%), teamwork (70%), delivering bad news (68%), and managing difficult families (60%). Other topics identified by fewer programs included consults, error disclosure, feedback, adolescent topics (eating disorders, lesbian, gay, bisexual, transgender, and queer [ie, LGBTQ] patients, confidentiality), and tele-medicine. Most programs (62%) reported having 2 to 5 faculty members involved in teaching communication skills. Thirty-eight percent of programs reported additional costs associated with resident communication training, such as costs for standardized patients, simulation, retreats, and outside professional facilitators.
      Assessment methods also varied among programs. The most commonly reported assessment methods used were direct observation of patient interactions (94%), direct observation of handoffs (86%), rotation evaluation questions using milestone language (86%), and anecdotal input from the Clinical Competency Committee (CCC) (80%). Only 16% of programs reported using a formal assessment tool. The most frequently used sources for CCC determination of communication milestones were rotation questions using milestone language (86%), input from CCC members (83%), and free text on evaluations (80%). Less frequently, programs used rotation questions without milestone language, a formal assessment tool, and/or multisource (360, nursing, parent) evaluations.
      Programs’ satisfaction with their own communication training and the accuracy of their assessments were mixed. Most programs (59%) reported being somewhat satisfied with the communication skills training at their institutions, with 23% reporting being somewhat dissatisfied or very dissatisfied. About one half (52%) felt their programs evaluated communication subcompetencies well, whereas 40% felt either neutral, or that these subcompetencies were evaluated poorly or very poorly. Similarly, 54% of programs were somewhat confident with the accuracy of communication-specific milestones levels assigned by the CCC, whereas 28% were neutral, not very confident, or not at all confident.

       Qualitative Results

      Programs reported a variety of needs regarding the teaching and assessment of communication skills, although several common needs were identified (Figure 1, Figure 2). For teaching communication, time was the most commonly identified need. Programs specifically identified both resident and faculty time, and time for delivery of the curriculum. Many programs also identified curriculum and specific curricular content as the greatest need. For assessment of communication skills, a specific assessment tool was the most commonly identified need, followed by faculty development and engagement.
      Figure 1
      Figure 1Greatest needs for teaching communication skills. Faculty Dev indicates faculty development; Stand PT/SIM, standardized patient/simulation; and Resident Engag., resident engagement.
      Figure 2
      Figure 2Greatest needs for assessing communication skills.

      Discussion

      Our survey results show that although communication training is occurring in most pediatric residency programs, the content, delivery, and modes of assessment are highly variable. Many of our findings suggest there is room for improvement in both delivery of educational content and in assessment of communication skills in pediatric residents. We found that communication skills training is infrequent, often taught using passive techniques, and often not specific to the level of the learner. Only a small number of faculty are involved in this educational process, despite communication being a universally required skill for physicians. In addition, almost one quarter of programs were somewhat or very dissatisfied with their institutional communication skills training programs.
      Programs reported needing more time, specific curricular content, faculty development, and overall engagement in communication skills development. These unmet needs mirror the known difficulties in delivering educational experiences to help trainees achieve competence in required skills when programs do not have a standardized implementation model.
      • Peterson EB
      • Boland KA
      • Bryant KA
      • et al.
      Development of a comprehensive communication skills curriculum for pediatrics residents.
      One such example of a successful standardized model is the I-PASS handoff curriculum, which was developed to specifically standardize handover communication. This tool was made widely available and studied and thus has accumulated validity evidence.
      • O'Toole JK
      • West DC
      • Starmer AJ
      • et al.
      Placing faculty development front and center in a multisite educational initiative: lessons from the I-PASS Handoff study.
      • Starmer AJ
      • O'Toole JK
      • Rosenbluth G
      • et al.
      Development, implementation, and dissemination of the I-PASS handoff curriculum: a multisite educational intervention to improve patient handoffs.
      This example serves as a model for how communication curricula can be implemented and evaluated. Our study identified room for improvement in educational strategies and materials that actually meet the needs and demands of programs for communication training: curricular materials that 1) are low cost, 2) require only a moderate time for delivery 3) require minimal faculty training, and 4) employ communication skills teaching that is incorporated into regular workflow, so all faculty and residents are participating.
      Communication skills assessment is also occurring broadly in pediatric residency training, but the large majority of programs do not have a formal assessment tool to evaluate communication skills. Only 5 programs used a tool that has been previously described and studied in the literature. The scant use of standardized tools and the frequent use of summative rotation evaluations and anecdotal input may compromise the consistency and integrity of communication assessment. Despite this, about one half of programs still felt they assessed communication skills well, highlighting a possible disconnect between perception and reality of the accuracy of assessment.
      The most commonly identified assessment need was a useful tool, which mirrors our finding that few programs actually use one for communication assessment. One reason may be that although there are many different communication tools reported,
      • Oczkowski SJ
      • Chung HO
      • Hanvey L
      • et al.
      Communication tools for end-of-life decision-making in ambulatory care settings: a systematic review and meta-analysis.
      • Davis J
      • Roach C
      • Elliott C
      • et al.
      Feedback and assessment tools for handoffs: a systematic review.
      many have limited validity evidence, making it difficult for programs to identify the most useful and relevant one(s). A high-quality, relevant assessment tool may aid not only in summative assessment, but more importantly in formative assessment, providing rich feedback for learners. Faculty development and engagement was also a frequently identified need for communication assessment, again stressing the need for assessment materials which require little time and training, and incorporate assessment activities into regular workflow.
      Limitations to our study include a potential lack of generalizability of these data to all pediatric residency programs, in part because we had a relatively low survey response rate. In addition, university-based (compared with university affiliated, community-based) programs were overrepresented in our sample population, potentially skewing results toward preferences of these programs. There could have been bias in responders: those with interest in communication skills development or with self-identified struggles with communication skills training for residents may have been more likely to complete the survey.
      Next steps include exploring ways to meet the needs identified by programs to improve communication skills training and assessment, specifically focusing on 1) curricula and 2) assessment tools. Development of easy-to-use, accessible curricula would be most beneficial to programs. One such possibility could be a “Just-in-Time” communication training for faculty to use on rounds, just before a resident is to navigate a conversation with a family. Faculty could coach for less than 5 minutes outside the room, directly observe the communication encounter, then give immediate feedback. This kind of skills training would incorporate more faculty into communication training, be directly and immediately relevant for residents, and is not time-intensive. More work is needed to further understand other curricular content needs, and which communication skills are most needed and high-yield in pediatric training.
      In addressing the need for an assessment tool, further work should be done to identify and establish validity evidence for tool(s) which are most applicable to communication skills in pediatrics. Key components of an ideal tool would be brevity and ease of use with little training and the ability to use the tool in real-time, such as during rounds.

      Conclusions

      We found that although communication skills training and assessment is commonly occurring in pediatric residency programs, it is inconsistent and variable, and many programs are not satisfied with their current communication curriculum for their residents. There is need for development of and access to appropriate and useful curricula as well as practical and useful tools for assessment.

      Acknowledgments

      We acknowledge Kristina Dzara, PhD, MMSc, for her statistical support and the APPD Research and Scholarship Task Force for their review of our survey.

      Appendix. Supplementary Data

      References

        • Dwamena F
        • Holmes-Rovner M
        • Gaulden CM
        • et al.
        Interventions for providers to promote a patient-centred approach in clinical consultations.
        Cochrane Database Syst Rev. 2012; 12CD003267
        • Boissy A
        • Windover AK
        • Bokar D
        • et al.
        Communication skills training for physicians improves patient satisfaction.
        J General Intern Med. 2016; 31: 755-761
        • Keir A
        • Wilkinson D.
        Communication skills training in paediatrics.
        J Paediatr Child Health. 2013; 49: 624-628
        • Ishikawa H
        • Son D
        • Eto M
        • et al.
        The information-giving skills of resident physicians: relationships with confidence and simulated patient satisfaction.
        BMC Med Educ. 2017; 17: 34
        • Trudel JG
        • Leduc N
        • Dumont S
        Perceived communication between physicians and breast cancer patients as a predicting factor of patients' health-related quality of life: a longitudinal analysis.
        Psychooncology. 2014; 23: 531-538
        • Merel SE
        • McKinney CM
        • Ufkes P
        • et al.
        Sitting at patients' bedsides may improve patients' perceptions of physician communication skills.
        J Hosp Med. 2016; 11: 865-868
        • Rassbach CE
        • Bogetz AL
        • Orlov N
        • et al.
        The effect of faculty coaching on resident attitudes, confidence, and patient-rated communication: a multi-institutional randomized controlled trial.
        Acad Pediatr. 2019; 19: 186-194
        • Peterson EB
        • Boland KA
        • Bryant KA
        • et al.
        Development of a comprehensive communication skills curriculum for pediatrics residents.
        J Grad Med Educ. 2016; 8: 739-746
      1. Accreditation Council for Graduate Medical Education. Common program requirements. http://www.acgme.org/What-We-Do/Accreditation/Common-Program-Requirements. Accessed December 1, 2018

        • Artino Jr., AR
        • Gehlbach H
        • Durning SJ
        AM last page: avoiding five common pitfalls of survey design.
        Acad Med. 2011; 86: 1327
        • Magee C
        • Rickards G
        • A Byers L
        • et al.
        Tracing the steps of survey design: a graduate medical education research example.
        J Grad Med Educ. 2013; 5: 1-5
        • Willis GB
        • Artino Jr., AR
        What do our respondents think we're asking? Using cognitive interviewing to improve medical education surveys.
        J Grad Med Educ. 2013; 5: 353-356
        • Willis G.
        Cognitive interviewing as a tool for improving the informed consent process.
        J Empir Res Hum Res Ethics. 2006; 1: 9-24
        • Mallory LA
        • Calaman S
        • Lee White M
        • et al.
        Targeting simulation-based assessment for the pediatric milestones: a survey of simulation experts and program directors.
        Acad Pediatr. 2016; 16: 290-297
        • Mann KJ
        • Craig MS
        • Moses JM
        Quality improvement educational practices in pediatric residency programs: survey of pediatric program directors.
        Acad Pediatr. 2014; 14: 23-28
        • O'Toole JK
        • West DC
        • Starmer AJ
        • et al.
        Placing faculty development front and center in a multisite educational initiative: lessons from the I-PASS Handoff study.
        Acad Pediatr. 2014; 14: 221-224
        • Starmer AJ
        • O'Toole JK
        • Rosenbluth G
        • et al.
        Development, implementation, and dissemination of the I-PASS handoff curriculum: a multisite educational intervention to improve patient handoffs.
        Acad Med. 2014; 89: 876-884
        • Oczkowski SJ
        • Chung HO
        • Hanvey L
        • et al.
        Communication tools for end-of-life decision-making in ambulatory care settings: a systematic review and meta-analysis.
        PLoS One. 2016; 11e0150671
        • Davis J
        • Roach C
        • Elliott C
        • et al.
        Feedback and assessment tools for handoffs: a systematic review.
        J Grad Med Educ. 2017; 9: 18-32