Advertisement

Teaching and Assessing Communication Skills in Pediatric Residents: How Do Parents Think We Are Doing?

      Abstract

      Objective

      Curricula designed to teach and assess the communication skills of pediatric residents variably integrates the parent perspective. We compared pediatric residents' communication skills in an objective structured clinical exam (OSCE) case as assessed by Family Faculty (FF), parents of pediatric patients, versus standardized patients (SP).

      Methods

      Residents participated in an OSCE case with a SP acting as a patient's parent. We compared resident performance as assessed by FF and SP with a behaviorally-anchored checklist. Items were rated as not done, partly done or well done, with well-done indicating mastery. The residents evaluated the experience.

      Results

      42 residents consented to study participation. FF assessed a lower percentage of residents as demonstrating skill mastery as compared to SP in 19 of the 23 behaviors. There was a significant difference between FF and SP for Total Mastery Score and Mastery of the Competency Scores in three domains (Respect and Value, Information Sharing and Participation in Care and Decision Making). The majority of residents evaluated the experience favorably.

      Conclusion

      Involving parents of pediatric patients in the instructive and assessment components of a communication curriculum for pediatric residents adds a unique perspective and integrates the true stakeholders in parent—physician communication.

      Keywords

      What's New
      Successful parent-physician communication is foundational in pediatrics. Parents of pediatric patients provide a different perspective than standardized patients when assessing pediatric resident communication skills in an OSCE case. Including parents in communication skill training for pediatric residents integrates true stakeholders.
      Clear physician-parent communication that promotes partnership is one of the foundations of practicing effective pediatric medicine. A physician's ability to successfully communicate has a strong impact on patient experience and satisfaction.
      • Boissy A
      • Windover AK
      • Bokar D
      • et al.
      Communication skills training for physicians improves patient satisfaction.
      ,
      • Brinkman WB
      • Geraghty SR
      • Lanphear BP
      • et al.
      Evaluation of resident communication skills and professionalism: a matter of perspective?.
      Given the importance of communication to health outcomes, it is not surprising that the Accreditation Council for Graduate Medical Education (ACGME) emphasizes the need to develop this crucial skill.

      Accreditation Council for Graduate Medical Education. ACMGE program requirements for resident education in pediatrics. Available at: https://www.acgme.org/Specialties/Program-Requirements-and-FAQs-and-Applications/pfcatid/16/Pediatrics. Accessed June 7, 2021.

      The majority of pediatric residency programs provide formal communication skill training.
      • Reed S
      • Frey-Vogel A
      • Frost M.
      Look who's talking: a survey of pediatric program directors on communication skills education in pediatric residency programs.
      Previous publications support the importance of such curricula, demonstrating that the communication skills of residents improve after participation.
      • Newcomb AB
      • Liu C
      • Trickey AW
      • et al.
      Patient perspectives of surgical residents' communication: do skills improve over time with a communication curriculum?.
      ,
      • Keir A
      • Wilkinson D.
      Communication skills training in paediatrics.
      Residents report that completing such curricula is valuable and effective and leads to improvement in self-assessed comfort.
      • Morgan ER
      • Winter RJ.
      Teaching communication skills. An essential part of residency training.
      ,
      • Peterson EB
      • Boland KA
      • Bryant KA
      • et al.
      Development of a comprehensive communication skills curriculum for pediatrics residents.
      Despite the commonplace presence of communication curricula in pediatric residency programs, there is inconsistency and variability in how programs teach these skills to their trainees and assess competency.
      • Reed S
      • Frey-Vogel A
      • Frost M.
      Look who's talking: a survey of pediatric program directors on communication skills education in pediatric residency programs.
      Simulation-based education, specifically an Objective Structured Clinical Examination (OSCE), has well documented benefits and can fill the gap in difficult to assess competencies such as interpersonal communication.
      • Mallory LA
      • Calaman S
      • Lee White M
      • et al.
      Targeting simulation-based assessment for the pediatric milestones: a survey of simulation experts and program directors.
      • Talwalkar JS
      • Murtha TD
      • Prozora S
      • et al.
      Assessing advanced communication skills via objective structured clinical examination: a comparison of faculty versus self, peer, and standardized patient assessors.
      • Hanley K
      • Gillespie C
      • Zabar S
      • et al.
      Monitoring communication skills progress of medical students: establishing a baseline has value, predicting the future is difficult.
      In pediatrics, standardized patients (SPs) can act in the role of a patient's parent and provide learner assessments, but the degree to which this reflects the true parent perspective is unknown.
      Including the parent perspective is especially important to advancing a culture of safe, patient and family-centered care.
      • Brinkman WB
      • Geraghty SR
      • Lanphear BP
      • et al.
      Evaluation of resident communication skills and professionalism: a matter of perspective?.
      ,
      • Rassbach CE
      • Bogetz AL
      • Orlov N
      • et al.
      The effect of faculty coaching on resident attitudes, confidence, and patient-rated communication: a multi-institutional randomized controlled trial.
      ,
      • DeBlasio D
      • Real FJ
      • Ollberding NJ
      • et al.
      Provision of parent feedback via the communication assessment tool: does it improve resident communication skills?.
      We include parent perspective in the communication skills training for our pediatric residents through a partnership with the Sala Patient and Family Faculty Program. Family Faculty (FF) are parents of pediatric patients cared for at our institution who employ their personal experiences in the observation, evaluation, and provision of feedback to trainees and staff. We developed an OSCE case for pediatric residents focused on physician-parent communication that utilized FF as educators and assessors. The objective of this study is to compare pediatric resident communication skills in an OSCE case as assessed by FF versus SPs.

      Methods

       Study Design

      This is a cohort study of pediatric residents in post graduate years (PGY) 1-3 at New York University Grossman School of Medicine. Residents participated in a single OSCE case as the initial session of a new communication curriculum for our residency program. The participants did not have any previous communication skills training during their residency. Participation in the OSCE case is an educational requirement for all residents, but data for research purposes were collected only from those residents who consented to study participation as part of our institutional review board-approved Medical Education Research Registry.
      Training materials for the OSCE case were developed by conducting in-person focus groups and mock-simulations with FF, SPs, and clinicians at New York Simulation Center. There were 8 participating SPs who are professional actors and who are paid $25 per hour. There were 10 FF who volunteer their time. The SPs and FF underwent approximately 6 hours of training that was led by family engagement experts from the Sala Institute for Child and Family Centered Care at Hassenfeld Children's Hospital. They were taught the core competencies of patient and family-centered care

      Institute for Patient- and Family-Centered Care. Available at: https://www.ipfcc.org/. Accessed June 7, 2021.

      and the key behaviors that the residents would be evaluated on. They were trained how to objectively complete the assessment tool using specific behavioral anchors and how to give feedback to the resident aligned with the tool and the case specific learning objectives. The SPs received additional coaching from FF to ensure authenticity and standardization in their character portrayal and reactions.

       The OSCE Case

      The residents individually participated in a single scenario OSCE in which they were tasked with disclosing a medical error to the SP, who was acting in the role of the patient's parent (Supplement A). The OSCE case was designed by members of our departmental education team, reviewed by FF with integration of their feedback and pilot tested on faculty. The scenario required the resident to disclose that an incorrect lab test had been ordered and that the patient would require repeat venipuncture. The residents were told that the learning objective was to increase comfort and skill in having difficult conversations and that they were to focus on communication rather than the specific error. This particular case required the resident to disclose the error, respond to the parent's emotion and then partner with the parent in an actionable next step (ie, a repeat blood draw). The 10-minute encounter between the resident and SP was observed by a FF, a pediatric clinical faculty member and trained debrief facilitator through one-way glass. The debrief facilitators have clinical backgrounds in social work, psychology, psychiatry or child life, and facilitate groups as part of their professional role. They undergo annual training specifically about debriefing with FF and they use a facilitator guide to ensure consistency in the debrief sessions. Prior to the case starting, the resident was oriented to the scenario, informed of the observers and reminded that simulation is a safe learning environment. Following the encounter, there was a 20-minute, 360 degree debrief led by the trained facilitator that included the resident, the SP, the FF and the clinical faculty member.

       Assessment

      After the conclusion of the live scenario, but prior to the debrief, the FF and SP each independently completed a 23 item behaviorally anchored assessment tool, the Patient and Family Centered Communication Assessment Tool (Supplement B). The tool was adapted from the NYU Communication Skills Assessment Tool (Supplement C), which is a behaviorally anchored checklist with previously published evidence of reliability and validity.
      • Hanley K
      • Gillespie C
      • Zabar S
      • et al.
      Monitoring communication skills progress of medical students: establishing a baseline has value, predicting the future is difficult.
      ,
      • Hanley K
      • Zabar S
      • Altshuler L
      • et al.
      Opioid vs nonopioid prescribers: variations in care for a standardized acute back pain case.
      ,
      • Yedidia MJ
      • Gillespie CC
      • Kachur E
      • et al.
      Effect of communications training on medical student performance.
      With input from clinicians and FF, the language of the tool was modified to be family centered and the items were grouped into 5 domains (Respect and Value, Information Sharing, Participation in Care and Decision-Making, Follow-up and Next Steps, and Working as a Team) that reflect and expand upon the core concepts of patient and family centered care from the Institute for Patient and Family Centered Care.

      Institute for Patient- and Family-Centered Care. Available at: https://www.ipfcc.org/. Accessed June 7, 2021.

      The tool was refined through an iterative process in which clinical faculty, FF, and SPs participated in pilot OSCEs and provided feedback on the tool. Finally, a pilot OSCE was done with pediatric subspecialty fellows showing feasibility of the tool to assess trainees. In alignment with the NYU Communication Skills Assessment Tool, the items were rated as not done, partly done or well done, with well-done indicating mastery. After the debrief, the residents each completed a three question evaluation of the curriculum using a 5-point Likert scale in which they rated statements from very untrue to very true.

       Statistical Analysis

      For each behavior, the percent of residents who achieved “well-done” was calculated for the FF and SP assessments. For the 4 domains with more than one item, a Mastery of the Competency Score was determined by collapsing the behaviors in that domain and calculating the average percent “well-done.” Total Mastery Score was determined by collapsing all 23 behaviors and calculating the average percent “well-done.” A Cronbach's alpha was computed to assess internal consistency of the scales. Paired t tests were done for the Mastery of the Competency Scores and the Total Mastery Score. A nonparametric Wilcoxon signed rank test was used to examine the difference between the FF and SP ratings for the single item in the Working as a Team domain.

      Results

      Fifty-two (90%) of the 58 pediatric residents completed the OSCE and 42 (81%) consented to study participation. Of the participating residents 11 (26%) were PGY1, 15 (36%) were PGY2, and 16 (38%) were PGY3. Twenty-nine (69%) of the residents identify as female. A Cronbach's alpha was computed to assess internal consistency, with scores of 0.78 for the FF ratings and 0.86 for the SP ratings, indicating internal consistency for both the FF and SP score. In 19 of the 23 behaviors measured, the FF assessed a lower percentage of residents as demonstrating skill mastery as compared to the SP (Table).
      TableComparison of Percent of Residents Who Demonstrated Mastery of a Behavior as Assessed by the Family Faculty (FF) Versus the Standardized Patient (SP), N = 42
      CompetencyBehaviorFF Assessment, n (%)SP Assessment, n (%)P Value
      Respect and valueIntroduction29 (69)35 (83)
      Elicited patient story using appropriate questions14 (33)28 (67)
      Acknowledged emotions and feelings18 (43)23 (55)
      Demonstrated acceptance and lack of judgment30 (71)39 (93)
      Affirmed that the family is a key member of the care team13 (31)22 (52)
      Mastery of Competency Score, %5070<.05
      Information sharingNonverbal behavior23 (55)28 (67)
      Encouraged family to ask questions, interrupt if any misinformation17 (40)17 (40)
      Used reflective listening to confirm understanding13 (31)24 (57)
      Avoided interruption31 (74)29 (69)
      Used of appropriate and understandable language, avoiding jargon22 (52)32 (76)
      Communicated concern or intention to help20 (48)36 (86)
      Inquired about existing knowledge, ideas and readiness10 (24)9 (21)
      Provided clear explanations/information19 (45)29 (69)
      Honesty/Transparency27 (64)38 (90)
      Managed of the narrative flow18 (43)31 (74)
      Appropriately Paced of encounter23 (55)35 (83)
      Mastery of Competency Score, %4867<.05
      Participation in care and decision-makingCollaborated with parent to identify and decide on plan20 (48)30 (71)
      Elicited parent's thoughts on symptoms, underlying concerns, etc.14 (33)29 (69)
      Mastery of Competency Score, %4070<.05
      Follow-up and next stepsClarified information to ensure understanding22 (52)20 (48)
      Provided resources14 (33)26 (62)
      Provided follow-up plan22 (52)31 (74)
      Closed the encounter21 (50)23 (55)
      Mastery of Competency Score, %4760.081
      Working as a teamSupported fellow clinicians’ expertise14 (33)22 (52).077
      Total Mastery Score, %4766<.05
      Table shows the number and percent of residents who scored "well-done," with "well-done" indicating mastery, for each behavior. For domains with more than one behavior, a Mastery of Competency Score was calculated by averaging the percent "well-done" for all the behaviors in that competency. The Total Mastery Score was calculated by averaging the percent "well-done" for all 23 behavior items.
      On the paired t test, there was a significant difference between FF and SP on the Total Mastery Score, (mean score FF = 0.47, SP = 0.66, P = .0001). Mastery of the Competency Score in each domain was lower as assessed by the FF when compared to the SP with significant differences noted in three of five domains (Table). The paired t test for the domain Follow-up and Next Steps did not reach statistical significance, nor did the domain Working as a Team using the Wilcoxon (Figure). Resident feedback showed that 83% felt that the SP interaction was helpful to their future practice, 88% felt that the feedback they received directly from FF was helpful to their future practice, and 90% thought that the 360 degree debrief was a helpful learning tool.
      Figure
      FigureComparison of Total Mastery Score and Mastery of the Competency Scores as assessed by the Family Faculty (FF) versus the Standardized Patient (SP), N = 42. The figure compares the Total Mastery Score and the Mastery of Competency Scores with standard deviation. The Total Mastery Score was calculated by collapsing all 23 behaviors and calculating the percent “well done” for all items. For the 4 domains with more than one item, the Mastery of the Competency Score was calculated by averaging the percent "well-done" for all the behaviors within that domain.

      Discussion

      In this study, we found that FF, parents of pediatric patients, assessed pediatric residents differently than SPs in a communication skills OSCE case scoring them lower in all domains. Previous studies show that including the parent perspective in communication skills training is likely valuable, but the ideal approach is unclear.
      • Brinkman WB
      • Geraghty SR
      • Lanphear BP
      • et al.
      Evaluation of resident communication skills and professionalism: a matter of perspective?.
      ,
      • Rassbach CE
      • Bogetz AL
      • Orlov N
      • et al.
      The effect of faculty coaching on resident attitudes, confidence, and patient-rated communication: a multi-institutional randomized controlled trial.
      ,
      • Bogetz AL
      • Rassbach CE
      • Chan T
      • et al.
      Exploring the educational value of patient feedback: a qualitative analysis of pediatric residents' perspectives.
      ,
      • Brinkman WB
      • Geraghty SR
      • Lanphear BP
      • et al.
      Effect of multisource feedback on resident communication skills and professionalism: a randomized controlled trial.
      Parent feedback after actual patient encounters shows variable impact to resident communication skills.
      • Brinkman WB
      • Geraghty SR
      • Lanphear BP
      • et al.
      Evaluation of resident communication skills and professionalism: a matter of perspective?.
      ,
      • DeBlasio D
      • Real FJ
      • Ollberding NJ
      • et al.
      Provision of parent feedback via the communication assessment tool: does it improve resident communication skills?.
      ,
      • Brinkman WB
      • Geraghty SR
      • Lanphear BP
      • et al.
      Effect of multisource feedback on resident communication skills and professionalism: a randomized controlled trial.
      The real clinical environment can be a high-stakes setting for residents to hone complex communication skills. There is also inconsistency in the ability to standardly assess residents in real encounters since the content of the exchange and the aptitude of the parent to provide effective feedback can vary widely. Studies also show that while residents value patient and family feedback from real clinical encounters, they are skeptical about accuracy and have concern about the educational utility.
      • Bogetz AL
      • Rassbach CE
      • Chan T
      • et al.
      Exploring the educational value of patient feedback: a qualitative analysis of pediatric residents' perspectives.
      Many residency programs use simulation-based education, such as an OSCE with SPs, to standardize content delivery and assessment of communication skills, but the degree to which the assessments done by SPs reflect the perspective of parents of real pediatric patients is unknown. The benefits of simulation-based education and assessment, specifically an OSCE, are well documented.
      • Talwalkar JS
      • Murtha TD
      • Prozora S
      • et al.
      Assessing advanced communication skills via objective structured clinical examination: a comparison of faculty versus self, peer, and standardized patient assessors.
      ,
      • Hanley K
      • Gillespie C
      • Zabar S
      • et al.
      Monitoring communication skills progress of medical students: establishing a baseline has value, predicting the future is difficult.
      It is uniquely valuable for learners to refine skills without impact to real patients, especially with a complex and vital skill such as communication. An OSCE targeting communication allows experiential learning and serves as an educational intervention while also providing skill assessment. SPs are trained to portray a medical scenario in a consistent way and can reliably perform checklist-based assessment.
      • Dickter DN
      • Stielstra S
      • Lineberry M.
      Interrater reliability of standardized actors versus nonactors in a simulation based assessment of interprofessional collaboration.
      SP assessment of physician competence is the gold standard for OSCEs.
      • Talwalkar JS
      • Murtha TD
      • Prozora S
      • et al.
      Assessing advanced communication skills via objective structured clinical examination: a comparison of faculty versus self, peer, and standardized patient assessors.
      Previous studies of resident communication skills comparing SP versus adult patient assessments found that SPs were more stringent in their scoring, while our study found FF were less likely than SPs to assess residents as having met skill mastery.
      • Rezaei R
      • Mehrabani G.
      A comparison of the scorings of real and standardized patients on physician communication skills.
      ,
      • Fiscella K
      • Meldrum S
      • Franks P
      • et al.
      Patient trust: is it related to patient-centered behavior of primary care physicians?.
      In this study, the FF and SPs who participated in the OSCE case underwent similar training prior to the OSCE case. Despite consistent training and the use of a behaviorally anchored assessment tool that defined what was considered skill mastery, the differences in assessments were significant in three domains. This leads us to infer that the perspective of the assessor plays a role in how resident behavior is interpreted. There are various factors that possibly contribute to the differences seen. While FF may add authenticity that better reflects the reality that residents will face in the clinical environment, their personal experience of caring for an ill child may make them less objective. On the other hand, the SP has the unique vantage of being an active participant in the scenario, rather than a passive observer, which also adds authenticity and potentially contributes to the observed differences. We do not know personal information about the SPs, such as if they are parents themselves, potentially with children who are frequent utilizers of the health care system, which is a study limitation.
      This study supports the notion that multisource feedback is likely valuable in communication skills training. Interestingly, the three competencies that we found to be significantly different (Respect and Value, Information Sharing and Participation in Care and Decision Making) are rooted in a resident's ability to convey empathy and form therapeutic partnerships. We found that the most disparate behaviors within the three significant domains were a resident's ability to elicit a parent's thoughts on their child's symptoms or ask for the parent's input (FF assessed 33% of residents to meet skill mastery vs 69% as assessed by the SP) and the ability of a resident to communicate concern or intention to help (FF assessed 48% of residents to meet skill mastery vs 85% as assessed by the SP). These findings highlight the need to include empathy skill development in communication curriculum.
      This study has informed the development of our residency program communications curriculum. We include the perspective of FF in both the instructive and assessment aspects. The didactics are designed to teach the core concepts of family-centered care, to provide tools for building authentic parent-physician partnerships, and to continually expose residents to the parent perspective as a way to develop empathy. Every OSCE case in our curriculum utilizes multisource feedback from SPs, FF, and clinical faculty. The overwhelming majority of residents felt there was value in having feedback from multiple observers and that the perspective of the FF would be helpful to their future practice.
      More work needs to be done to see if including FF feedback when teaching residents communication skills ultimately impacts residents’ competency. This study was limited by small sample size and the single institution participation. Lastly, we acknowledge that FF programs and the use of an OSCE as an educational tool require resources that may not be readily available to all training programs. Many hospitals have Patient and Family Advisory Committees with volunteers who could potentially be trained to provide feedback to residents on their communication skills.

      Conclusion

      In conclusion, we demonstrated that FF assess pediatric residents differently than SPs in a communication skills OSCE case. FF provide a unique perspective that likely adds value to resident education by integrating the true stakeholders in parent-physician communication. More work needs to be done to fully understand the root of these differences and how best to integrate multisource feedback into communication skills training.

      Acknowledgments

      Financial statement: No funding support.
      Authorship statement: All authors have made substantial contributions to all of the following: 1) the conception and design of the study, or acquisition of data, or analysis and interpretation of data, 2) drafting the article or revising it critically for important intellectual content, and 3) final approval of the version to be submitted.
      Sala Institute for Child and Family Centered Care at Hassenfeld Children's Hospital New York Simulation Center (NYSIM) a public-private partnership between NYU Langone Health and the City University of New York.
      OSCE Case Authors: Cynthia J. Osman, MD, Linda R. Tewksbury, MD, Nicole L. Gerber, MD, Laura S. Schroeder. MD.

      Appendix. Supplementary Data

      References

        • Boissy A
        • Windover AK
        • Bokar D
        • et al.
        Communication skills training for physicians improves patient satisfaction.
        J Gen Intern Med. 2016; 31: 755-761
        • Brinkman WB
        • Geraghty SR
        • Lanphear BP
        • et al.
        Evaluation of resident communication skills and professionalism: a matter of perspective?.
        Pediatrics. 2006; 118: 1371-1379
      1. Accreditation Council for Graduate Medical Education. ACMGE program requirements for resident education in pediatrics. Available at: https://www.acgme.org/Specialties/Program-Requirements-and-FAQs-and-Applications/pfcatid/16/Pediatrics. Accessed June 7, 2021.

        • Reed S
        • Frey-Vogel A
        • Frost M.
        Look who's talking: a survey of pediatric program directors on communication skills education in pediatric residency programs.
        Acad Pediatr. 2020; 20: 9-13
        • Newcomb AB
        • Liu C
        • Trickey AW
        • et al.
        Patient perspectives of surgical residents' communication: do skills improve over time with a communication curriculum?.
        J Surg Educ. 2018; 75: e142-e149
        • Keir A
        • Wilkinson D.
        Communication skills training in paediatrics.
        J Paediatr Child Health. 2013; 49: 624-628
        • Morgan ER
        • Winter RJ.
        Teaching communication skills. An essential part of residency training.
        Arch Pediatr Adolesc Med. 1996; 150: 638-642
        • Peterson EB
        • Boland KA
        • Bryant KA
        • et al.
        Development of a comprehensive communication skills curriculum for pediatrics residents.
        J Grad Med Educ. 2016; 8: 739-746
        • Mallory LA
        • Calaman S
        • Lee White M
        • et al.
        Targeting simulation-based assessment for the pediatric milestones: a survey of simulation experts and program directors.
        Acad Pediatr. 2016; 16: 290-297
        • Talwalkar JS
        • Murtha TD
        • Prozora S
        • et al.
        Assessing advanced communication skills via objective structured clinical examination: a comparison of faculty versus self, peer, and standardized patient assessors.
        Teach Learn Med. 2020; 32: 294-307
        • Hanley K
        • Gillespie C
        • Zabar S
        • et al.
        Monitoring communication skills progress of medical students: establishing a baseline has value, predicting the future is difficult.
        Patient Educ Couns. 2019; 102: 309-315
        • Rassbach CE
        • Bogetz AL
        • Orlov N
        • et al.
        The effect of faculty coaching on resident attitudes, confidence, and patient-rated communication: a multi-institutional randomized controlled trial.
        Acad Pediatr. 2019; 19: 186-194
        • DeBlasio D
        • Real FJ
        • Ollberding NJ
        • et al.
        Provision of parent feedback via the communication assessment tool: does it improve resident communication skills?.
        Acad Pediatr. 2019; 19: 152-156
      2. Institute for Patient- and Family-Centered Care. Available at: https://www.ipfcc.org/. Accessed June 7, 2021.

        • Hanley K
        • Zabar S
        • Altshuler L
        • et al.
        Opioid vs nonopioid prescribers: variations in care for a standardized acute back pain case.
        Subst Abus. 2017; 38: 324-329
        • Yedidia MJ
        • Gillespie CC
        • Kachur E
        • et al.
        Effect of communications training on medical student performance.
        JAMA. 2003; 290: 1157-1165
        • Bogetz AL
        • Rassbach CE
        • Chan T
        • et al.
        Exploring the educational value of patient feedback: a qualitative analysis of pediatric residents' perspectives.
        Acad Pediatr. 2017; 17: 4-8
        • Brinkman WB
        • Geraghty SR
        • Lanphear BP
        • et al.
        Effect of multisource feedback on resident communication skills and professionalism: a randomized controlled trial.
        Arch Pediatr Adolesc Med. 2007; 161: 44-49
        • Dickter DN
        • Stielstra S
        • Lineberry M.
        Interrater reliability of standardized actors versus nonactors in a simulation based assessment of interprofessional collaboration.
        Simul Healthc. 2015; 10: 249-255
        • Rezaei R
        • Mehrabani G.
        A comparison of the scorings of real and standardized patients on physician communication skills.
        Pak J Med Sci. 2014; 30: 664-666
        • Fiscella K
        • Meldrum S
        • Franks P
        • et al.
        Patient trust: is it related to patient-centered behavior of primary care physicians?.
        Med Care. 2004; 42: 1049-1055