Advertisement

Validity of Faculty and Resident Global Assessment of Medical Students' Clinical Knowledge during their Pediatrics Clerkship

  • Robert A. Dudas
    Correspondence
    Address correspondence to Robert A. Dudas, MD, Johns Hopkins Bayview Medical Center, 4940 Eastern Ave, Baltimore, MD 21224.
    Affiliations
    Department of Pediatrics (Drs Dudas and Barone), Office of Medical Education (Dr Colbert), and Department of Surgery (Dr Goldstein), Johns Hopkins University School of Medicine, Baltimore, Md
    Search for articles by this author
  • Jorie M. Colbert
    Affiliations
    Department of Pediatrics (Drs Dudas and Barone), Office of Medical Education (Dr Colbert), and Department of Surgery (Dr Goldstein), Johns Hopkins University School of Medicine, Baltimore, Md
    Search for articles by this author
  • Seth Goldstein
    Affiliations
    Department of Pediatrics (Drs Dudas and Barone), Office of Medical Education (Dr Colbert), and Department of Surgery (Dr Goldstein), Johns Hopkins University School of Medicine, Baltimore, Md
    Search for articles by this author
  • Michael A. Barone
    Affiliations
    Department of Pediatrics (Drs Dudas and Barone), Office of Medical Education (Dr Colbert), and Department of Surgery (Dr Goldstein), Johns Hopkins University School of Medicine, Baltimore, Md
    Search for articles by this author
Published:November 04, 2011DOI:https://doi.org/10.1016/j.acap.2011.09.002

      Abstract

      Objective

      Medical knowledge is one of six core competencies in medicine. Medical student assessments should be valid and reliable. We assessed the relationship between faculty and resident global assessment of pediatric medical student knowledge and performance on a standardized test in medical knowledge.

      Methods

      Retrospective cross-sectional study of medical students on a pediatric clerkship in academic year 2008–2009 at one academic health center. Faculty and residents rated students’ clinical knowledge on a 5-point Likert scale. The inter-rater reliability of clinical knowledge ratings was assessed by calculating the intra-class correlation coefficient (ICC) for residents’ ratings, faculty ratings, and both rating types combined. Convergent validity between clinical knowledge ratings and scores on the National Board of Medical Examiners (NBME) clinical subject examination in pediatrics was assessed with Pearson product moment correlation correction and the coefficient of the determination.

      Results

      There was moderate agreement for global clinical knowledge ratings by faculty and moderate agreement for ratings by residents. The agreement was also moderate when faculty and resident ratings were combined. Global ratings of clinical knowledge had high convergent validity with pediatric examination scores when students were rated by both residents and faculty.

      Conclusions

      Our findings provide evidence for convergent validity of global assessment of medical students’ clinical knowledge with NBME subject examination scores in pediatrics.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Academic Pediatrics
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

      1. AAMC. Association of American Medical Colleges, Learning objectives for medical student education: Guidelines for medical schools. Available at: https://services.aamc.org/publications/showfile.cfm?file=version87.pdf&prd_id=198&prv_id=239&pdf_id=87. Accessed May 24, 2011.

      2. Accreditation Council for Graduate Medical Education. The ACGME Outcome Project: An introduction. Available at: http://www.acgme.org/outcome/project/timeline/TIMELINE_index_frame.htm. Accessed May 24, 2011.

        • Awad S.S.
        • Liscum K.R.
        • Aoki N.
        • et al.
        Does the Subjective Evaluation of Medical Student Surgical Knowledge Correlate with Written and Oral Exam Performance?.
        J Surg Res. 2002; 104: 36-39
        • Farrell T.M.
        • Kohn G.P.
        • Owen S.M.
        • et al.
        Low Correlation between Subjective and Objective Measures of Knowledge on Surgery Clerkships.
        J Am Coll Surg. 2010; 210: 680-683
        • Kolars J.C.
        • Mcdonald F.S.
        • Subhiyah R.G.
        • Edson R.S.
        Knowledge base evaluation of medicine residents on the gastroenterology service: Implications for competency assessments by faculty.
        Clinical Gastroenterology and Hepatology. 2003; 1: 64-68
        • Barlas D.
        • Ryan J.G.
        Does Correlation of Faculty Assessment of Emergency Medicine Residents’ Medical Knowledge Competency With Performance on the In-Training Examination Improve With Advancement Through the Program?.
        Ann Emerg Med. 2009; 54 (S33–S33)
        • Wise S.
        • Lynwood Stagg P.
        • Szucs R.
        • et al.
        Assessment of resident knowledge: Subjective assessment versus performance on the ACR in-training examination.
        Acad Radiol. 1999; 6: 66-71
      3. Eva KW, Regehr G. Self-Assessment in the Health Professions: A Reformulation and Research Agenda. Academic Medicine. RIME: Proceedings of the Forty-fourth Annual Conference November 6-9. 2005;80(10Supplement):S46–S54, October 2005.

        • Edwards R.K.
        • Kellner K.R.
        • Sistrom C.L.
        • Magyari E.J.
        Medical student self-assessment of performance on an obstetrics and gynecology clerkship.
        Obstet Gynecol. 2003; 188: 1078-1082
        • Weiss P.M.
        • Koller C.A.
        • Hess L.W.
        • Wasser T.
        How do medical student self-assessments compare with their final clerkship grades?.
        Med Teach. 2005; 27: 445-449
        • White C.B.
        • Waller J.L.
        • Freed G.
        • et al.
        The State of Undergraduate Pediatric Medical Education in North America: The COMSEP Survey.
        Teaching & Learning in Medicine. 2007; 19: 264-270
        • Portney L.G.
        • Watkins M.
        Foundations of clinical research: Applications to practice.
        Prentice Hall, New Jersey2000
        • Cohen J.
        “A power primer”.
        Psych Bulletin. 1992; 112: 155-159
        • Hoffman K.
        The USMLE, The NBME Subject Examinations, and Assessment of Individual Academic Achievement.
        Acad Med. 1993; 68: 740-747
        • Cox M.
        • Irby D.M.
        • Epstein R.M.
        Assessment in Medical Education.
        N Engl J Med. 2007; 356: 387-396