Conducting Quantitative Medical Education Research: From Design to Dissemination

Published:November 05, 2017DOI:


      Rigorous medical education research is critical to effectively develop and evaluate the training we provide our learners. Yet many clinical medical educators lack the training and skills needed to conduct high-quality medical education research. We offer guidance on conducting sound quantitative medical education research. Our aim is to equip readers with the key skills and strategies necessary to conduct successful research projects, highlighting new concepts and controversies in the field. We utilize Glassick's criteria for scholarship as a framework to discuss strategies to ensure that the research question of interest is worthy of further study and how to use existing literature and conceptual frameworks to strengthen a research study. Through discussions of the strengths and limitations of commonly used study designs, we expose the reader to particular nuances of these decisions in medical education research and discuss outcomes generally focused on, as well as strategies for determining the significance of consequent findings. We conclude with information on critiquing research findings and preparing results for dissemination to a broad audience. Practical planning worksheets and comprehensive tables illustrating key concepts are provided in order to guide researchers through each step of the process. Medical education research provides wonderful opportunities to improve how we teach our learners, to satisfy our own intellectual curiosity, and ultimately to enhance the care provided to patients.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Academic Pediatrics
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Klein M.D.
        • Li S.T.
        Building on the shoulders of giants: a model for developing medical education scholarship using I-PASS.
        Acad Pediatr. 2016; 16: 499-500
        • Levinson W.
        • Rubenstein A.
        Integrating clinician-educators into academic medical centers: challenges and potential solutions.
        Acad Med. 2000; 75: 906-912
        • Cook D.A.
        • Beckman T.J.
        • Bordage G.
        Quality of reporting of experimental studies in medical education: a systematic review.
        Med Educ. 2007; 41: 737-745
        • Castiglioni A.
        • Aagaard E.
        • Spencer A.
        • et al.
        Succeeding as a clinician educator: useful tips and resources.
        J Gen Intern Med. 2013; 28: 136-140
        • Glassick C.E.H.M.
        • Maeroff G.I.
        Scholarship Assessed—Evaluation of the Professoriate.
        Jossey-Bass, San Francisco, Calif1997
        • Yarris L.M.
        • Deiorio N.M.
        Education research: a primer for educators in emergency medicine.
        Acad Emerg Med. 2011; 18: S27-S35
        • Beckman T.J.
        • Cook D.A.
        Developing scholarly projects in education: a primer for medical teachers.
        Med Teach. 2007; 29: 210-218
        • Hanson J.L.
        • Balmer D.F.
        • Giardino A.P.
        Qualitative research methods for medical educators.
        Acad Pediatr. 2011; 11: 375-386
        • Jerardi K.E.
        • Mogilner L.
        • Turner T.
        • et al.
        Investment in faculty as educational scholars: outcomes from the national educational scholars program.
        J Pediatr. 2016; 171 (e1): 4-5
        • Association of American Medical Colleges
        Medical Education Research Certificate Program (MERC).
        (Available at)
        Date accessed: November 17, 2017
        • Trimm F.
        • Caputo G.
        • Bostwick S.
        • et al.
        Developing leaders in pediatric graduate medical education: the APPD LEAD Program.
        Acad Pediatr. 2015; 15: 143-146
        • Campbell D.M.
        • Barozzino T.
        • Farrugia M.
        • et al.
        High-fidelity simulation in neonatal resuscitation.
        Paediatr Child Health. 2009; 14: 19-23
        • Bordage G.
        Conceptual frameworks to illuminate and magnify.
        Med Educ. 2009; 43: 312-319
        • Sullivan G.M.
        • Simpson D.
        • Cook D.A.
        • et al.
        Redefining quality in medical education research: a consumer's view.
        J Grad Med Educ. 2014; 6: 424-429
        • Bandura A.
        Social Foundations of Thought and Action: A Social Cognitive Theory.
        Prentice-Hall, Englewood Cliffs, NJ1986
        • Young H.N.
        • Schumacher J.B.
        • Moreno M.A.
        • et al.
        Medical student self-efficacy with family-centered care during bedside rounds.
        Acad Med. 2012; 87: 767-775
        • Ericsson K.A.
        Acquisition and maintenance of medical expertise: a perspective from the expert-performance approach with deliberate practice.
        Acad Med. 2015; 90: 1471-1486
        • Hunt E.A.
        • Duval-Arnould J.M.
        • Nelson-McMillan K.L.
        • et al.
        Pediatric resident resuscitation skills improve after “rapid cycle deliberate practice” training.
        Resuscitation. 2014; 85: 945-951
        • Kolb D.A.
        Experiential Learning: Experience as the Source of Learning and Development.
        Prentice-Hall, Upper Saddle River, NJ1984
        • Klein M.
        • Vaughn L.M.
        Teaching social determinants of child health in a pediatric advocacy rotation: small intervention, big impact.
        Med Teach. 2010; 32: 754-759
        • Ericsson K.A.K.R.
        • Tesch-Römer C.
        The role of deliberate practice in the acquisition of expert performance.
        Psychol Rev. 1993; 100: 363-406
        • Centre for Evidence-Based Medicine
        Asking focused questions.
        (Oxford, UK: Nuffield Department of Primary Care Health Science; n.d.; Available at)
        Date accessed: November 17, 2017
        • Doran G.T.
        There's a SMART way to write management's goals and objectives.
        Manage Rev. 1981; 70: 35-36
        • Kirkpatrick D.
        Great ideas revisited. Techniques for evaluating training programs. Revisiting Kirkpatrick's four-level model.
        Technol Dev. 1996; 50: 54-59
        • Hostetter M.
        • Klein S.
        In Focus: Using behavioral economics to advance population health and improve the quality of health care services.
        (New York, NY: Commonwealth Fund; Available at)
        • Davis D.
        • O'Brien M.A.
        • Freemantle N.
        • et al.
        Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes?.
        JAMA. 1999; 282: 867-874
        • Agency for Healthcare Research and Quality
        • Innovations Exchange Team
        Sustaining and spreading quality improvement.
        (Available at)
        • Mayer-Mihalski N.
        • DeLuca M.J.
        Effective education leading to behavior change.
        (Available at)
        • Keune J.D.
        • Brunsvold M.E.
        • Hohmann E.
        • et al.
        The ethics of conducting graduate medical education research on residents.
        Acad Med. 2013; 88: 449-453
        • Miser W.F.
        Educational research—to IRB, or not to IRB?.
        Fam Med. 2005; 37: 168-173
        • Kraus C.K.
        • Guth T.
        • Richardson D.
        • et al.
        Ethical considerations in education research in emergency medicine.
        Acad Emerg Med. 2012; 19: 1328-1332
        • Downing S.M.
        Validity: on meaningful interpretation of assessment data.
        Med Educ. 2003; 37: 830-837
        • Rickards G.
        • Magee C.
        • Artino Jr, A.R.
        You can't fix by analysis what you've spoiled by design: developing survey instruments and collecting validity evidence.
        J Grad Med Educ. 2012; 4: 407-410
        • Windish D.M.
        • Diener-West M.
        A clinician-educator's roadmap to choosing and interpreting statistical tests.
        J Gen Intern Med. 2006; 21: 656-660
        • du Prel J.B.
        • Hommel G.
        • Rohrig B.
        • et al.
        Confidence interval or P-value? Part 4 of a series on evaluation of scientific publications.
        Dtsch Arztebl Int. 2009; 106: 335-339
        • Cummings P.
        • Rivara F.P.
        Reporting statistical information in medical journal articles.
        Arch Pediatr Adolesc Med. 2003; 157: 321-324
        • Gelman A.
        • Hill J.
        • Yajima M.
        Methodological studies: why we (usually) don't have to worry about multiple comparisons.
        J Res Educ Eff. 2012; 5: 189-211
        • Schwartz A.
        • Young R.
        • Hicks P.J.
        Medical education practice-based research networks: Facilitating collaborative research.
        Med Teach. 2016; 38: 64-74
        • Abramson E.L.
        • Naifeh M.M.
        • Stevenson M.D.
        • et al.
        Research training among pediatric residency programs: a national assessment.
        Acad Med. 2014; 89: 1674-1680
        • Frintner M.P.
        • Liebhart J.L.
        • Lindros J.
        • et al.
        Are graduating pediatric residents prepared to engage in obesity prevention and treatment?.
        Acad Pediatr. 2016; 16: 394-400
        • Laraque-Arena D.
        • Frintner M.P.
        • Cull W.L.
        Underserved areas and pediatric resident characteristics: is there reason for optimism?.
        Acad Pediatr. 2016; 16: 401-410
        • Durning S.J.
        • Carline J.D.
        AAMC Review Criteria for Research Manuscripts.
        2nd ed. Association of American Medical Colleges, Washington, DC2015 (Available at)
        • Li S.T.K.M.
        • Gusic M.
        • Vinci R.
        • et al.
        Crossing the finish line: getting your medical education work published.
        (Paper presented at:; Pediatric Academic Societies Workshop)2016 (Baltimore, Md; May)
        • Meyer H.S.
        • Durning S.J.
        • Sklar D.
        • et al.
        Making the first cut: an analysis of academic medicine editors' reasons for not sending manuscripts out for external peer review.
        Acad Med. 2017;
        • Reed D.A.
        • Beckman T.J.
        • Wright S.M.
        • et al.
        Predictive validity evidence for medical education research study quality instrument scores: quality of submissions to JGIM's Medical Education special issue.
        J Gen Intern Med. 2008; 23: 903-907
        • Moreau K.A.
        Has the new Kirkpatrick generation built a better hammer for our evaluation toolbox?.
        Med Teach. 2017; 39: 999-1001