Advertisement

Scholarly Evaluation of Curricula and Educational Programs: Using a Systematic Approach to Produce Publishable Scholarship

      Abstract

      The goal of designing innovative curricula and programs in medical education is to create interventions that will change the attitudes, knowledge, skills, and behaviors of learners in order to prepare them to contribute to the health of patients and communities they serve. Systematic evaluation allows curriculum/program developers to optimize their curriculum/program and ensure that the goals of the curriculum/program are met. While curriculum/program evaluation is distinct from educational research, when approached in a rigorous manner, curriculum/program evaluation can be published as educational scholarship. The goal of our paper is to equip readers with tools to apply a scholarly approach to curriculum/program evaluation to produce findings that will advance the field. We start by defining curricula, educational programs, learner assessment, and evaluation. We then briefly describe examples of best practice models for curriculum/program development that incorporate the critical step of planning for evaluation. Building on published work, we distinguish the use of best practice models and conceptual frameworks to inform curriculum/program development and evaluation. More specifically, we outline steps to plan an evaluation that demonstrates WHAT was effective, describes HOW the curriculum/program contributed to the outcomes achieved, and points to WHY the curriculum/program led to the outcomes observed. We conclude with key considerations for publishing findings of an evaluation, including what to include in each section of a manuscript.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Academic Pediatrics
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Shaw I
        Qualitative Evaluation.
        Sage Publications, London; Thousand Oaks, Calif1999
        • Patton MQ
        Qualitative Research & Evaluation Methods: Integrating Theory and Practice.
        4th ed. SAGE Publications, Inc., Thousand Oaks, Calif2015
        • Stufflebeam DL
        • Coryn CLS
        Evaluation Theory, Models, and Applications.
        2nd ed. Jossey-Bass & Pfeiffer Imprints, Wiley, San Francisco2014
        • Shulman LS
        From Minsk to Pinsk: why a scholarship of teaching and learning?.
        J Sch Teach Learn. 2000; 1: 48-53
        • Abramson EL
        • Paul CR
        • Petershack J
        • et al.
        Conducting quantitative medical education research: from design to dissemination.
        Acad Pediatr. 2018; 18: 129-139
        • Hanson JL
        • Balmer DF
        • Giardino AP
        Qualitative research methods for medical educators.
        Acad Pediatr. 2011; 11: 375-386
        • Thomas PA
        • Kern DE
        • Hughes MT
        • et al.
        Curriculum Development for Medical Education: A Six-Step Approach.
        3rd ed. Johns Hopkins University Press, Baltimore2016
        • Patton MQ
        Utilization-Focused Evaluation.
        4th ed. Sage Publications, Thousand Oaks2008
        • Frye AW
        • Hemmer PA
        Program evaluation models and related theories: AMEE guide no. 67.
        Med Teach. 2012; 34: e288-e299
        • Gibbs T
        • Brigden D
        • Hellenberg D
        Assessment and evaluation in medical education.
        S Afr Fam Pract. 2006; 48: 5-7
      1. Program Performance and Evaluation Office at Centers for Disease Control and Prevention. CDC approach to evaluation-framework-program evaluation framework checklist for step 2—describe the program. Available at: https://www.cdc.gov/eval/steps/step2/index.htm. Accessed September 20, 2019.

        • Zackoff MW
        • Real FJ
        • Abramson EL
        • et al.
        Enhancing educational scholarship through conceptual frameworks: a challenge and roadmap for medical educators.
        Acad Pediatr. 2019; 19: 135-141
        • Bordage G
        Conceptual frameworks to illuminate and magnify.
        Med Educ. 2009; 43: 312-319
        • Varpio L
        • Paradis E
        • Uijtdehaage S
        • et al.
        The distinctions between theory, theoretical framework, and conceptual framework.
        Acad Med. 2020; 95: 989-994
        • Thomas PA
        • Kern DE
        • Hughes MT
        • et al.
        Curriculum Development for Medical Education: A Six-Step Approach.
        3rd ed. 2016 (Available at:)
        • Ericsson KA.
        Acquisition and maintenance of medical expertise: a perspective from the expert-performance approach with deliberate practice.
        Acad Med. 2015; 90: 1471-1486
        • Ericsson KA
        Peak: Secrets From the New Science of Expertise. 2016;
        • Moreau KA
        Has the new Kirkpatrick generation built a better hammer for our evaluation toolbox?.
        Med Teach. 2017; 39: 999-1001
        • Kirkpatrick JD
        • Kirkpatrick WK
        Kirkpatrick's Four Levels of Training Evaluation.
        ATD Press, Alexandria, Va2016
      2. World Health Organization. Towards more physical activity: transforming public spaces to promote physical activity—a key contributor to achieving the sustainable development goals in Europre. 2017. Available at:http://www.euro.who.int/__data/assets/pdf_file/0018/353043/2017_WHO_Report_FINAL_WEB.pdf?ua=1. Accessed May 31, 2020.

        • McChesney C
        • Covey S
        • Huling J
        The 4 Disciplines of Execution: Achieving Your Wildly Important Goals.
        1st Free Press hardcover ed. Free Press, New York2012
        • Li ST
        • Gusic ME
        • Vinci RJ
        • et al.
        A structured framework and resources to use to get your medical education work published.
        MedEdPORTAL: J Teach Learn Res. 2018; 14: 10669
        • Bhanji F
        • Gottesman R
        • de Grave W
        • et al.
        The retrospective pre-post: a practical method to evaluate learning from an educational program.
        Acad Emerg Med. 2012; 19: 189-194
        • Trowbridge RL
        • Snydman LK
        • Skolfield J
        • et al.
        A systematic review of the use and effectiveness of the objective structured teaching encounter.
        Med Teach. 2011; 33: 893-903
        • Haji F
        • Morin MP
        • Parker K
        Rethinking programme evaluation in health professions education: beyond ‘did it work?’.
        Med Educ. 2013; 47: 342-351
        • Cianciolo AT
        • Regehr G
        Learning theory and educational intervention: producing meaningful evidence of impact through layered analysis.
        Acad Med. 2019; 94: 789-794
        • Hutchings P
        • Shulman LS
        The scholarship of teaching: new elaborations, new developments.
        Change. 1999; 31: 10-15
        • Cleland JA
        • Durning SJ
        Researching Medical Education.
        John Wiley and Sons, Inc., Chichester, West Sussex2015
        • Shailaja S
        • Hilda SS
        • Pinto PA
        • et al.
        Evaluation of resident satisfaction and change in knowledge following use of high-fidelity simulation teaching for anaesthesia residents.
        Indian J Anaesth. 2019; 63: 908-914
        • Mirzazadeh A
        • Gandomkar R
        • Hejri SM
        • et al.
        Undergraduate medical education programme renewal: a longitudinal context, input, process and product evaluation study.
        Perspect Med Educ. 2016; 5: 15-23
      3. Durning Steven J. Carline Jan D. Review Criteria for Research Manuscripts. 2nd ed. Association of American Medical Colleges, Washington, DC2015