Advertisement

The Impact of a Gamified Curriculum Using Kahoot! on Musculoskeletal Knowledge and Skill Acquisition Among Pediatric Residents

  • Katherine Schultz
    Correspondence
    Address correspondence to Katherine Schultz, MD, Division of Pediatric Rheumatology, Allergy, and Immunology, Department of Pediatrics, University of Iowa Stead Family Children's Hospital, 200 Hawkins Dr, Iowa City, IA, 52242
    Affiliations
    Division of Pediatric Rheumatology, Allergy and Immunology, Department of Pediatrics (K Schultz), University of Iowa Stead Family Children's Hospital, Iowa City, Iowa
    Search for articles by this author
  • Melissa Klein
    Affiliations
    Department of Pediatrics (M Klein, H Sucharew, D DeBlasio, S Poynter, J Huggins, FJ Real), University of Cincinnati College of Medicine, Cincinnati, Ohio

    Division of General and Community Pediatrics, Department of Pediatrics (M Klein, D DeBlasio, E Cooperstein, FJ Real), Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio
    Search for articles by this author
  • Heidi Sucharew
    Affiliations
    Department of Pediatrics (M Klein, H Sucharew, D DeBlasio, S Poynter, J Huggins, FJ Real), University of Cincinnati College of Medicine, Cincinnati, Ohio

    Division of Biostatistics and Epidemiology, Department of Pediatrics (H Sucharew), Children's Hospital Medical Center, Cincinnati, Ohio
    Search for articles by this author
  • Joseph McDonald
    Affiliations
    Division of Pediatric Rheumatology, Department of Pediatrics (J McDonald), University of Chicago, Chicago, Ill
    Search for articles by this author
  • Dominick DeBlasio
    Affiliations
    Department of Pediatrics (M Klein, H Sucharew, D DeBlasio, S Poynter, J Huggins, FJ Real), University of Cincinnati College of Medicine, Cincinnati, Ohio

    Division of General and Community Pediatrics, Department of Pediatrics (M Klein, D DeBlasio, E Cooperstein, FJ Real), Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio
    Search for articles by this author
  • Emily Cooperstein
    Affiliations
    Division of General and Community Pediatrics, Department of Pediatrics (M Klein, D DeBlasio, E Cooperstein, FJ Real), Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio
    Search for articles by this author
  • Sue Poynter
    Affiliations
    Department of Pediatrics (M Klein, H Sucharew, D DeBlasio, S Poynter, J Huggins, FJ Real), University of Cincinnati College of Medicine, Cincinnati, Ohio

    Division of Pediatric Critical Care Medicine, Department of Pediatrics (S Poynter), Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio
    Search for articles by this author
  • Jennifer Huggins
    Affiliations
    Department of Pediatrics (M Klein, H Sucharew, D DeBlasio, S Poynter, J Huggins, FJ Real), University of Cincinnati College of Medicine, Cincinnati, Ohio

    Division of Pediatric Rheumatology, Department of Pediatrics (J Huggins), Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio
    Search for articles by this author
  • Francis J. Real
    Affiliations
    Department of Pediatrics (M Klein, H Sucharew, D DeBlasio, S Poynter, J Huggins, FJ Real), University of Cincinnati College of Medicine, Cincinnati, Ohio

    Division of General and Community Pediatrics, Department of Pediatrics (M Klein, D DeBlasio, E Cooperstein, FJ Real), Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio
    Search for articles by this author
Published:February 12, 2022DOI:https://doi.org/10.1016/j.acap.2022.02.003

      Abstract

      Objective

      To determine whether a musculoskeletal curriculum involving gamification via Kahoot! (an online classroom response system) was acceptable and more effective at teaching pediatric residents musculoskeletal knowledge and skills than a nongamified curriculum.

      Methods

      A prospective, randomized controlled trial was conducted at an urban, academic pediatric clinic. All participants received a curriculum that included brief didactics and knowledge questions. The knowledge questions were delivered via Kahoot! to the intervention group and administered via paper to the control group. The primary outcome was knowledge and skill acquisition following curriculum participation.

      Results

      A total of 73 of 85 (86%) residents completed the study (intervention group: 46; control group: 27). Following participation in the curriculum, intervention and control residents demonstrated an improvement in musculoskeletal knowledge (P < .05) measured via questionnaire, as well as an improvement in physical exam skills during a standardized patient encounter (P < .05). There was no difference in knowledge or skill improvement between groups. Intervention participants indicated positive attitudes toward Kahoot!.

      Conclusions

      Our musculoskeletal curriculum demonstrated improvements in knowledge and skills among residents, though inclusion of Kahoot! did not enhance the experimental effect. Further research is needed to identify strategies to optimize gamification for learning.

      Keywords

      What's New
      We describe the creation, implementation, and evaluation of a new curriculum using gamification via Kahoot! to teach musculoskeletal knowledge and skills to pediatric residents. Knowledge and skills improved in gamified and non-gamified groups, and residents reported positive attitudes towards Kahoot!
      Pediatricians report low confidence assessing musculoskeletal (MSK) complaints, with only half of pediatric trainees recalling prior pediatric MSK education during their medical education.
      • Jandial S
      • Myers A
      • Wise E
      • et al.
      Doctors likely to encounter children with musculoskeletal complaints have low confidence in their clinical skills.
      Deficient chart documentation for MSK complaints during clinical care
      • Myers A
      • McDonagh JE
      • Gupta K
      • et al.
      More “cries from the joints”: assessment of the musculoskeletal system is poorly documented in routine paediatric clerking.
      and suboptimal performance on knowledge exams
      • Haywood BL
      • Porter SL
      • Grana WA
      Assessment of musculoskeletal knowledge in primary care residents.
      underscore current gaps in MSK knowledge (ie, the understanding of MSK pathology) and skills (ie, history taking, physical examination, and use of diagnostic reasoning). With MSK complaints comprising 10% to 30% of pediatric primary care visits,
      • De Inocencio J
      Epidemiology of musculoskeletal pain in primary care.
      adequate training of future pediatricians in MSK knowledge and skills is critical to promote timely recognition of abnormalities requiring subspecialty evaluation.
      Arthralgia, defined as joint pain, comprises 42% of the MSK complaints presenting to pediatricians.
      • De Inocencio J
      Epidemiology of musculoskeletal pain in primary care.
      Arthralgia without arthritis (joint inflammation) is often incorrectly interpreted as evidence of potential rheumatic disease, resulting in avoidable referrals to pediatric rheumatologists.
      • McGhee JL
      • Burks FN
      • Sheckels JL
      • et al.
      Identifying children with chronic arthritis based on chief complaints: absence of predictive value for musculoskeletal pain as an indicator of rheumatic disease in children.
      Prior training for pediatric residents on MSK skills has been time-intensive and costly,
      • Day CS
      • Ho P
      Progress of medical school musculoskeletal education in the 21st century.
      necessitating a time-efficient, easily implementable, and effective curricula to train pediatricians in MSK skills. Gamification, defined as the use of gaming elements (ex. score boards, points system) in a non-gaming environment,
      • Rutledge C
      • Walsh CM
      • Swinger N
      • et al.
      Gamification in action.
      ,
      • Pitt MB
      • Borman-Shoap EC
      • Eppich WJ
      Twelve tips for maximizing the effectiveness of game-based learning.
      is a potential educational strategy. Gamification been shown to be an effective means of teaching clinical content by increasing levels of engagement, motivation, and enjoyment.
      • Pitt MB
      • Borman-Shoap EC
      • Eppich WJ
      Twelve tips for maximizing the effectiveness of game-based learning.
      • Kerfoot BP
      • Armstrong EG
      • O'Sullivan PN
      Interactive spaced-education to teach the physical examination: a randomized controlled trial.
      • Nevin CR
      • Westfall AO
      • Martin Rodriguez J
      • et al.
      Gamification as a tool for enhancing graduate medical education.
      • Hamari J
      • Koivisto J
      • Sarsa H
      Does gamification work? - A literature review of empirical studies on gamification.
      Here, we describe a curriculum entitled, MSK Gamification for Residents in Pediatrics (MGRIP), a novel intervention to train pediatric residents to differentiate arthralgia from arthritis using Kahoot!,

      Kahoot! Available at: https://create.kahoot.it/. Accessed August 15, 2019

      an online, game-based classroom response system that allows learners to answer questions/prompts via smartphone devices with Kahoot! software interpreting and aggregating responses in real time.
      Premarathne PBTK
      A study on incorporating gamification into ESL classroom via Kahoot!.
      Kahoot! harnesses Social Facilitation Theory whereby having an audience motivates and compels the learner to perform better.
      • Sanders GS
      Driven by distraction: an integrative review of social facilitation theory and research.
      While different models have been proposed to explain the mechanism of action behind Social Facilitation Theory, the predominating feature is that a learner's arousal when completing a task is increased by the presence of others if the complexity of the task is appropriate.
      • Strauss B
      Social facilitation in motor tasks: a review of research and theory.
      Kahoot! has been successfully used in undergraduate medical education to augment knowledge acquisition,
      • Neureiter D
      • Klieser E
      • Neumayer B
      • et al.
      Feasibility of Kahoot! as a real-time assessment tool in (Histo-)pathology classroom teaching.
      though its impact on training in general pediatrics has not been evaluated. This study evaluated the efficacy of MGRIP to enhance pediatric residents’ MSK knowledge and skills. We hypothesized that residents exposed to the gamified version of MGRIP, which incorporated Kahoot!, would demonstrate increased knowledge and skills when compared to a control group that did not incorporate gamified elements.

      Methods

      Study Design

      We utilized a prospective, randomized control design comparing a gamified version of MGRIP to a non-gamified version. All participants received the standard MGRIP curriculum (included brief didactics and knowledge questions). The intervention group completed the knowledge questions via Kahoot! while the control group utilized paper. The primary outcome was knowledge and skill acquisition following MGRIP participation.

      Setting and Participants

      MGRIP occurred at the Cincinnati Children's Hospital Medical Center (CCHMC) Pediatric Primary Care Center (PPCC), an urban academic clinic that serves as continuity clinic for ∼80 pediatric residents. All CCHMC pediatric residents (post-graduate years [PGY] 1-4) with continuity clinic at PPCC were eligible to participate. Residents were randomized into control or intervention groups based on previously assigned continuity clinic day. This study was determined exempt by the CCHMC Institutional Review Board.

      MGRIP Curriculum

      MGRIP included 3 live, 15-minute sessions that occurred during a pre-existing preclinic teaching time. Each didactic session was facilitated by a pediatric rheumatology fellow (KS) and included 3-4 knowledge questions to reinforce key concepts. Three to 10 residents participated in MGRIP during each clinic session. The first didactic used case examples to describe key components of the MSK history and physical exam to define distinctions between arthralgia and arthritis. The second didactic highlighted key differences in historical and physical exam findings to identify causes of arthralgia, including hypermobility, pain syndromes, and somatic manifestations of mood disorders. The third didactic included 3 knowledge questions that reviewed content previously presented. For intervention participants, questions were delivered via Kahoot! which allowed residents to view questions on a large screen, respond real-time on their smartphones, display their answer selections anonymously, and receive immediate feedback. After each question, intervention residents were ranked and could see their standing against others on a leaderboard, with a winner declared at the end of each session. Kahoot! also included audio-visual stimuli to promote user engagement. For control residents, the same questions and answers were delivered via paper but without point tracking and audio-visual stimuli.

      Study Overview

      MGRIP was implemented September-October 2020. All participating residents completed a survey with MSK knowledge and demographic (eg, gender, age, PGY level) questions and participated in a standardized patient encounter 1 month prior to curriculum implementation. In February 2021, the MSK knowledge questions and standardized patient encounter were repeated. Intervention residents also completed an attitude formation survey. Primary outcome measurements included change in knowledge and skill following MGRIP participation.

      Outcome Metrics

      Knowledge acquisition was measured via a 20-item multiple choice questionnaire (MSQ) administered before (MSQ1) and after (MSQ2) the MGRIP Intervention. Questions were based on the American Board of Pediatrics Content Outline

      General Pediatrics Content Outline The American Board of Pediatrics. Available at: https://www.abp.org/become-certified/about-our-certifying-exams/general-pediatrics-content-outline. Accessed August 27, 2019

      and on questions from the American Academy of Pediatrics PREP Self-Assessment.

      American Academy of Pediatrics. PREP self-assessments. Available at: https://shop.aap.org/professional-education/self-assessments/. Accessed February 24, 2020

      Questions were divided into 4 domains for sub-analysis: History, Physical Exam, Causes of Arthralgia, and Referrals. Content validity evidence was established through iterative review by experts in pediatric rheumatology, general pediatrics, and medical education. Question order was randomized at the 2 time points to decrease recall bias. Residents were not provided answers until all participants had completed MSQ2.
      Skill acquisition was measured via standardized patient encounters using an observation checklist (SPEC) derived from an existing tool entitled “pediatric Gait, Arms, Legs, and Spine” (pGALs),
      • Goff I
      • Bateman B
      • Myers A
      • et al.
      Acceptability and practicality of musculoskeletal examination in acute general pediatric assessment.
      • Foster HE
      • Kay LJ
      • Friswell M
      • et al.
      Musculoskeletal screening examination (pGALS) for school-age children based on the adult GALS screen.

      Matters PM. pGALS Checklist. Published 2021. Available at: http://www.pmmonline.org/page.aspx?id=1624. Accessed July 25, 2019

      a screening tool for evaluating acute MSK complaints in children. It has demonstrated excellent sensitivity and specificity for identifying joint abnormalities and has established acceptability by practitioners and patients as it can be completed within 2 minutes.
      • Goff I
      • Bateman B
      • Myers A
      • et al.
      Acceptability and practicality of musculoskeletal examination in acute general pediatric assessment.
      ,
      • Foster HE
      • Kay LJ
      • Friswell M
      • et al.
      Musculoskeletal screening examination (pGALS) for school-age children based on the adult GALS screen.
      The observation checklist was divided into 2 domains: History and Physical Exam. The maximum checklist score possible was 30 points. The knee exam was used to evaluate MSK skills as knee pain is a frequent MSK complaint, and evaluation includes well-recognized maneuvers

      Matters PM. pGALS Checklist. Published 2021. Available at: http://www.pmmonline.org/page.aspx?id=1624. Accessed July 25, 2019

      described in the MGRIP curriculum. All residents completed 2 standardized patient encounters. The first consisted of a complaint of left knee arthralgia in the setting of hypermobility, while the second consisted of a complaint of left knee arthralgia in the setting of inflammatory arthritis.
      Intervention residents’ attitudes toward gamified elements of MGRIP were assessed using a 5-point Likert scale (strongly disagree to strongly agree) questionnaire (Appendix, Figure 1). Questions pertained to the competitiveness, anonymity, efficacy, and motivation to learn when using Kahoot!. Items were modified from a prior questionnaire of undergraduate students attitudes towards Kahoot!.
      • Lin DTA
      • Ganapathy M
      • Kaur M
      Kahoot! it: gamification in higher education.

      Data Analysis

      Categorical demographic data were presented as number and percentage. Differences between intervention and control groups were evaluated using chi-square tests. MSQ and SPEC scores were summarized as mean (standard deviation). We employed a paired T test to evaluate differences within groups on MSQ and SPEC, and an independent T test to evaluate differences between groups. Secondary analyses used a paired T test to evaluate differences of the MSQ Domains and the SPEC Components within groups, and an independent T test evaluating these differences between groups. Statistical significance was established at p < 0.05. All analyses were conducted using SAS software, version 9.4 (SAS Institute, NC).

      Results

      Seventy-three (86%) of 85 eligible residents participated with 46 residents in the intervention group and 27 in the control group. The majority were female and Caucasian, with no significant differences between groups (Table 1).
      Table 1Demographic Description of Residents in the Intervention and Control Groups
      Intervention Group

      n = 46 (%)
      Control Group

      n = 26
      One participant did not complete demographic survey.
      (%)
      P Value
      Post Graduate Year0.97
       115 (33)9 (35)
       216 (35)10 (38)
       314 (30)7 (27)
       41 (2)0
      Gender Identity0.87
       Male15 (33)8 (31)
       Female31 (67)18 (69)
      Race0.49
       White/Caucasian33 (72)19 (73)
       Black/African American3 (7)1 (4)
       Other8 (17)3 (12)
       Prefer not to answer2 (4)3 (12)
      Between groups analyzed with chi-square test.
      low asterisk One participant did not complete demographic survey.
      In terms of MSK knowledge, both intervention and control groups demonstrated a statistically significant increase in scores following MGRIP participation (Table 2). Intervention residents’ scores increased from a mean of 71.5% to 79.6% (P < .001), and control residents’ scores increased from a mean of 68.7% to 77.4% (P = .02). The Physical Exam Domain demonstrated a statistically significant improvement among intervention residents only (from 54.4% to 66.8%, P = .005). The change in knowledge between groups did not demonstrate a statistically significant difference (P = .38).
      Table 2Differences in the Knowledge Questionnaire (MSQ) and the Skills Checklist (SPEC) Between Groups Pre- and Post-MGRIP Curriculum
      Preintervention Mean (SD)Postintervention Mean (SD)Within Group P Value
      Knowledge Results - MSQTotalTotal Sum Score (%)
       Intervention Group71.5 (13.5)79.6 (9.0)<.0001
      denotes significant results (P < .05). Within groups were analyzed with paired T test.
       Control Group68.7 (14.2)77.4 (11.1).02
      denotes significant results (P < .05). Within groups were analyzed with paired T test.
      DomainsHistory Based Questions
       Intervention Group74.5 (18.9)81.0 (19.0).16
       Control Group72.6 (20.9)73.9 (17.5).72
      Physical Exam Based Questions
       Intervention Group54.4 (22.4)66.8 (16.5).005
      denotes significant results (P < .05). Within groups were analyzed with paired T test.
       Control Group51.1 (17.8)63.5 (20.6).07
      Causes of Arthralgia Questions
       Intervention Group78.3 (17.6)85.0 (12.8).09
       Control Group78.4 (19.0)87.7 (16.1).2
      Referrals Questions
       Intervention Group77.9 (22.6)86.0 (15.9).03
      denotes significant results (P < .05). Within groups were analyzed with paired T test.
       Control Group71.3 (24.7)83.7 (16.2).02
      denotes significant results (P < .05). Within groups were analyzed with paired T test.
      Skills Results - SPECTotalTotal Sum Score (points)
       Intervention Group12.5 (2.8)13.1 (3.3).23
       Control Group11.9 (3.6)13.4 (2.5).11
      ComponentsHistory
       Intervention Group9.5 (2.1)9.1 (2.2).78
       Control Group9.7 (3.0)9.4 (1.5).65
      Physical Exam
       Intervention Group3.0 (1.5)4.0 (2.4).03
      denotes significant results (P < .05). Within groups were analyzed with paired T test.
       Control Group2.3 (1.4)4.0 (2.5).002
      denotes significant results (P < .05). Within groups were analyzed with paired T test.
      SD indicates standard deviation.
      The number of participants with data available for:
      Knowledge Assessment - Intervention: pre n = 43, post n = 41, paired pre-post n = 39; Control: pre n = 27, post n = 23, paired pre-post n = 23;
      Skills Assessment - Intervention: pre n = 46, post n = 38, paired pre-post n = 38; Control: pre n = 27, post n = 25, paired pre-post n = 22;
      low asterisk denotes significant results (P < .05). Within groups were analyzed with paired T test.
      In terms of MSK skills, neither group demonstrated a statistically significant increase in total scores following MGRIP participation (P = .23 intervention, P = .11 control) (Table 2). On the Physical Exam Component of the checklist, residents’ scores statistically improved with Intervention residents’ scores increasing from 3 to 4 points (P = .03) and control residents increasing 2.3 to 4 points (P = .002). However, skill changes between groups did not demonstrate a difference (P = .62).
      In terms of attitudes toward gamified learning, 28/46 (61%) of intervention residents completed the questionnaire. The majority indicated that Kahoot! was an effective teaching tool (89% agreement) that should be used in residency education (86% agreement). Residents also agreed that Kahoot! augmented learning by elevating motivation (76% agreement) and attention (89% agreement) through encouraging competitiveness (96% agreement). Finally, residents responded that the anonymity of the sessions encouraged their participation (100%) and decreased the stress of answering incorrectly (93%).

      Discussion

      Pediatric residents in both groups demonstrated a significant improvement in MSK knowledge and physical exam skills following participation in MGRIP. There was no difference in knowledge or skill improvement between groups; the incorporation of gamification did not enhance the effect as hypothesized. Despite this, residents did indicate acceptability of the Kahoot! platform.
      Our findings confirm prior evidence describing residents’ challenges related to MSK knowledge and skills.
      • Jandial S
      • Myers A
      • Wise E
      • et al.
      Doctors likely to encounter children with musculoskeletal complaints have low confidence in their clinical skills.
      ,
      • Haywood BL
      • Porter SL
      • Grana WA
      Assessment of musculoskeletal knowledge in primary care residents.
      We found that residents’ baseline knowledge and skills assessed prior to curriculum implementation left room for improvement on both the MSQ and SPEC. In addition, baseline residents’ scores for knowledge and skill assessments were higher for items related to history when compared to physical exam items. This finding aligns with a prior study that demonstrated that pediatric residents generally perform better on knowledge and skill assessment related to MSK history than MSK physical exam.
      • Hergenroeder AC
      • Chorley JN
      • Laufman L
      • et al.
      Pediatric residents’ performance of ankle and knee examinations after an educational intervention.
      Despite its short duration, MGRIP did result in overall improvement in both MSK knowledge and physical exam skills, potentially filling a gap for MSK training.
      Our study did not find a statistical difference between the intervention group that utilized Kahoot! and the control group. This may be partly due to the short exposure duration and the task complexity according to Social Facilitation Theory.
      • Sanders GS
      Driven by distraction: an integrative review of social facilitation theory and research.
      The exposure duration occurred over 8 weeks, whereas prior studies have had exposure periods to gamification spanning up to a year.
      • Nevin CR
      • Westfall AO
      • Martin Rodriguez J
      • et al.
      Gamification as a tool for enhancing graduate medical education.
      Longer duration of exposure may guide learners to more thoroughly identify their knowledge deficits, allowing for self-reflection and assessment, thus ultimately enhancing knowledge gains.
      • Ismail MAA
      • Ahmad A
      • Mohammad JAM
      • et al.
      Using Kahoot! as a formative assessment tool in medical education: a phenomenological study.
      The role of task complexity on motivation is one of balance: the presence of others necessarily results in splitting of learners’ attention, which increases awareness in less complex tasks.
      • Baron RS
      Distraction-conflict theory: progress and problems.
      However, in more complex tasks, the attention split becomes a distractor. In our case, curricular material was new to many residents, requiring a substantial portion of their attention to assimilate, perhaps limiting the potential impact of gamification.
      Our findings also support existing evidence that Kahoot! is acceptable, with the responses on the attitude formation survey largely favorable. Indeed, we found that that majority of residents found Kahoot! effective as a teaching tool that encouraged competitiveness and motivation. Although not assessed in our curriculum, prior literature supports use of Kahoot! by instructors who found the platform improved learners’ participation with more timely feedback.
      • Calle S
      • Bonfante E
      • Riascos R
      Introduction of the game- based learning platform, Kahoot, as a tool in radiology resident training.
      ,
      • Donkin R
      • Rasmussen R.
      Student perception and the effectiveness of Kahoot!: a scoping review in histology, anatomy, and medical education.
      As one of the first studies to assess Kahoot!’s ability to impact skill acquisition among residents,
      • Neureiter D
      • Klieser E
      • Neumayer B
      • et al.
      Feasibility of Kahoot! as a real-time assessment tool in (Histo-)pathology classroom teaching.
      ,
      • Öz GÖ
      • Ordu Y
      The effects of web based education and Kahoot usage in evaluation of the knowledge and skills regarding intramuscular injection among nursing students.
      our work may provide a foundation on which to explore Kahoot!’s efficacy in other areas of medical training.
      There are several limitations to this study. First, it was conducted at a single site in a large academic setting; this may affect the generalizability. Second, there was limited availability of relevant assessment instruments for this work, which we attempted to overcome by using or modifying instruments with previously established validity evidence. Lastly, we were not able to relate the results to clinical outcomes. While residents did improve on both MSQ and SPEC, it's uncertain how improvement in these scores relates to alterations in residents’ clinical practice and, if so, how much change in score is necessary to result in those alterations.
      Despite these limitations, this study supports the importance of MSK curricula in pediatric residency. Given residents’ positive attitudes toward our gamified approach, further research is warranted to understand what “dosage” of gamification or content areas might be best suited for its inclusion.

      Acknowledgments

      Financial statement: This project was supported by the Association of Pediatric Program Directors (APPD). Role of funding/support: APPD and CCHMC did not have a role in the design of the study, collection or analysis of data, or the decision to create and submit an article for publication.

      References

        • Jandial S
        • Myers A
        • Wise E
        • et al.
        Doctors likely to encounter children with musculoskeletal complaints have low confidence in their clinical skills.
        J Pediatr. 2009; 154: 267-271https://doi.org/10.1016/j.jpeds.2008.08.013
        • Myers A
        • McDonagh JE
        • Gupta K
        • et al.
        More “cries from the joints”: assessment of the musculoskeletal system is poorly documented in routine paediatric clerking.
        Rheumatology. 2004; 43: 1045-1049https://doi.org/10.1093/rheumatology/keh245
        • Haywood BL
        • Porter SL
        • Grana WA
        Assessment of musculoskeletal knowledge in primary care residents.
        Am J Orthop (Belle Mead NJ). 2006; 35 (Available at:) (Accessed March 31, 2021): 273-275
        • De Inocencio J
        Epidemiology of musculoskeletal pain in primary care.
        Arch Dis Child. 2004; 89: 431-434https://doi.org/10.1136/adc.2003.028860
        • McGhee JL
        • Burks FN
        • Sheckels JL
        • et al.
        Identifying children with chronic arthritis based on chief complaints: absence of predictive value for musculoskeletal pain as an indicator of rheumatic disease in children.
        Pediatrics. 2002; 110: 354-359https://doi.org/10.1542/peds.110.2.354
        • Day CS
        • Ho P
        Progress of medical school musculoskeletal education in the 21st century.
        J Am Acad Orthop Surg. 2016; 24: 762-768https://doi.org/10.5435/JAAOS-D-15-00577
        • Rutledge C
        • Walsh CM
        • Swinger N
        • et al.
        Gamification in action.
        Acad Med. 2018; 93: 1014-1020https://doi.org/10.1097/ACM.0000000000002183
        • Pitt MB
        • Borman-Shoap EC
        • Eppich WJ
        Twelve tips for maximizing the effectiveness of game-based learning.
        Med Teach. 2015; 37: 1013-1017https://doi.org/10.3109/0142159X.2015.1020289
        • Kerfoot BP
        • Armstrong EG
        • O'Sullivan PN
        Interactive spaced-education to teach the physical examination: a randomized controlled trial.
        J Gen Intern Med. 2008; 23: 973-978https://doi.org/10.1007/s11606-008-0533-0
        • Nevin CR
        • Westfall AO
        • Martin Rodriguez J
        • et al.
        Gamification as a tool for enhancing graduate medical education.
        Postgrad Med J. 2014; 90: 685-693https://doi.org/10.1136/postgradmedj-2013-132486
        • Hamari J
        • Koivisto J
        • Sarsa H
        Does gamification work? - A literature review of empirical studies on gamification.
        in: Proceedings of the Annual Hawaii International Conference on System Sciences. IEEE, 2014: 3025-3034https://doi.org/10.1109/HICSS.2014.377
      1. Kahoot! Available at: https://create.kahoot.it/. Accessed August 15, 2019

        • Premarathne PBTK
        A study on incorporating gamification into ESL classroom via Kahoot!.
        in: Int Conf Humanit (ICH), 2017 Fac Humanit Univ Kelaniya, Sri Lanka. Published online 2017: 54
        • Sanders GS
        Driven by distraction: an integrative review of social facilitation theory and research.
        J Exp Soc Psychol. 1981; 17: 227-251https://doi.org/10.1016/0022-1031(81)90024-X
        • Strauss B
        Social facilitation in motor tasks: a review of research and theory.
        Psychol Sport Exerc. 2002; 3: 237-256https://doi.org/10.1016/S1469-0292(01)00019-X
        • Neureiter D
        • Klieser E
        • Neumayer B
        • et al.
        Feasibility of Kahoot! as a real-time assessment tool in (Histo-)pathology classroom teaching.
        Adv Med Educ Pract. 2020; 11: 695-705https://doi.org/10.2147/AMEP.S264821
      2. General Pediatrics Content Outline The American Board of Pediatrics. Available at: https://www.abp.org/become-certified/about-our-certifying-exams/general-pediatrics-content-outline. Accessed August 27, 2019

      3. American Academy of Pediatrics. PREP self-assessments. Available at: https://shop.aap.org/professional-education/self-assessments/. Accessed February 24, 2020

        • Goff I
        • Bateman B
        • Myers A
        • et al.
        Acceptability and practicality of musculoskeletal examination in acute general pediatric assessment.
        J Pediatr. 2010; 156: 657-662https://doi.org/10.1016/j.jpeds.2009.10.047
        • Foster HE
        • Kay LJ
        • Friswell M
        • et al.
        Musculoskeletal screening examination (pGALS) for school-age children based on the adult GALS screen.
        Arthritis Rheum. 2006; 55: 709-716https://doi.org/10.1002/art.22230
      4. Matters PM. pGALS Checklist. Published 2021. Available at: http://www.pmmonline.org/page.aspx?id=1624. Accessed July 25, 2019

        • Lin DTA
        • Ganapathy M
        • Kaur M
        Kahoot! it: gamification in higher education.
        Pertanika J Soc Sci Humanit. 2018; 26 (Available at:) (Accessed February 20, 2020): 565-582
        • Hergenroeder AC
        • Chorley JN
        • Laufman L
        • et al.
        Pediatric residents’ performance of ankle and knee examinations after an educational intervention.
        Pediatrics. 2001; 107: e52https://doi.org/10.1542/peds.107.4.e52
        • Ismail MAA
        • Ahmad A
        • Mohammad JAM
        • et al.
        Using Kahoot! as a formative assessment tool in medical education: a phenomenological study.
        BMC Med Educ. 2019; 19: 230https://doi.org/10.1186/s12909-019-1658-z
        • Baron RS
        Distraction-conflict theory: progress and problems.
        Adv Exp Soc Psychol. 1986; 19: 1-40https://doi.org/10.1016/S0065-2601(08)60211-7
        • Calle S
        • Bonfante E
        • Riascos R
        Introduction of the game- based learning platform, Kahoot, as a tool in radiology resident training.
        Acad Med. 2019; 18https://doi.org/10.22037/jme.v18i2.25825
        • Donkin R
        • Rasmussen R.
        Student perception and the effectiveness of Kahoot!: a scoping review in histology, anatomy, and medical education.
        Anat Sci Educ. 2021; 14: 572-585https://doi.org/10.1002/ASE.2094
        • Öz GÖ
        • Ordu Y
        The effects of web based education and Kahoot usage in evaluation of the knowledge and skills regarding intramuscular injection among nursing students.
        Nurse Educ Today. 2021; 103104910