Advertisement

The Community Health and Advocacy Milestones Profile: A Novel Tool Linking Community Pediatrics and Advocacy Training to Assessment of Milestones-Based Competence in Pediatric Residency Training

Published:March 19, 2016DOI:https://doi.org/10.1016/j.acap.2016.03.006
      In 1996, the Pediatric Residency Review Committee of the Accreditation Council for Graduate Medical Education (ACGME) required structured, formal training for pediatricians in community health and advocacy to “prepare residents to advocate on behalf of the health of children within communities.”
      • Garfunkel L.C.
      • Sidelinger D.E.
      • Rezet B.
      • et al.
      Achieving consensus on competency in community pediatrics.
      In response to this mandate, many programs developed block and/or longitudinal curricula to introduce concepts and develop skills in community health and advocacy.
      In 2005, recognizing the need to not only assess resident competence in community health and advocacy, but also to link that assessment to the 6 competency domains,

      Accreditation Council for Graduate Medical Education. Next Accreditation System. Available at: https://www.acgme.org/acgmeweb/tabid/435/ProgramandInstitutionalAccreditation/NextAccreditationSystem.aspx. Accessed March 11, 2015.

      the American Academy of Pediatrics (AAP) Community Pediatrics Training Initiative (CPTI) convened an expert panel of residency program directors, advocacy training directors, representatives of the Association of Pediatric Program Directors (APPD), the Dyson Initiative, the Pediatric Residency Review Committee, and the AAP Section on Medical Students, Resident, and Fellowship Trainees. This panel developed a set of best practice goals and objectives for pediatric residents in community health and advocacy, building upon work started by the Dyson Initiative.
      • Rezet B.
      • Riska W.
      • Blaschke G.
      Competency in community pediatrics: consensus statement of the Dyson Initiative Curriculum Committee.
      These objectives

      American Academy of Pediatrics. Community Pediatrics Training Initiative. Community Pediatrics Goals and Objectives. Available at: http://www2.aap.org/commpeds/cpti/resources.htm. Accessed March 11, 2015.

      were adopted by the Academic Pediatric Association (APA) as a part of their Educational Guidelines for Pediatric Residency

      Kittredge D, Baldwin CD, Bar-on ME, Beach PS, Trimm RF, eds. Academic Pediatric Association Educational Guidelines for Pediatric Residency. Available at: http://www.academicpeds.org/egwebnew/index.cfm. Accessed March 11, 2015.

      and have been widely used by residency training programs.
      • Rezet B.
      • Hoffman B.D.
      • Kaczorowski J.
      Pediatrics in the community: integrating community pediatrics into residency training.
      Currently, pediatric residency programs are mandated to assess trainee performance on the basis of the work of the Pediatric Milestones Project.

      Accreditation Council for Graduate Medical Education. ACGME Program Requirements for Graduate Medical Education in Pediatrics. Available at: https://www.acgme.org/acgmeweb/Portals/0/PFAssets/ProgramRequirements/320_pediatrics_2016.pdf. Accessed March 11, 2015.

      The Pediatric Milestones Project. A Joint Initiative of the American Board of Pediatrics and the Accreditation Council for Graduate Medical Education. Available at: https://www.abp.org/sites/abp/files/pdf/milestones.pdf. Accessed March 11, 2015.

      • Carraccio C.
      • Gusic M.E.
      • Hicks P.J.
      The Pediatric Milestones Project.
      Because many programs' curricula were developed before the era of Milestones, program directors face the daunting task of assessing resident performance using tools and curricular objectives designed before development of Milestone language.
      The purpose of this work was to create a tool to help residency programs map their curricula in community health and advocacy training to the objectives developed by the CPTI, and link these curricular activities to Milestone-based competencies. The process described for mapping curricula to Milestones can serve as a model for mapping curricula in other parts of the training program to the competencies.

      Developing the Community Health and Advocacy Milestones Profile and the Community Health and Advocacy Milestones Profile Mapping Tool

      The authors used a combination of expert consensus and a modified Delphi process to achieve consensus among a group of content experts between January 2013 and May 2014.

      Initial Mapping: Expert Consensus

      For the initial phase, we identified 10 individuals from the APPD and the APA Advocacy Training Special Interest Group with experience as residency program leaders and as directors of community health and advocacy training, to participate in the project.
      In randomly selected pairs, the 10 participants reviewed the Pediatric Milestones Project document

      The Pediatric Milestones Project. A Joint Initiative of the American Board of Pediatrics and the Accreditation Council for Graduate Medical Education. Available at: https://www.abp.org/sites/abp/files/pdf/milestones.pdf. Accessed March 11, 2015.

      and identified 5 competency domains that could be assessed through community health and advocacy training: systems-based practice, practice-based learning and improvement, interpersonal and communication skills, professionalism, and personal/professional development.
      Each pair was assigned 1 competency domain (eg, practice-based learning and improvement), and individually mapped each of the competencies within that domain to the CPTI objectives. There are a total of 36 objectives in 8 content areas: culturally effective care (5 objectives), child advocacy (6 objectives), medical home (6 objectives), special populations (4 objectives), pediatrician as a consultant/collaborative leader/partner (3 objectives), educational and child care settings (3 objectives), public health and prevention (5 objectives), and inquiry and application (4 objectives). Participant pairs reconciled their differences by consensus and submitted their completed map to the principal investigator. A total of 250 objective/competency matches were identified.

      Modified Delphi Process

      We then invited an additional 41 colleagues, including members of the APPD, the APA Advocacy Training Special Interest Group, the Institute on Medicine as a Profession, resident trainees, and the AAP CPTI to participate in a modified Delphi process to achieve consensus on the objective/competency matches. Each was asked to review the map and complete a survey rating their agreement with each of the 250 objective/competency matches in the 5 competency domains using a 5-point Likert scale (5 = strongly agree, 4 = agree, 3 = neutral, 2 = disagree, 1 = strongly disagree). A total of 18 of 41 (44%) colleagues completed the entire survey, in addition to the initial 10 experts, resulting in 28 completed surveys.

      Mapping Competencies to CPTI Objectives

      All agreement scores were averaged, and 5 objective/competency matches with a mean score <3.5 were removed from the map. The remaining 245 matches had a mean agreement score of 4.31. Each CPTI objective was mapped to a mean of 6.8 competencies. A total of 35 of the 48 competencies in the Pediatric Milestones Project are represented in the final product, including 12 of the 21 currently being reported to the ACGME. Of the 12 mapped competencies that are currently reported, each was mapped to a mean of 10 CPTI objectives (range, 3–20). The resulting product is the Community Health and Advocacy Milestones Profile (CHAMP) and this framework was used to develop the CHAMP Mapping Tool.

      Using CHAMP and the CHAMP Mapping Tool

      CHAMP defines the relationship between training objectives in community health/advocacy and Milestones-associated competencies. The CHAMP Mapping Tool allows programs to map their curricula and identify strengths and areas of need, while also helping programs meet the requirements for Milestones assessment and reporting. An example of how CHAMP connects Milestones to training objectives in the content area of child advocacy is shown in the Table.
      TableMilestone Competencies; Goals and Objectives for Child Advocacy
      Graduates Are Expected to:ICSSBPPBLIPROFPPD
      1. Identify and discuss individual, family, and community (local, state and/or national) concerns that affect children's health.ICS1 (ICS1)

      ICS2 (ICS2)

      ICS3
      SPB4 (SPB2)

      SBP7
      PBLI-3 (PBLI2)PROF 4 (PROF2)
      2. Formulate an attainable plan of action in response to a community health need.ICS1 (ICS1)

      ICS3

      ICS4
      SBP1

      SBP4 (SPB2)

      SBP7
      PBLI-2

      PBLI-3 (PBLI2)
      PROF 4 (PROF2)
      3. Identify and describe resources to effectively advocate for the well-being of patients, families, and communities.ICS1 (ICS1)

      ICS3
      SBP2 (SBP1)

      SPB4 (SPB2)

      SBP7
      PROF 4 (PROF2)PPD6

      PPD8 (PROF6)
      4. Communicate effectively with community groups and the media.ICS1 (ICS1)

      ICS2 (ICS2)

      ICS3

      ICS4
      PBLI-8

      PBLI-9
      PROF 4 (PROF2)
      5. Find and use evidence and data to communicate, educate, effect attitude change, and/or obtain funding to achieve specific health outcomes.ICS1 (ICS1)

      ICS3

      ICS 4
      SBP4 (SPB2)

      SBP7
      PBLI-2

      PBLI-6
      PROF 4 (PROF2)
      6. Describe and discuss key features of the legislative process, and identify and communicate with key legislators, community leaders, child advocates, and/or agency administrators about child and family health concerns.ICS1 (ICS1)

      ICS2 (ICS2)

      ICS3
      SBP4 (SPB2)

      SBP7
      PBLI-8

      PBLI-9
      PROF 2

      PROF 4 (PROF2)
      PPD6
      SBP indicates systems-based practice; PBLI, practice-based learning and improvement; ICS, interpersonal and communication skills; PROF, professionalism; and PPD, personal/professional development.
      Goals and objectives for child advocacy: recognizing their unique roles, pediatricians should advocate for the well-being of patients, families, and communities. They must develop advocacy skills to address relevant individual, community, and population health issues. All Milestone competencies are labeled according to the original Pediatric Milestones document.

      The Pediatric Milestones Project. A Joint Initiative of the American Board of Pediatrics and the Accreditation Council for Graduate Medical Education. Available at: https://www.abp.org/sites/abp/files/pdf/milestones.pdf. Accessed March 11, 2015.

      Those in bold and underlined are among the 21 being reported upon at the time of publication. In some cases, the number assigned to a competency changed in the transition, and those in italic represent the label assigned to that competency in the list of 21 Milestones being reported on at the time of publication.

      The Pediatric Milestones Project. A Joint Initiative of the American Board of Pediatrics and the Accreditation Council for Graduate Medical Education. Available at: https://www.abp.org/sites/abp/files/pdf/milestones.pdf. Accessed March 11, 2015.

      The CHAMP Mapping Tool may be used in a number of ways:
      Starting with the community health and advocacy curriculum: Take individual curricular activities (eg, visit to local Medicaid office) and run down the map, adding the activity to all CPTI objectives that could be met with that activity. The level of learning and assessment information would then be added.
      Starting with CPTI objectives: Those with a deeper understanding of training curricula and assessment methods could review each of the CPTI objectives and identify curricular activities within the community health and advocacy training, and in other experiences in the residency training program, that address that objective.
      Regardless of the initial approach, columns may be used by each program differently. For some objectives, there might be multiple curricular elements that are used to meet them. In such cases, users may divide the cells, keeping specific assessment methodology for each element. Some programs might prefer to group curricula and assessment more broadly. The use of the CHAMP Mapping Tool will allow educational leaders and residency directors to recognize the value of community health and advocacy training in the overall assessment paradigm while bolstering their curricula with learning activities that lead to deeper learning. Linking CPTI training objectives, mapped to Milestones-based competencies, with specific curricular activities will allow programs to better meet the requirements of Milestones reporting for each resident.
      The CHAMP Mapping Tool is designed to assist community health/advocacy training directors and residency program leadership. The tool can serve as a guide to map existing curricula, identify where residents might be assessed along the Milestones continuum using their existing curricula, and help identify gaps in current rotation activities. It might also be used to assist in the design or modification of assessment tools to include competencies that might be more difficult to measure in traditional, hospital-based components of the curriculum. This could be accomplished through identification of high-needs competencies, with subsequent development or identification of curricular activities and associated methods of assessment.
      Using the language provided by the CPTI objectives and/or the Pediatrics Milestones Project, one can modify existing assessment tools to include elements that assess particular competencies. For example, a community health and advocacy director might have residents write a reflection piece about a particular experience during their advocacy rotation. This could include a question about identifying cultural biases that they brought to a particular experience during the rotation and how they managed them (CPTI Culturally Effective Care Objective 1). Through their discussion with the resident about this experience and review of the reflection piece, advocacy directors can assess where the resident is on PROF[professionalism]6 (recognize that ambiguity is part of clinical medicine and respond by using appropriate resources in dealing with uncertainty), which maps to this CPTI objective (Fig). By developing activities and assessment tools that evaluate resident performance toward these competencies, community health and advocacy training directors may demonstrate to program leadership, in a real and practical way, how their curricula aid in overall resident assessment and Milestones mapping.
      Figure thumbnail gr1
      FigureCommunity Pediatrics Training Initiative (CPTI) Culturally Effective Care Objectives from the Community Health and Advocacy Milestones Profile (CHAMP) Mapping Tool, and how they can be mapped to curricular activities.
      Residency program leadership can use the CHAMP Mapping Tool to identify programmatic strengths and needs in achieving defined objectives for community health and advocacy training.

      American Academy of Pediatrics. Community Pediatrics Training Initiative. Community Pediatrics Goals and Objectives. Available at: http://www2.aap.org/commpeds/cpti/resources.htm. Accessed March 11, 2015.

      This can help spur innovation and curriculum development. By mapping the CPTI objectives to the entire curriculum, residency leaders might better understand how to achieve the CPTI objectives through curricula within, and beyond, their community health/advocacy-specific experiences. For example, in completing the CHAMP Mapping Tool, a residency leader might recognize that the residents' experience in discharge planning during their neonatal intensive care unit (NICU) rotation is an activity that helps them to understand and gain experience in “coordinating care to meet the special needs of patients with acute and chronic conditions” (CPTI Medical Home Objective 3). This objective maps to 5 of the 21 competencies currently reported to the ACGME, so the residency leader might modify the existing NICU rotation, or develop an assessment tool to be used in the NICU rotation, to allow assessment of resident performance in these competencies.
      Further, program leaders will be able to use the map as part of an overall approach to resident assessment in Milestones reporting to the ACGME. CHAMP maps the CPTI objectives to 12 of the 21 competencies currently being reported.

      The Pediatric Milestones Project. A Joint Initiative of the American Board of Pediatrics and the Accreditation Council for Graduate Medical Education. Available at: https://www.abp.org/sites/abp/files/pdf/milestones.pdf. Accessed March 11, 2015.

      Many of the competencies that can be assessed through community health/advocacy training are those that can be difficult to assess in other parts of residency training, thereby increasing the value of community health/advocacy training to the residency program.
      Finally, the CHAMP Mapping Tool allows each of the objectives to be mapped to a curricular activity that includes level of learning expected, borrowed from Miller's assessment paradigm,
      • Miller G.E.
      The assessment of clinical skills/competence/performance.
      as well as identification of the assessment method. This might enable educators to identify curricular activities that yield deeper learning, and include more robust assessment, leading to identification of best-practice examples of curricular activities and their matched competencies.
      CHAMP and the CHAMP Mapping Tool are available on the AAP CPTI Web site at: http://www2.aap.org/commpeds/cpti/CHAMP.html.

      Limitations

      This work is the product of a collaborative of community health/advocacy training directors, residency program leaders, pediatric educators, and trainees. It is a consensus-based product, and consequently, might not represent the views and conclusions of the broader public. Of 41 invitees to the mapping process, only 18 participated. It is possible that some bias was introduced secondary to the modest participation rate. Finally, although the authors endeavored to be as inclusive as possible, it is possible that other educators might have reached different conclusions in developing the CHAMP Mapping Tool.

      Next Steps

      This methodology could be used to develop a library of curricular activities in community health and advocacy training that is stratified on the basis of level of learning achieved, and curated in a scholarly way that will enable programs to share curricula. Programs that have mapped their curricula might then identify activities that meet their needs in terms of resources available, and level of learning demonstration desired. This, it is hoped, will lead to more robust training of students, residents, and fellows, and eventually to pediatricians who are more effective as advocates for children.
      Examples of completed CHAMP Mapping Tools from the Children's National Health System, Oregon Health & Science University, the University of Illinois at Chicago, and the University of Rochester are available at http://www2.aap.org/commpeds/cpti/CHAMP.html.
      We invite programs who have completed the mapping exercise to share them with the AAP CPTI, and we will publish them on our Web site to be shared with other programs. Please submit completed or partially completed maps to the Principal Investigator at [email protected] .

      Conclusions

      Through the process of developing CHAMP and the CHAMP Mapping Tool, we have shown that it is possible to link pediatric residency training curricula to competencies and milestones for resident assessment. Community health and advocacy training curricula are vital for meeting CPTI objectives and can also be used to assess competencies that are difficult to measure within pediatric residency programs.
      The CHAMP Mapping Tool has multiple benefits. It is an effective means for gauging how well a residency program provides experiences that prepare residents for their professional roles as advocates for children, might aid educators in identifying curricula and activities that involve deeper learning and foster more robust assessment of residents, and can help programs identify objectives in community health and advocacy training for residents that are not being assessed. Additionally the process described for mapping curricula to milestones can serve as a model for mapping curricula in other parts of the training program to the core competencies.

      Acknowledgments

      The authors thank the staff and leadership of the AAP CPTI for their tireless assistance, and the APPD for their feedback and endorsement.
      Financial disclosure: This work was conceptualized and initiated as the primary author (B.D.H.) was a 2013 Gold Humanism Scholar for the Harvard Macy Program for Educators, funded by the Arnold P. Gold Foundation. The authors are indebted to the residency training programs that were recipients of funding from the Anne E. Dyson Foundation, for their pioneering work in this arena.
      Authorship Statement: B.D.H. conceptualized and designed the project, interpreted results of data analyses, drafted the initial manuscript, made critical revisions to the manuscript, and approved the final manuscript as submitted. M.B. interpreted results of data analyses, made critical revisions to the manuscript, completed a map of her training program, and approved the final manuscript as submitted. C.F. interpreted results of data analyses, made critical revisions to the manuscript, completed a map of her training program, and approved the final manuscript as submitted. C.G. interpreted results of data analyses, made critical revisions to the manuscript, completed a map of her training program, and approved the final manuscript as submitted. C.L. interpreted results of data analyses, made critical revisions to the manuscript, completed a map of her training program, and approved the final manuscript as submitted. J.D. interpreted results of data analyses, made critical revisions to the manuscript, and approved the final manuscript as submitted. J.K. interpreted results of data analyses, made critical revisions to the manuscript, completed a map of his training program, and approved the final manuscript as submitted.
      The CHAMP Study Group members are as follows:
      Initial Mapping Experts: Michelle Barnes, Gregory Blaschke, Caren Gellin, Benjamin Hoffman, Wendy Hobson-Rohrer, Alice Kuo, Anda Kuo, Cara Lichtenstein, Beth Rezet, and Jennifer Walthall.
      Delphi Participants: Michelle Arandes (APPD representative), Barbara Bayldon, Lisa Chamberlain, Esther Chung, Marny Dunlap, Elizabeth Hanson, Jeffrey Kaczorowski, Dina Lieser, Dodi Meyer, Leora Mogilner, Dipesh Navsaria, Diane Pappas, Rita Patel (APPD representative), Katie Plax, Adam Rosenberg, Franklin Trimm, Brenna VanFrank (trainee), and Serena Yang.

      References

        • Garfunkel L.C.
        • Sidelinger D.E.
        • Rezet B.
        • et al.
        Achieving consensus on competency in community pediatrics.
        Pediatrics. 2005; 115: 1167-1171
      1. Accreditation Council for Graduate Medical Education. Next Accreditation System. Available at: https://www.acgme.org/acgmeweb/tabid/435/ProgramandInstitutionalAccreditation/NextAccreditationSystem.aspx. Accessed March 11, 2015.

        • Rezet B.
        • Riska W.
        • Blaschke G.
        Competency in community pediatrics: consensus statement of the Dyson Initiative Curriculum Committee.
        Pediatrics. 2005; 115: 1172-1183
      2. American Academy of Pediatrics. Community Pediatrics Training Initiative. Community Pediatrics Goals and Objectives. Available at: http://www2.aap.org/commpeds/cpti/resources.htm. Accessed March 11, 2015.

      3. Kittredge D, Baldwin CD, Bar-on ME, Beach PS, Trimm RF, eds. Academic Pediatric Association Educational Guidelines for Pediatric Residency. Available at: http://www.academicpeds.org/egwebnew/index.cfm. Accessed March 11, 2015.

        • Rezet B.
        • Hoffman B.D.
        • Kaczorowski J.
        Pediatrics in the community: integrating community pediatrics into residency training.
        Pediatr Rev. 2010; 31: 159-160
      4. Accreditation Council for Graduate Medical Education. ACGME Program Requirements for Graduate Medical Education in Pediatrics. Available at: https://www.acgme.org/acgmeweb/Portals/0/PFAssets/ProgramRequirements/320_pediatrics_2016.pdf. Accessed March 11, 2015.

      5. The Pediatric Milestones Project. A Joint Initiative of the American Board of Pediatrics and the Accreditation Council for Graduate Medical Education. Available at: https://www.abp.org/sites/abp/files/pdf/milestones.pdf. Accessed March 11, 2015.

        • Carraccio C.
        • Gusic M.E.
        • Hicks P.J.
        The Pediatric Milestones Project.
        Acad Pediatr. 2014; 14: S1-S3
        • Miller G.E.
        The assessment of clinical skills/competence/performance.
        Acad Med. 1990; 65: S63-S67