Advertisement

A Learning Collaborative Model to Improve Human Papillomavirus Vaccination Rates in Primary Care

      Abstract

      Objective

      Human papillomavirus (HPV) vaccination rates remain low, in part because of missed opportunities (MOs) for vaccination. We used a learning collaborative quality improvement (QI) model to assess the effect of a multicomponent intervention on reducing MOs.

      Methods

      Study design: pre-post using a QI intervention in 33 community practices and 14 pediatric continuity clinics over 9 months to reduce MOs for HPV vaccination at all visit types. Measures: outcome measures comprised baseline and postproject measures of 1) MOs (primary outcome), and 2) HPV vaccine initiation and completion. Process measures comprised monthly chart audits of MOs for HPV vaccination for performance feedback, monthly Plan-Do-Study-Act surveys and pre-post surveys about office systems. Intervention: providers were trained at the start of the project on offering a strong recommendation for HPV vaccination. Practices implemented provider prompts and/or standing orders and/or reminder/recall if desired, and were provided monthly feedback on MOs to assess their progress. Analyses: chi-square tests were used to assess changes in office practices, and logistic regression used to assess changes in MOs according to visit type and overall, as well as HPV vaccine initiation and completion.

      Results

      MOs overall decreased (from 73% to 53% in community practices and 62% to 55% in continuity clinics; P < .01, and P = .03, respectively). HPV vaccine initiation increased for both genders in community practices (from 66% to 74% for female, 57% to 65% for male; P < .01), and for male patients in continuity clinics (from 68% to 75%; P = .05). Series completion increased overall in community practices (39% to 43%; P = .04) and for male patients in continuity clinics (from 36% to 44%; P = .03).

      Conclusions

      Office systems changes using a QI model and multicomponent interventions decreased rates of MO for HPV vaccination and increased initiation and completion rates among some gender subgroups. A learning collaborative model provides an effective forum for practices to improve HPV vaccine delivery.

      Keywords

      What's New
      A multicomponent quality improvement-based intervention in community practices as well as in continuity clinics designed to change office systems reduced missed opportunities for human papillomavirus vaccinations, and, for some subgroups, improved rates of human papillomavirus vaccine initiation and completion.
      Annually in the United States, approximately 19,400 women and 12,100 men are affected by human papillomavirus (HPV)-related cancers,
      Centers for Disease Control and Prevention
      Number of HPV-associated cancer cases per year. Available at.
      yet US HPV vaccination rates have lagged behind those for other adolescent vaccines. Because raising HPV vaccination rates is difficult, more than 1 strategy should be implemented simultaneously.
      The Commumity Guide
      Vaccination.
      Many physicians recommend HPV vaccine inconsistently, tepidly, or for older adolescents because they fear parental hesitancy and prolonged office visits discussing HPV vaccination.
      • Gilkey M.B.
      • Malo T.L.
      • Shah P.D.
      • et al.
      Quality of physician communication about human papillomavirus vaccine: findings from a national survey.
      • Gilkey M.B.
      • Moss J.L.
      • Coyne-Beasley T.
      • et al.
      Physician communication about adolescent vaccination: how is human papillomavirus vaccine different?.
      A strong recommendation by a trusted health care professional is consistently cited as one of the most important factors in parents' vaccination decisions.
      • Lau M.
      • Lin H.
      • Flores G.
      Factors associated with human papillomavirus vaccine-series initiation and healthcare provider recommendation in US adolescent females: 2007 National Survey of Children's Health.
      • Holman D.M.
      • Benard V.
      • Roland K.B.
      • et al.
      Barriers to human papillomavirus vaccination among US adolescents: a systematic review of the literature.
      • Ylitalo K.R.
      • Lee H.
      • Mehta N.K.
      Health care provider recommendation, human papillomavirus vaccination, and race/ethnicity in the US National Immunization Survey.
      Recent evidence underscores the importance of offering the HPV vaccine in the same way, on the same day as other adolescent vaccines.
      Centers for Disease Control and Prevention
      Talking to parents about HPV vaccine.
      • Gilkey M.B.
      • Calo W.A.
      • Moss J.L.
      • et al.
      Provider communication and HPV vaccination: the impact of recommendation quality.
      Another potential improvement involves taking advantage of all office visits for vaccination. Traditionally vaccines are recommended at preventive visits. However, fewer than half of adolescents make regular preventive visits during any single year.
      • Dempsey A.F.
      • Freed G.L.
      Health care utilization by adolescents on Medicaid: implications for delivering vaccines.
      • Rand C.M.
      • Shone L.P.
      • Albertin C.
      • et al.
      National health care visit patterns of adolescents: implications for delivery of new adolescent vaccines.
      Missed opportunities (MOs) for adolescent vaccination are common (69%–82%).
      • Wong C.A.
      • Taylor J.A.
      • Wright J.A.
      • et al.
      Missed opportunities for adolescent vaccination, 2006-2011.
      National data show that coverage with ≥1 dose of HPV vaccine before age 13 for female adolescents could have reached 91% if HPV vaccine were given concomitantly with other vaccines.
      • Stokley S.
      • Jeyarajah J.
      • Yankey D.
      • et al.
      Human papillomavirus vaccination coverage among adolescents, 2007-2013, and postlicensure vaccine safety monitoring, 2006-2014–United States.
      Thus, encouraging clinicians to recommend HPV vaccine at every encounter is key.
      Three more strategies deserve note. Provider prompts to discuss and order vaccines can be an effective strategy to reduce MOs.
      • Fiks A.G.
      • Grundmeier R.W.
      • Biggs L.M.
      • et al.
      Impact of clinical alerts within an electronic health record on routine childhood immunization in an urban pediatric population.
      Although not always effective when used alone,
      • Fiks A.G.
      • Hunter K.F.
      • Localio A.R.
      • et al.
      Impact of electronic health record-based alerts on influenza vaccination for children with asthma.
      • Szilagyi P.G.
      • Serwint J.R.
      • Humiston S.G.
      • et al.
      Effect of provider prompts on adolescent immunization rates: a randomized trial.
      electronic health record (EHR) prompts have been effective when combined with audit and feedback about MOs.
      • Fiks A.G.
      • Grundmeier R.W.
      • Mayne S.
      • et al.
      Effectiveness of decision support for families, clinicians, or both on HPV vaccine receipt.
      Patient reminder-recall, designed to increase patient demand for vaccination, is a well studied strategy to increase rates of immunization for adolescents, including HPV vaccination.
      • Jacobson Vann J.C.
      • Jacobson R.M.
      • Coyne-Beasley T.
      • et al.
      Patient reminder and recall interventions to improve immunization rates. Cochrane Database Syst Rev.
      • Kharbanda E.O.
      • Stockwell M.S.
      • Fox H.W.
      • et al.
      Text message reminders to promote human papillomavirus vaccination.
      • Rand C.M.
      • Brill H.
      • Albertin C.
      • et al.
      Effectiveness of centralized text message reminders on human papillomavirus immunization coverage for publicly insured adolescents.
      • Szilagyi P.G.
      • Schaffer S.
      • Barth R.
      • et al.
      Effect of telephone reminder/recall on adolescent immunization and preventive visits: results from a randomized clinical trial.
      • Rand C.M.
      • Vincelli P.
      • Goldstein N.P.
      • et al.
      Effects of phone and text message reminders on completion of the human papillomavirus vaccine series.
      • Stockwell M.S.
      • Kharbanda E.O.
      • Martinez R.A.
      • et al.
      Text4Health: impact of text message reminder-recalls for pediatric and adolescent immunizations.
      Standing orders are another recommended strategy to help increase vaccination rates.
      The Commumity Guide
      Vaccination.
      Standing orders allow nurses (or others, per state law
      • Stewart A.M.
      • Lindley M.C.
      • Cox M.A.
      State law and standing orders for immunization services.
      ) to administer vaccinations according to a protocol. Reminder-recall and standing orders are underused in pediatric primary care.
      • Pereira J.A.
      • Quach S.
      • Heidebrecht C.L.
      • et al.
      Barriers to the use of reminder/recall interventions for immunizations: a systematic review.
      • Albert S.M.
      • Nowalk M.P.
      • Yonas M.A.
      • et al.
      Standing orders for influenza and pneumococcal polysaccharide vaccination: correlates identified in a national survey of U.S. Primary care physicians.
      • Tierney C.D.
      • Yusuf H.
      • McMahon S.R.
      • et al.
      Adoption of reminder and recall messages for immunizations by pediatricians and public health clinics.
      Immunization experts and the Community Preventive Services Task Force
      The Commumity Guide
      Vaccination.
      recognize that multiple interventions, implemented simultaneously, are the best way to raise coverage substantially. Quality improvement (QI) learning collaboratives (LCs) might help clinicians implement HPV vaccination interventions by sharing expertise and knowledge among similar practice sites. Previously, we successfully decreased HPV MOs in a 12-month long LC of resident continuity clinics.

      Rand CM, Schaffer SJ, Dhepyasuwan N, et al. Provider communication, prompts, and feedback to improve HPV vaccination rates in resident clinics. Pediatrics. In press.

      The goal of the current study was to assess the effect of a shorter (9-month) QI intervention on community practices as well as a larger group of continuity clinics, because resident training sites are unique and most patients across the country are seen in community practices. The specific aim of this study was to assess the effect of a multicomponent QI intervention on decreasing MO rates for HPV vaccination. The goal of the QI intervention was to reduce MOs by 20%. A secondary aim was to assess the effect on HPV vaccination initiation and completion, with the goal of achieving at least a 10% improvement in the proportion of adolescents who have initiated the HPV vaccination series.

      Methods

      Context

      We conducted a 9-month QI project in 33 community practices and 14 pediatric continuity clinics. The study was approved by the University of Rochester institutional review board. The National Improvement Partnership Network recruited state improvement partnerships, which then recruited community practices using their local relationships. Practices were located in Alabama, Maine, New Hampshire, New Jersey, Tennessee, and Vermont. Likewise, the Continuity Research Network (CORNET),
      Academic Pediatric Association
      Continuity Research Network (CORNET).
      a national practice-based research network composed of pediatric resident continuity practices, recruited 14 clinics via its e-mail newsletter. Continuity clinics received $2000 per practice to compensate for time needed for chart reviews. American Board of Pediatrics, part 4 Maintenance of Certification, and American Board of Family Medicine, Part IV Maintenance of Certification for Family Physicians were offered for all clinicians as an incentive to participate.

      Intervention

      All participants were trained via webinar to deliver strong provider recommendations about HPV vaccination at the project's initiation. All practices were encouraged to implement provider prompts and participants received training on reducing MOs for HPV vaccine at all visit types by implementing such prompts. They also received monthly feedback reports on MOs. Participants had the option of implementing 2 other evidence-based strategies—standing orders and/or reminder-recall.
      We encouraged each practice to form a QI team, including at least 1 physician and nurse, resident physicians (CORNET practices), and an additional office staff member. The QI teams were responsible for educating other nurses and providers in their respective practice about true HPV vaccination contraindications, minimal dosing intervals, vaccinating at all visits, and the importance of following a single agreed upon immunization schedule. Before rooming the patient, staff (nurses or medical assistants) were expected to review the patient's immunization history and to prompt the physician to order HPV vaccine using cues such as a vaccine information statement, sticker, or EHR prompt. Clinicians shared best practices to the intervention during monthly LC calls. Each call, moderated by the study team, began with a specific presentation (eg, addressing vaccine safety, using standing orders), followed by practices sharing successes and barriers they encountered.

      Measures

      Our primary outcome measure was MO for HPV vaccination, defined as any visit in which a patient eligible to receive HPV vaccine was not vaccinated. This was assessed from a random sample of 50 patient charts (25 of each gender), for patients 11 to 17 years who had visits in the 9 months before the intervention, and again after the intervention. Our secondary outcome measures were HPV vaccination rates (proportion of adolescents who initiated HPV vaccination (termed “initiation” and proportion who completed the HPV vaccine series, termed “completion”). Chart reviews included the patient's race/ethnicity, insurance status, age in years, visit type, whether the patient received HPV vaccine at the visit, and if not, the documented reason. HPV initiation rate was defined as the number of patients who had ≥1 dose of HPV vaccination divided by the total number of patient charts reviewed in the 9-month period (50). The HPV completion rate was defined as the number of patients who had 3 doses of HPV vaccination divided by the total number of patient charts reviewed in the 9-month period (50).
      Our process measures were: 1) monthly MOs for vaccination, and 2) the reason a vaccine was not given, if due. To assess monthly MO data, providers reviewed charts for a convenience sample of 10 patients aged 11 to 17 years who were eligible to receive an HPV vaccine at the visit being reviewed. Chart reviews included the HPV vaccine history, visit type (well care, acute, chronic, nurse-only, or other), and the documented reason the patient was not vaccinated (eg, refused/declined, postponed, contraindication, other). As a balancing measure, we asked each practice if the QI effort increased the administrative burden for staff; 69% stated it did not.
      Additionally, to understand changes in office practices due to the project, we asked the lead clinician to complete a survey (after they discussed it with their teams) at baseline as well as at the end of the project. Questions included perceptions of the strength of providers' HPV vaccine recommendations according to patient gender and age, previous QI experience, and assessment of office strategies to reduce MOs. Practices filled out a monthly Plan-Do-Study-Act tool to report which intervention their site was using, which parts of the intervention worked, and what barriers were encountered.

      Analysis

      Primary Outcome

      For comparison of baseline versus postintervention data, we used McNemar chi-square tests to assess changes in office practices, and logistic regression with site fixed effects and a clustered sandwich estimator at the patient level to assess changes in MOs according to visit type and overall, as well as HPV vaccine initiation and completion (without a clustered sandwich estimator). We used logistic regression to assess differences in MOs on the basis of the type and number of interventions and previous QI experience. The unit of analysis was the visit for all MO analyses and was the patient for all HPV series initiation and completion analyses. All pre-post analyses were performed with Stata Version 12 (StataCorp, College Station, Texas).

      Process Measure

      Practices received individual site and aggregate MO data in monthly practice reports during the intervention, which included run charts for MOs overall, according to gender, and visit type, in addition to the documented reasons that patients were not vaccinated. For the qualitative analysis, 3 authors (C.M.R., H.T., R.W.-B.) reviewed the Plan-Do-Study-Act logs to assess barriers and facilitators to reducing MOs.

      Results

      Practices

      All continuity clinics and most of the community practices were pediatric offices, with 3 family medicine community practices included (Table 1). Most patients in community practices were white, non-Hispanic, and privately insured, whereas one-quarter of patients in the continuity clinics were Hispanic, and most were publicly insured.
      Table 1Practice Demographic Characteristics (Preintervention)
      Community Practices (n = 33)Continuity Clinics (n = 14)
      States, n611
      Number of patients 11–17 years of age seen annually, n28,27215,163
      Specialty, n
       Pediatrics3014
       Family medicine30
      Patient race, %
       White6141
       Black/African American1919
       Asian43
       Unknown/not noted1434
       Other24
      Patient ethnicity, %
       Hispanic/Latino624
       Not Hispanic/Latino7955
       Unknown/not noted1521
      Patient insurance, %
       Public4655
       Private5135
       Uninsured/unknown311
      Total records reviewed, n1646683

      MOs for HPV Vaccination

      MOs declined from 73.4% to 52.8% (P = .03) for community practices and from 61.8% to 54.7% (P < .01) for continuity practices (Table 2). Overall, 26 of 33 (79%) community and 9 of 14 (64%) continuity practices reduced their MOs during the project. The greatest MO reduction occurred at nurse-only and well care visits for both groups.
      Table 2Percent of Missed Opportunity According to Visit Type and Practice Group
      Practice TypeVisit TypePre, n (%)Post, n (%)Percentage Point DifferenceP
      Logistic regression with site fixed effects and clustered sandwich estimator for patient.
      Community practicesWell care446 (53.2)303 (34.2)−19.0<.01
      Acute/chronic care1302 (92.5)805 (82.3)−10.2<.01
      Nurse only107 (38.8)50 (14.8)−24.0<.01
      Other32 (66.7)15 (68.2)+1.5.60
      Total eligible visits1887 (73.4)1173 (52.8)−20.6<.01
      Continuity clinicsWell care132 (44.0)113 (34.6)−9.4.06
      Acute/chronic care295 (85.8)287 (80.2)−5.6<.01
      Nurse only28 (32.2)22 (23.4)−8.8.02
      Other10 (45.5)11 (84.6)+39.1.03
      Total eligible visits465 (61.8)433 (54.7)−7.1.03
      Logistic regression with site fixed effects and clustered sandwich estimator for patient.
      Although the number of practices that self-selected each combination of interventions was small, we performed an exploratory analysis to assess rates of MOs for each subgroup (Table 3). Among continuity clinics, the rates of MOs were lowest for practices that implemented prompts as well as reminder-recall; this pattern was not clearly seen among community practices. However, among community as well as continuity clinic practices, the additional use of standing orders did not appear to be beneficial.
      Table 3Missed Opportunities Percentages on the Basis of the Number of Interventions
      Practice TypeVisit TypeInterventions
      Prompts Only

      CP, n = 7

      CC, n = 8
      Prompts and SO

      CP, n = 13

      CC, n = 2
      Prompts and R/R

      CP, n = 5

      CC, n = 1
      Prompts, SO, and R/R

      CP, n = 8

      CC, n = 3
      P
      Joint test from logistic regression with site fixed effects and clustered sandwich estimator for patient.
      CPWell care139 (40.1)289 (44.3)77 (41.4)244 (45.3)<.01
      Acute/chronic care373 (82.3)787 (88.7)187 (87.8)762 (91.4)<.01
      Nurse only21 (30.4)72 (29.8)21 (30.4)43 (18.4).63
      Other1 (100.0)31 (73.8)13 (72.2)2 (22.2).91
      Total534 (61.4)1179 (64.7)298 (61.3)1051 (65.0)<.01
      CCWell care134 (36.2)37 (48.1)18 (32.1)57 (41.9).29
      Acute/chronic care388 (79.7)72 (87.8)18 (90.0)115 (87.1).11
      Nurse only26 (28.0)15 (39.5)1 (6.7)8 (22.9).16
      Other7 (63.6)0 (0.0)2 (100.0)12 (57.1).94
      Total555 (57.8)124 (62.6)39 (41.9)192 (59.3).02
      CP indicates community practices; CC, continuity clinics; SO, standing order; and R/R, reminder/recall.
      Data are presented as n (%).
      Joint test from logistic regression with site fixed effects and clustered sandwich estimator for patient.
      We also performed an exploratory analysis to assess whether practices with previous QI experiences achieved larger reduction in MOs than practices without QI experience. All continuity clinics and 88% of community practices had QI experience. Community practices with QI experience had a greater reduction in MOs than those without, by 6 percentage points overall (69% to 63%).

      Patterns of MOs

      Monthly run charts revealed variation in practice improvement according to visit type (Fig. 1). MOs at acute visits declined dramatically at first, then plateaued and rose slightly in February. MOs at well visits declined slowly, but steadily. At nurse visits, MOs increased from October to December when many nurse visits were scheduled for influenza vaccination, but subsequently declined (Fig. 1).
      Figure thumbnail gr1
      Figure 1Run chart of missed opportunities according to visit type.

      Reasons for MOs

      Although the proportion of visits for which documentation for the reason for MOs increased, only 11% and 7% of charts in community and continuity practices, respectively, had documentation of HPV vaccination refusal (data not shown).

      Vaccination Rates

      In community practices, HPV vaccine initiation rates improved significantly for female adolescents (from 66% to 74% of female adolescents who had initiated the HPV series; P < .01), male adolescents (57% to 65%; P < .01), and overall (62% to 70%; P < .01) (Table 4). In continuity clinics, initiation rates increased significantly only for male adolescents (from 68% to 75%; P = .05) and overall (71% to 77%; P < .01). Completion rates for the HPV vaccination series improved overall for community practices (from 39% to 43%; P = .04) and for male adolescents in continuity clinic practices (36% to 44%; P = .03).
      Table 4HPV Vaccine Series Initiation and Completion Rates Before and After QI Intervention
      HPV DosePractice TypeFemaleMaleTotal
      PrePostDifferenceP
      Logistic regression with site fixed effects.
      PrePostDifferenceP
      Logistic regression with site fixed effects.
      PrePostDifferenceP
      Logistic regression with site fixed effects.
      InitiationCommunity practice66.374.3+8.0<.0157.165.1+8.0<.0161.769.7+8.0<.01
      Continuity clinic73.479.6+6.2.0668.074.9+6.9.0570.677.3+6.7<.01
      CompletionCommunity practice45.348.5+3.2.2133.137.1+4.0.0739.342.8+3.5.04
      Continuity clinic51.751.1−0.61.0036.243.9+7.7.0343.647.5+3.9.09
      HPV indicates human papillomavirus; QI, quality improvement.
      Data are presented as percentages except where otherwise noted.
      Logistic regression with site fixed effects.

      Changes in Office Systems

      Practices reported significant improvements in office systems to reduce MOs (Fig. 2). By study end, more practices posted a common schedule (from 42% to 71%), educated staff on valid doses (from 50% to 83%), and had an office policy to vaccinate at all visits (from 50% to 85%; P < .01 for each). Strong recommendations for the vaccine for 11- to 12-year-old patients were almost universal at baseline and postintervention for continuity clinics (for female patients 100% pre and post, for male patients from 93% to 100%; P = 1), but community practices improved from 65% to 85% (P = .06) for female patients and 56% to 82% (P = .02) for male patients (data not shown). As shown in Figure 3, all practices except 1 implemented provider prompts, some community practices instituted standing orders, and several community and continuity practices instituted reminder-recall.
      Figure thumbnail gr2
      Figure 2Strategies to reduce missed opportunities for all practices combined.
      Figure thumbnail gr3
      Figure 3Human papillomavirus vaccine office systems in place before (pre) and after (post) intervention according to clinic type.

      Practice Barriers and Facilitators (Qualitative Data)

      The main practice barriers included concerns related to prompts, time constraints, reaching parents for reminders, parent education and refusal, and system issues. Regarding prompts, practices noted that prompts might not be seen by providers, might be ignored, and were sometimes missing or incorrect. Some practices overcame these issues by changing the prompt location to make it more visible, adding the prompt to the visit preparation materials, and discussing vaccination in the previsit huddle. Practices that struggled were not able to change the work flow, had difficulty getting the staff to regularly prompt the provider, and reported substantial staff turnover. Regarding time constraints, practices noted that the prompt or reminder-recall could be time-consuming to implement on busy days. Some noted that care coordinators assisted with the process. Practices that used reminder-recall noted texting reminders and scheduling visits before patients leaving the office to be successful. Regarding parent education and refusal, some practices cited parent refusal as a consistent barrier. Several practices overcame this, in part, by educating staff to talk to parents, giving patients printed material before seeing the physician, and having parents sign a declination form if they refused. System issues included difficulties with engaging leadership and competing demands. Frequent reminders to staff about the project and consistent prerounding helped practices to perform well.

      Discussion

      In this QI study, community practices and continuity clinics reduced MOs for HPV vaccination via a multicomponent intervention that included training in a strong provider recommendation, provider prompts, and audit with feedback (plus optionally using reminder-recall and standing orders). This shows that the combined intervention we previously found to be successful in continuity clinics was even more successful in community practices, which had higher rates of MOs at baseline. Community practices and continuity clinics reduced MOs by 20 and 7 percentage points, respectively. HPV vaccine initiation rates improved by 8 and 6 percentage points, respectively, in community practices and continuity clinics; HPV vaccine completion rates also improved.
      As noted previously,
      • Zimmerman R.K.
      • Moehling K.K.
      • Lin C.J.
      • et al.
      Improving adolescent HPV vaccination in a randomized controlled cluster trial using the 4 Pillars Practice Transformation Program.
      our study suggests that changes in office systems are critical for reducing MOs and raising HPV vaccination rates. In addition to educating staff on valid doses and designating a clinic champion, the 2 greatest changes noted by participating practices were having every provider agree on a common schedule and vaccinating at all visits. Both strategies have been recommended by the Centers for Disease Control and Prevention.

      Centers for Disease Control and Prevention. Epidemiology and prevention of vaccine-preventable diseases. Available at: https://www.cdc.gov/vaccines/pubs/pinkbook/chapters.html. Accessed November 15, 2016.

      Staff education was crucial to encouraging their participation in reviewing vaccine records, and helped nurses gain comfort addressing parental hesitation independently. Practices that were unable to decrease MO rates reported difficulty engaging the leadership and/or experienced other priorities or changes that took precedence. Successful practice system changes require leadership and staff support, sufficient resources, and effective communication.
      • Pexton C.
      Overcoming the barriers to change in healthcare system.
      Provider prompts were implemented to reduce MOs at all visits. Although many practices reported having some provider prompts at baseline, these were insufficient to avoid MOs. Because our previous study of EHR prompts
      • Szilagyi P.G.
      • Serwint J.R.
      • Humiston S.G.
      • et al.
      Effect of provider prompts on adolescent immunization rates: a randomized trial.
      did not show a reduction in MOs, for this current QI study we ensured that practices that had EHR prompts also implemented nurse-generated provider prompts as well, and that providers and nurses were trained about how to alter procedures to implement prompts. Our qualitative findings suggest that prompts need to follow the “5 rights” of clinical decision support: the right information for the right person in the right format via the right channel at the right time.
      • Campbell R.
      The five “rights” of clinical decision support.
      Successful practices were able to optimize their EHR to present the right information in a feasible work flow in addition to having nurses or medical assistants review the information. Other practices had inconveniently located EHR prompts and staff who frequently failed to remind the provider because of forgetting or lack of training.
      Vaccinations are traditionally administered at preventive visits. In our study, MO rates were much lower at well visits than other visits, as expected. We believe the decrease in MOs that occurred at well visits was likely because of providers offering a strong HPV vaccine recommendation, rather than prompts. The MO decrease at acute visits was likely because of a combination of changing office practice to vaccinate at these visits (incorporating prompts), along with improved communication.
      Overall, our multicomponent intervention was effective in reducing MOs across all visit types. One novel finding in this study is the potential for using nurse-only visits to reduce MOs for HPV vaccination. Studies have not previously focused on nurse-only visits, yet they constitute a sizeable proportion of all visits to primary care practices. Vaccinating at nurse-only visits is efficient. Practices improved dramatically at these visits by having active nursing involvement. However, during influenza vaccination season MOs increased because neither families nor clinicians were prepared for HPV vaccine administration at influenza vaccine clinics. These findings suggest that it might be challenging during the few months of influenza vaccinations to use nurse visits for HPV vaccination, but practices might wish to target nurse-only visits during the other months of the year for HPV vaccinations.
      HPV vaccine initiation rates increased for both genders in community practices, and for male patients in continuity clinics, likely because of a strong recommendation as well as practice change. Vaccine completion rates improved overall for community practices and for male patients in continuity clinics. Baseline vaccination rates for male patients were lower than that for female patients, allowing more opportunity to improve. The now recommended 2-dose schedule for adolescents starting vaccination before age 15 years also is expected to help with completion rates.
      • Meites E.
      • Kempe A.
      • Markowitz L.E.
      Use of a 2-dose schedule for human papillomavirus vaccination - updated recommendations of the Advisory Committee on Immunization Practices.
      Our study has numerous strengths. A large number of community practices and pediatric continuity clinics with sizeable patient numbers participated. We trained many health care professionals in providing a strong HPV vaccine recommendation, which we expect to be sustainable. We investigated a real-life, multicomponent intervention in a relatively rapid cycle.
      There are several study limitations and characteristics that might limit generalizability. The continuity practices received a modest participation incentive and most community practices already had QI infrastructure in place. Although the practices reviewed their own charts and entered data, the study team, who were supported by grant funding, provided feedback reports and coaching. Some QI aspects could be replicated on a larger scale with insurers or health department reviews that report immunization rates back to practices. Finally, practices volunteered to participate, and might have been more willing to change than other practices.

      Conclusion

      This QI study shows that a multicomponent intervention, which includes training in giving a strong HPV vaccination recommendation, with provider prompts as well as feedback on MOs can reduce MOs for HPV vaccination in community practices as well as continuity clinics and improve HPV vaccination rates.

      Acknowledgments

      Financial disclosure: Publication of this article was supported by the Centers for Disease Control and Prevention.
      This work was funded by a grant from the Centers for Disease Control and Prevention (1H23IP000850). The findings and conclusions in this report are those of the authors and do not necessarily represent the views of the Centers for Disease Control and Prevention.
      We appreciate the hard work of the providers from CORNET and National Improvement Partnership Network practices for participating in this project. We acknowledge Nui Dhepyasuwan and Beth King for recruiting CORNET practices for this project and developing project materials.

      References

        • Centers for Disease Control and Prevention
        Number of HPV-associated cancer cases per year. Available at.
        • The Commumity Guide
        Vaccination.
        (Available at:)
        • Gilkey M.B.
        • Malo T.L.
        • Shah P.D.
        • et al.
        Quality of physician communication about human papillomavirus vaccine: findings from a national survey.
        Cancer Epidemiol Biomarkers Prev. 2015; 24: 1673-1679
        • Gilkey M.B.
        • Moss J.L.
        • Coyne-Beasley T.
        • et al.
        Physician communication about adolescent vaccination: how is human papillomavirus vaccine different?.
        Prev Med. 2015; 77: 181-185
        • Lau M.
        • Lin H.
        • Flores G.
        Factors associated with human papillomavirus vaccine-series initiation and healthcare provider recommendation in US adolescent females: 2007 National Survey of Children's Health.
        Vaccine. 2012; 30: 3112-3118
        • Holman D.M.
        • Benard V.
        • Roland K.B.
        • et al.
        Barriers to human papillomavirus vaccination among US adolescents: a systematic review of the literature.
        JAMA Pediatr. 2014; 168: 76-82
        • Ylitalo K.R.
        • Lee H.
        • Mehta N.K.
        Health care provider recommendation, human papillomavirus vaccination, and race/ethnicity in the US National Immunization Survey.
        Am J Public Health. 2013; 103: 164-169
        • Centers for Disease Control and Prevention
        Talking to parents about HPV vaccine.
        (Available at:)
        • Gilkey M.B.
        • Calo W.A.
        • Moss J.L.
        • et al.
        Provider communication and HPV vaccination: the impact of recommendation quality.
        Vaccine. 2016; 34: 1187-1192
        • Dempsey A.F.
        • Freed G.L.
        Health care utilization by adolescents on Medicaid: implications for delivering vaccines.
        Pediatrics. 2010; 125: 43-49
        • Rand C.M.
        • Shone L.P.
        • Albertin C.
        • et al.
        National health care visit patterns of adolescents: implications for delivery of new adolescent vaccines.
        Arch Pediatr Adolesc Med. 2007; 161: 252-259
        • Wong C.A.
        • Taylor J.A.
        • Wright J.A.
        • et al.
        Missed opportunities for adolescent vaccination, 2006-2011.
        J Adolesc Health. 2013; 53: 492-497
        • Stokley S.
        • Jeyarajah J.
        • Yankey D.
        • et al.
        Human papillomavirus vaccination coverage among adolescents, 2007-2013, and postlicensure vaccine safety monitoring, 2006-2014–United States.
        MMWR Morb Mortal Wkly Rep. 2014; 63: 620-624
        • Fiks A.G.
        • Grundmeier R.W.
        • Biggs L.M.
        • et al.
        Impact of clinical alerts within an electronic health record on routine childhood immunization in an urban pediatric population.
        Pediatrics. 2007; 120: 707-714
        • Fiks A.G.
        • Hunter K.F.
        • Localio A.R.
        • et al.
        Impact of electronic health record-based alerts on influenza vaccination for children with asthma.
        Pediatrics. 2009; 124: 159-169
        • Szilagyi P.G.
        • Serwint J.R.
        • Humiston S.G.
        • et al.
        Effect of provider prompts on adolescent immunization rates: a randomized trial.
        Acad Pediatr. 2015; 15: 149-157
        • Fiks A.G.
        • Grundmeier R.W.
        • Mayne S.
        • et al.
        Effectiveness of decision support for families, clinicians, or both on HPV vaccine receipt.
        Pediatrics. 2013; 131: 1114-1124
        • Jacobson Vann J.C.
        • Jacobson R.M.
        • Coyne-Beasley T.
        • et al.
        Patient reminder and recall interventions to improve immunization rates. Cochrane Database Syst Rev.
        2018 (1:CD003941)
        • Kharbanda E.O.
        • Stockwell M.S.
        • Fox H.W.
        • et al.
        Text message reminders to promote human papillomavirus vaccination.
        Vaccine. 2011; 29: 2537-2541
        • Rand C.M.
        • Brill H.
        • Albertin C.
        • et al.
        Effectiveness of centralized text message reminders on human papillomavirus immunization coverage for publicly insured adolescents.
        J Adolesc Health. 2015; 56: S17-S20
        • Szilagyi P.G.
        • Schaffer S.
        • Barth R.
        • et al.
        Effect of telephone reminder/recall on adolescent immunization and preventive visits: results from a randomized clinical trial.
        Arch Pediatr Adolesc Med. 2006; 160: 157-163
        • Rand C.M.
        • Vincelli P.
        • Goldstein N.P.
        • et al.
        Effects of phone and text message reminders on completion of the human papillomavirus vaccine series.
        J Adolesc Health. 2017; 60: 113-119
        • Stockwell M.S.
        • Kharbanda E.O.
        • Martinez R.A.
        • et al.
        Text4Health: impact of text message reminder-recalls for pediatric and adolescent immunizations.
        Am J Public Health. 2012; 102: e15-e21
        • Stewart A.M.
        • Lindley M.C.
        • Cox M.A.
        State law and standing orders for immunization services.
        Am J Prev Med. 2016; 50: e133-e142
        • Pereira J.A.
        • Quach S.
        • Heidebrecht C.L.
        • et al.
        Barriers to the use of reminder/recall interventions for immunizations: a systematic review.
        BMC Med Inform Decis Mak. 2012; 12: 145
        • Albert S.M.
        • Nowalk M.P.
        • Yonas M.A.
        • et al.
        Standing orders for influenza and pneumococcal polysaccharide vaccination: correlates identified in a national survey of U.S. Primary care physicians.
        BMC Fam Pract. 2012; 13: 22
        • Tierney C.D.
        • Yusuf H.
        • McMahon S.R.
        • et al.
        Adoption of reminder and recall messages for immunizations by pediatricians and public health clinics.
        Pediatrics. 2003; 112: 1076-1082
      1. Rand CM, Schaffer SJ, Dhepyasuwan N, et al. Provider communication, prompts, and feedback to improve HPV vaccination rates in resident clinics. Pediatrics. In press.

        • Academic Pediatric Association
        Continuity Research Network (CORNET).
        (Available at:)
        • Zimmerman R.K.
        • Moehling K.K.
        • Lin C.J.
        • et al.
        Improving adolescent HPV vaccination in a randomized controlled cluster trial using the 4 Pillars Practice Transformation Program.
        Vaccine. 2017; 35: 109-117
      2. Centers for Disease Control and Prevention. Epidemiology and prevention of vaccine-preventable diseases. Available at: https://www.cdc.gov/vaccines/pubs/pinkbook/chapters.html. Accessed November 15, 2016.

        • Pexton C.
        Overcoming the barriers to change in healthcare system.
        (Available at:)
        • Campbell R.
        The five “rights” of clinical decision support.
        J AHIMA. 2013; 84: 42-47
        • Meites E.
        • Kempe A.
        • Markowitz L.E.
        Use of a 2-dose schedule for human papillomavirus vaccination - updated recommendations of the Advisory Committee on Immunization Practices.
        MMWR Morb Mortal Wkly Rep. 2016; 65: 1405-1408