Advertisement

Early Outcomes of a Multilevel Human Papillomavirus Vaccination Pilot Intervention in Federally Qualified Health Centers

      Abstract

      Objective

      Human papillomavirus (HPV) vaccine coverage in the United States remains low compared with other adolescent vaccines. As the largest primary care network in the United States, safety net clinics such as federally qualified health centers (FQHCs) serve patients at a disproportionate risk of HPV-related cancers. In this pilot project, the American Cancer Society (ACS) leveraged its primary care workforce to implement quality improvement interventions in the unique context of 30 FQHC systems across the country, including 130 clinic sites reaching >20,000 adolescents in a variety of geographic settings.

      Methods

      FQHC systems were randomly selected to receive either a $90,000 2-year grant, a $10,000 3-month grant, or training and technical assistance without funding. All 3 intervention groups conducted provider training and education, completed a capacity assessment tool, and measured HPV vaccination rates. Annual HPV vaccine series initiation and completion rates for active, 11- to 12-year-old patients were measured to evaluate project outcomes.

      Results

      HPV vaccine series initiation rates among 11- to 12-year-old patients increased by 14.6 percentage points from a baseline of 41.2% before the intervention (2014) to the intervention year (2015). Changes in HPV second dose and series completion rates were not statistically significant. Meningococcal and tetanus, diphtheria, and acellular pertussis vaccination rates also increased significantly, by 13.9 and 9.9 percentage points from baseline rates of 49.1% and 52.5%, respectively.

      Conclusions

      The first year of this pilot project showed early success, particularly with HPV vaccine series initiation. On the basis of these promising results, ACS is expanding clinical quality improvement projects to increase HPV vaccination across the country.

      Keywords

      Human papillomavirus (HPV) causes approximately 33,000 cancers each year in the United States, and the cost of preventing and treating HPV-related cancers is over $7 billion USD each year.
      • Viens L.J.
      • Henley S.J.
      • Watson M.
      • et al.
      Human papillomavirus–associated cancers — United States, 2008–2012.
      • Insinga R.P.
      • Dasbach E.J.
      • Elbasha E.H.
      Assessing the annual economic burden of preventing and treating anogenital human papillomavirus-related disease in the US: analytic framework and review of the literature.
      HPV vaccination is the gold standard for prevention of HPV-related diseases and cancers; its efficacy has been shown in numerous clinical trials.
      • Centers for Disease Control and Prevention (CDC)
      FDA licensure of quadrivalent human papillomavirus vaccine (HPV4, Gardasil) for use in males and guidance from the Advisory Committee on Immunization Practices (ACIP).
      The Food and Drug Administration licensed the first HPV vaccine, the quadrivalent vaccine, in 2006, the bivalent vaccine in 2009, and the latest 9-valent vaccine in 2014. Despite vaccine availability, HPV vaccine coverage in the United States remains low compared with other adolescent vaccines. Data from 2015 show that only 63% of girls and 50% of boys had initiated the 3-dose HPV vaccine series, and only 42% of girls and 28% of boys had completed the series.
      • Reagan-Steiner S.
      • Yankey D.
      • Jeyarajah J.
      • et al.
      National, regional, state, and selected local area vaccination coverage among adolescents aged 13–17 Years — United States, 2015.
      In the same year, 86.4% of adolescents had received at least 1 dose of the tetanus, diphtheria, and acellular pertussis (Tdap) vaccine and 81.3% had received at least 1 dose of the meningococcal vaccine.
      Federally qualified health centers (FQHCs) play an essential role in addressing health care disparities. FQHCs serve patients at a disproportionate risk of HPV-related cancers.
      • Allen C.
      • Harris J.
      • Hannon P.
      • et al.
      Opportunities for improving cancer prevention at federally qualified health centers.
      • Cowburn S.
      • Carlson M.
      • Lapidus J.
      • et al.
      Insurance continuity and human papillomavirus vaccine uptake in Oregon and California federally qualified health centers.
      As the largest primary care network in the United States, safety net clinics such as FQHCs serve >25 million uninsured and underinsured patients and provide a rich landscape to study HPV vaccination.
      • Allen C.
      • Harris J.
      • Hannon P.
      • et al.
      Opportunities for improving cancer prevention at federally qualified health centers.
      • National Association of Community Health Centers
      About Our Health Centers. Available at.
      Although HPV vaccine series initiation tends to be higher among children who can access vaccination benefits through the Vaccines for Children program or who have Medicaid,
      • Farmar A.
      • Love-Osborne K.
      • Chichester K.
      • et al.
      Achieving high adolescent HPV vaccination coverage.
      completion of the series still remains critically low compared with the Healthy People 2020 goals, and boys still lag behind girls in initiation as well as completion of the vaccine series.
      • Farmar A.
      • Love-Osborne K.
      • Chichester K.
      • et al.
      Achieving high adolescent HPV vaccination coverage.
      • Perkins R.
      • Zisblatt L.
      • Legler A.
      • et al.
      Effectiveness of a provider-focused intervention to improve HPV vaccination rates in boys and girls.
      Specific barriers to initiation and completion of the HPV vaccine in the FQHC setting include missed opportunities to administer the vaccine,
      • Allen C.
      • Harris J.
      • Hannon P.
      • et al.
      Opportunities for improving cancer prevention at federally qualified health centers.
      • Sussman A.
      • Helitzer D.
      • Bennett A.
      • et al.
      Catching up with the HPV vaccine: challenges and opportunities in primary care.
      parental hesitancy,
      • Colón-López V.
      • Quiñones V.
      • Del Toro-Mejías L.
      • et al.
      HPV awareness and vaccine willingness among Dominican immigrant parents attending a federal qualified health clinic in Puerto Rico.
      • Gerend M.
      • Zapata C.
      • Reyes E.
      Predictors of human papillomavirus vaccination among daughters of low-income Latina mothers: the role of acculturation.
      provider perception of parental interest in the vaccine, and a provider's ability to give a confident recommendation.
      • Farmar A.
      • Love-Osborne K.
      • Chichester K.
      • et al.
      Achieving high adolescent HPV vaccination coverage.
      • Roland K.
      • Benard V.
      • Greek A.
      • et al.
      Primary care providers human papillomavirus vaccine recommendations for the medically underserved: a pilot study in U.S. federally qualified health centers.
      Additional systemic barriers within FQHCs include competing priorities and limited funding to implement preventive health measures for patients.
      • Perkins R.
      • Zisblatt L.
      • Legler A.
      • et al.
      Effectiveness of a provider-focused intervention to improve HPV vaccination rates in boys and girls.
      • Sussman A.
      • Helitzer D.
      • Bennett A.
      • et al.
      Catching up with the HPV vaccine: challenges and opportunities in primary care.
      • Head K.
      • Vanderpool R.
      • Mills L.
      Health care providers’ perspectives on low HPV vaccine uptake and adherence in Appalachian Kentucky.
      • Roland K.
      • Benard V.
      • Greek A.
      • et al.
      Primary care providers human papillomavirus vaccine recommendations for the medically underserved: a pilot study in U.S. federally qualified health centers.
      Interventions to increase HPV uptake have taken various approaches, including quality improvement (QI), organizational system changes, electronic health record (EHR) modifications, provider education, and patient recall initiatives. All of these intervention methods have shown promising results.
      • Farmar A.
      • Love-Osborne K.
      • Chichester K.
      • et al.
      Achieving high adolescent HPV vaccination coverage.
      • Perkins R.
      • Zisblatt L.
      • Legler A.
      • et al.
      Effectiveness of a provider-focused intervention to improve HPV vaccination rates in boys and girls.
      • Colón-López V.
      • Quiñones V.
      • Del Toro-Mejías L.
      • et al.
      HPV awareness and vaccine willingness among Dominican immigrant parents attending a federal qualified health clinic in Puerto Rico.
      • Fu L.
      • Bonhomme L.
      • Cooper S.
      • et al.
      Educational interventions to increase HPV vaccination acceptance: a systematic review.
      System-focused interventions appear to aid in standardizing patient care to reduce missed opportunities, whereas provider-focused interventions teach providers how to give confident recommendations for HPV vaccination that in turn increase patient receptivity and follow-up. QI in particular has shown great promise as an approach to improve adolescent vaccination rates,
      • Bernstein H.
      • Bocchini J.A.
      Committee on Infectious Diseases
      Practical approaches to optimize adolescent immunization.
      albeit one that requires significant investment of time and presents data management challenges.
      • Fieldston E.S.
      • Hart J.
      Quality improvement in primary care for children: interest and desire, but lack of action.

      Program Overview

      The American Cancer Society (ACS) is a nationwide, community-based, voluntary health organization dedicated to the eradication of all cancers, whose initiatives include developing, implementing, and researching cancer prevention interventions. ACS views the underuse of the HPV vaccine as a major concern, because the vaccine has the potential to dramatically reduce the rates of HPV-related cancers and associated mortality. In an effort to address this concern, the ACS, with support from the Centers for Disease Control and Prevention (CDC), created the HPV Vaccinate Adolescents against Cancers (VACs) program to increase HPV vaccination rates for adolescents ages 11 to 12 years. A central component of this broader program is piloting evidence-based, QI interventions with FQHC partners across the United States.
      ACS primary care systems staff, whose job function is to engage FQHCs and other primary care systems to increase cancer prevention and early detection efforts, implemented the project. In preparation for the project, ACS primary care systems staff received advanced training in QI processes, HPV vaccination science, and evidence-based strategies to increase HPV vaccination. Staff were tasked with providing FQHCs in-person and virtual technical assistance throughout the intervention, evidence-based project tools to maximize sustainable effects, on-site provider and staff training related to HPV vaccination prioritization, and networking with local, state, and national resources.

      Methods

      FQHC Selection and Sample

      Thirty FQHC systems were selected to participate in the project. FQHC partners were selected on the basis of a review of existing relationships with ACS, FQHC capacity, interest in participating in the project, and an adolescent patient population of at least 400. ACS staff had some level of engagement with 676 FQHC systems at the start of project planning. After narrowing the pool on the basis of adolescent patient population and FQHC and ACS staff capacity, 82 systems were selected to complete a HPV vaccination self-assessment tool to determine level of interest and capacity for implementing new HPV vaccination efforts. A total of 57 self-assessment forms were returned. From these, 30 FQHC systems were randomly selected for participation, and each was randomly placed in 1 of the 3 intervention groups. Most FQHC systems implemented the intervention in multiple clinic sites, up to a maximum of 10 sites. The intervention was implemented in a total of 130 clinic sites. See Table 1 for a description of FQHC characteristics.
      Table 1Demographic Characteristics of Selected FQHC Systems
      nPercentage of Total
      FQHC sites
       Total number of clinics/sites298
       Number of clinics/sites participating in project13044
      FQHC geography
       Urban, not inner city1039
       Urban, inner city623
       Rural623
       Suburban415
       Other00
      FQHC primary clinic lead for intervention
       Quality director/manager1352
       Chief medical officer/medical director416
       Other416
       Chief nursing officer/director of nursing312
       Chief operations officer/clinic operations director14
       Chief executive officer/executive director00
      FQHC full time employees
       Primary providers394
      Attending medical doctor (MD)19048
      Nurse practitioner (NP)11028
      Physician assistant (PA)6015
      Attending doctor of osteopathic (DO)215
      Certified nurse midwife (CNM)92
      Pediatric nurse practitioner (PNP)3<1
      Resident1<1
       Other staff809
      Medical assistant (MA)47058
      Licensed practical nurse (LPN)16020
      Registered nurse (RN)14718
      Certified nurse assistant (CNA)263
      Medical secretary3<1
      Advanced practice nurse (APN)2<1
      Advanced practice registered nurse (APRN)2<1
      FQHC indicates federally qualified health care center.
      Percentages might not sum to 100% due to rounding.
      The original study design included as a control group the remaining 27 FQHC systems that submitted a self-assessment but were not randomly selected for participation. However, the complexity of calculating HPV vaccination rates presented an obstacle to this study design. Most control FQHCs did not submit follow-up vaccination rate data. Upon further investigation, the control FQHCs that did go to the effort of calculating and submitting data had decided to work with their ACS staff partners to implement HPV interventions anyway. These challenges made this group unviable as a control, and led us to abandon this element of the study design.
      Of the 30 FQHCs that started the project, 4 left the project within the first 3 months because of loss of key personnel (QI director, chief medical officer), a disruptive transition to a new EHR system, or loss of the ACS primary care staff partner. Of the 26 FQHCs that participated fully in the project, 20 were able to submit accurate vaccination rates. These 20 systems formed the study sample for outcome evaluation. The 6 systems that were unable to submit accurate vaccination rates cited a variety of reasons, including a change in EHR software, lack of capacity due to staff turnover, and lack of confidence in data quality. All 20 systems that submitted accurate 2014 baseline vaccination rates also submitted 2015 rates.

      Intervention Design

      The 30 FQHC systems were randomly selected to receive either a $90,000 2-year grant, a $10,000 3-month grant, or training and technical assistance without funding, with 10 FQHCs per funding group. All 3 intervention groups were expected to engage with ACS in conducting provider training and education, complete a capacity assessment tool, and measure HPV baseline rates during an initial, 3-month capacity-building phase. Training and education focused on preparing providers to make an effective recommendation for HPV vaccination, whereas capacity assessment and rate calculation, both system-focused interventions, led FQHCs to better understand their current state, find room for improvement, systematize care, and measure change. In addition, the $90,000 and $10,000 groups were tasked with making modifications to their EHRs that would support the HPV vaccination project. The $90,000 group was also expected to implement at least 1 specific evidence-based strategy to increase HPV vaccination rates, such as provider assessment and feedback, standing orders for HPV vaccination, provider prompts, and patient reminders. However, although only the $90,000 group was required to implement at least 1 of these strategies, nearly all FQHCs in the project did so, including the $10,000 and unfunded groups. See Table 2 for details.
      Table 2HPV VACs FQHC Pilot Intervention Groups
      Grant Funded ($90,000)Grant Funded ($10,000)Technical Assistance
      Number of FQHC systems101010
      Number of clinic sites573736
      Length of project period2 years1 year1 year
      Required project activities
       Conduct provider training
       Complete HPV vaccination capacity assessment tool
       Set baseline vaccination rates
       Modify EHR to support data collection
       Implement other HPV-specific intervention strategies
      HPV indicates human papillomavirus; VACs, Vaccinate Adolescents against Cancers; FQHC, federally qualified health care center; and EHR, electronic health record.
      This report is focused on findings from the initial capacity-building and implementation phase of the pilot between July and December of 2015. Further findings from ongoing implementation of the project will be reported in subsequent publications.

      Measures

      Three core tools were used to guide partner FQHCs through project planning, implementation, and data collection. Each tool is described in the following sections.

      HPV Vaccination Capacity Assessment

      To understand clinic capacity for HPV vaccination improvement, ACS staff partners administered a capacity assessment tool measuring FQHCs' existing policies and clinical practices to increase HPV vaccination. The assessment consisted of 48 qualitative and quantitative questions that addressed 7 core areas for FQHCs to consider in their HPV vaccination efforts: QI activities, vaccination policies, EHR systems, staff training, parent education, recommending vaccination and tracking doses, and strengths and challenges related to HPV vaccination prioritization. In partnership with ACS staff, FQHCs used findings from their capacity assessment to create tailored intervention approaches for their system.

      Steps for Increasing HPV Vaccination in Practice: An Action Guide to Implement Evidence-Based Strategies for Clinicians

      This intervention guide, created by the ACS, was used as the core project tool to guide partners through evidence-based steps to increase HPV vaccination: assembling a QI team, calculating vaccination rates, identifying opportunities for improvement, engaging staff, and implementing strategies to address opportunities for improvement. ACS staff used the “Steps” guide to lead FQHC partners through the intervention steps and strategies discussed previously in the intervention design section and identified via the capacity assessment.

      Rate Report

      FQHCs provided system-level data on vaccination rates for each of the 3 doses of the HPV vaccine, as well as the meningococcal and Tdap vaccines. Vaccination rates were calculated as the percentage of the FQHC's active 11- to 12-year-old medical patients who were up to date with the vaccine during the measurement year. Active patients were defined as those who had 1 or more medical visits at the FQHC during the reporting year. The age group was defined according to birth date: patients born in 2002 and 2003 were included for the 2014 reporting year, and patients born in 2003 and 2004 were included for the 2015 reporting year.
      Most FQHCs used more than 1 data source to calculate vaccination rates. This was often necessary to account for vaccinations administered outside of their own system. In 2015, 38% of FQHCs submitting rate data used a single data source, 38% used 2 sources, and 24% used 3 or more sources. EHR systems were the most common data source, with 75% of FQHCs using their EHR system (usually along with other sources) to calculate rate data. Other common data sources included population management systems, immunization registries, and SQL databases. FQHCs that used more than 1 data source typically relied on a labor-intensive manual process to merge data from multiple sources.
      FQHCs calculated 2014 rates during the 3-month capacity-building phase (July to September 2015), and submitted data in October 2015, along with data on activities implemented during the capacity-building phase. FQHCs calculated 2015 rates in early 2016 and submitted data in April 2016.

      Analysis

      We evaluated intervention effectiveness at the FQHC system level by comparing HPV vaccination rates among 11- to 12-year-old patients in the year before the intervention (2014) with rates in the intervention year (2015). We compared HPV vaccine initiation, second dose, and series completion separately. Series completion was defined as 3 or more doses of the HPV vaccine, because the 2016 2-dose schedule recommendations had not yet come out at the time of the project. We also measured 2014 and 2015 meningococcal and Tdap vaccination rates to compare changes in HPV vaccination with the other adolescent vaccines. In addition, we evaluated the effectiveness of each of the 3 intervention groups by comparing vaccination rates separately for each group and testing for group differences.
      To evaluate overall intervention effectiveness, we used repeated measures t tests to compare 2014 and 2015 vaccination rates for each vaccine across all intervention groups. To evaluate differences in intervention effectiveness between the 3 intervention groups, we performed 2-way repeated measures analysis of variance (ANOVA) for each vaccine. We used the vaccination rate as the dependent variable, and year, intervention group, and an interaction term between the two as the independent variables. Finally, we performed separate repeated measures t tests for each intervention group and vaccine. All analyses were conducted in Stata version 14.2 (StataCorp, College Station, Texas) using 2-tailed significance tests and a critical α = .05. For ANOVA, Greenhouse-Geisser ε was used to correct for violation of sphericity if estimated at <0.75.
      Power analysis assumed a sample size of 30 FQHCs, baseline HPV series initiation rate of 45%, and an SD of differences between baseline and follow-up HPV vaccine initiation rate of 10% (α = .05). We estimated 80% power to detect a 5-percentage point difference between 2014 and 2015 HPV vaccine initiation rates.
      Because of the substantial nonresponse rate and high SD of differences, we also conducted post hoc power analysis. With 20 FQHCs providing 2014 and 2015 rates, actual baseline HPV vaccine initiation rate of 41%, actual SD of differences between baseline and follow-up HPV vaccine initiation rate of 18%, α = .05, we estimated 80% power to detect a 12-percentage point difference between 2014 and 2015 HPV vaccine initiation rates.

      Results

      HPV vaccine series initiation rates among 11- to 12-year-old patients increased by 14.6 percentage points (range: −18.7 percentage points to +66.4 percentage points) from before the intervention (2014) to the intervention year (2015), from an average rate of 41.2% to 55.8%. Although HPV second dose rates also increased, the change was not statistically significant. HPV series completion rates decreased slightly, but the change was not statistically significant. Meningococcal and Tdap vaccination rates also increased significantly, by 13.9 and 9.9 percentage points, respectively, from baselines rates of 49.1% and 52.5%. Results of repeated measures t tests are shown in Table 3.
      Table 3Mean Vaccination Rate in HPV VACs Pilot FQHC Sites, Patients 11 to 12 Years Old
      HPV Series InitiationHPV Second DoseHPV Series CompletionMeningococcalTdap
      RateChangeRateChangeRateChangeRateChangeRateChange
      2014
       TA40.814.26.950.553.7
       $10,00032.725.020.050.254.6
       $90,00051.226.316.046.849.0
       Total41.223.716.349.152.5
      2015
       TA55.3+14.614.105.2−1.773.3+22.866.1+12.4
       $10,00043.9+11.128.2+3.115.4−4.553.7+3.453.8−0.8
       $90,00069.7+18.4
      P < .05.
      40.0+13.721.0+5.068.7+21.9
      P < .05.
      71.5+22.5
       Total55.8+14.6
      P < .01.
      30.4+6.715.6−0.763.0+13.9
      P < .01.
      62.4+9.9
      P < .05.
      HPV indicates human papillomavirus; VACs, Vaccinate Adolescents against Cancers; FQHC, federally qualified health care center; Tdap, tetanus, diphtheria, pertussis; and TA, training and technical assistance.
      Rates and rate changes are given in percentages and might not sum due to rounding.
      P < .05.
      ∗∗ P < .01.
      Results of 2-way ANOVA are shown in Table 4. There was not a significant interaction effect between year and intervention group for any of the vaccine types. However, year had a significant main effect for HPV series initiation, meningococcal, and Tdap vaccines. These results mirror the results of the t tests shown in Table 3. T tests for each intervention group show considerable between-group variance, but with a small sample size, these differences were not statistically significant, even where they were relatively large. Considered separately, only the $90,000 group's HPV series initiation and meningococcal vaccination increase was statistically significant. Again, this is because of the small sample size when vaccination rate changes were tested for individual intervention groups.
      Table 4Results of 2-Way Analysis of Variance
      Year × Group InteractionYear
      FdfPFdfP
      HPV series initiation0.282,25.75711.831,25.003
      P < .01.
      HPV second dose0.862,24.4431.41,24.256
      HPV series completion2.82,25.0940.041,25.839
      Meningococcal2.742,24.09714.841,24.002
      P < .01.
      Tdap3.022,24.0796.831,24.02
      P < .05.
      HPV indicates human papillomavirus; Tdap, tetanus, diphtheria, pertussis.
      P < .05.
      ∗∗ P < .01.

      Discussion

      Results for the project support our hypothesis that HPV vaccine series initiation rates in FQHCs would increase after participating in the HPV VACs intervention. Increasing HPV series initiation by 14.6 percentage points in the first 6 months of the intervention shows substantial early project success. For comparison, from 2014 to 2015, national HPV vaccination rates among 13- to 17-year-old patients increased by 5.4 percentage points.
      • Reagan-Steiner S.
      • Yankey D.
      • Jeyarajah J.
      • et al.
      National, regional, state, and selected local area vaccination coverage among adolescents aged 13–17 Years — United States, 2015.
      Results for HPV second dose and HPV series completion were less substantial, with HPV second dose having a moderate but not statistically significant increase, and HPV series completion remaining effectively flat. This is likely in part because of the intervals recommended between doses. During the intervention, the CDC recommended 3 doses given at 0, 2, and 6 months. With only 6 months of the intervention period in the 2015 measurement year, it became less likely as the year progressed that patients who received the first dose during the intervention period could also receive subsequent doses during the intervention period. We expect that with ongoing intervention, second dose and series completion rates will begin to increase along with series initiation.
      Although we hypothesized that meningococcal and Tdap vaccination rates would also increase because of spillover effects, the magnitude of the increase was surprising. Meningococcal and Tdap vaccination rates increased nearly as much as HPV series initiation. This suggests that the intervention might have affected the systems and processes around adolescent vaccination in general as much as the HPV vaccine specifically. Intervention strategies like EHR modification, provider prompting, and improving clinical work flows could certainly be applied to other vaccines.
      We also hypothesized that FQHCs in intervention groups receiving more funding would outperform those receiving less. Although there were measurable (sometimes large) between group differences for all vaccines, comparing intervention groups statistically proved difficult because of the small sample size and significant within group variance. Considering all vaccines together does suggest that the $90,000 group might have outperformed the others, because it had the highest increase for all but 1 vaccine (meningococcal). However, in our separate ANOVA for each vaccine, we cannot reject the null hypothesis, because the differences between groups could have been due to chance. We believe that the $10,000 and unfunded systems outperformed expectations, in terms of implementation as well as outcomes, because of the support of ACS staff serving as trusted coaching partners. Future research will explore this theory by studying the relative importance of funding and ACS staff support to partners' buy-in and level of effort in cancer prevention interventions.

      Strengths and Limitations

      The HPV VACs project leveraged the ACS′ primary care systems workforce to implement evidence-based QI interventions in the unique context of FQHCs across the country. This allowed the project to reach 30 FQHC systems with 130 clinic sites in a wide variety of geographic settings, training >700 clinical staff in the first phase of the pilot, and reaching >20,000 adolescents. The research design included elements of randomization in system selection, a standardized, evidence-based intervention, and consistent measure definitions. Although the sample size provided sufficient power to detect hypothesized effects for all intervention groups combined, it did not allow for powerful statistical comparison of intervention groups.
      Maintaining consistent measures also led to a significant nonresponse rate, because some systems were simply unable to extract accurate and complete immunization data. The age range in our measure was also quite narrow. Although this allowed us to hone in on the target population (patients who turned 11 or 12 during the measurement year), it also excluded any patients who turned 13 during the measurement year, many of whom might still have been vaccinated during the year. Furthermore, patients who turned 11 late in the year were included in the measure, despite having very little time to get vaccinated. This likely led to underestimation of project effects.
      Our active patient definition (at least 1 visit in the reporting year) might have excluded some patients who were in fact patients of the FQHC, but who had not had a recent visit. These patients would also be the least likely to have received the HPV vaccine. However, extending the active patient definition past the measurement year also would have included many patients who were no longer patients of the FQHC. Clinical staff are also accustomed to a 12-month definition from Uniform Data System and Healthcare Effectiveness Data and Information Set reporting. In any case, the effect of this and other measurement tradeoffs is mitigated by the fact that the measure definition was consistent across years.
      For some FQHCs, EHR system capacities presented significant challenges for data quality and submission. EHRs that did not interface with state immunization registries, population management systems, or other electronic tracking systems made some FQHCs unable to determine if patients had received doses of the HPV vaccine outside of their clinical system, which left them unsure of how they should count individual cases. Other EHR systems were unable to track dates of doses or indicate which dose was given. These challenges led to several FQHC systems being unable to submit reliable data for the project. An additional 4 FQHCs dropped out of the project completely, because of disruptive EHR or personnel transitions. This attrition has the potential for bias in project results.
      Finally, the lack of a control group prevents us from conclusively attributing changes in HPV vaccination rates to the intervention. Although the results are promising, especially considering the magnitude of series initiation rate change compared with the national trend, a future randomized controlled trial would provide stronger evidence.

      Conclusion

      Over the first 6 months of the ACS HPV VACs pilot project, partner FQHCs made promising improvements to their HPV vaccination rates, especially in HPV series initiation. The 3-month capacity-building phase of the project allowed time for partners to enhance EHR capacity, and provided an opportunity to identify and address concerns with data accuracy. The role of provider training was also critical to ensuring strong provider buy-in for the project and changes within clinical processes. On the basis of these promising results, ACS is expanding clinical QI projects to increase HPV vaccination across the country. Future evaluation of our cohorts will determine if rate improvements made during the pilot are sustained over time.

      Acknowledgments

      Financial disclosure: Publication of this article was supported by the Centers for Disease Control and Prevention.
      This work was supported by the CDC (NH23IP000953-02). The findings and conclusions are those of the authors and do not necessarily reflect those of the CDC.
      Disclaimer: The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.
      Ethical approval: This work was approved as exempt by the Morehouse School of Medicine's IRB on August 24, 2015 (IRB 296110-1).

      References

        • Viens L.J.
        • Henley S.J.
        • Watson M.
        • et al.
        Human papillomavirus–associated cancers — United States, 2008–2012.
        MMWR. 2016; 65: 661-666
        • Insinga R.P.
        • Dasbach E.J.
        • Elbasha E.H.
        Assessing the annual economic burden of preventing and treating anogenital human papillomavirus-related disease in the US: analytic framework and review of the literature.
        Pharmacoeconomics. 2005; 23: 1107-1122
        • Centers for Disease Control and Prevention (CDC)
        FDA licensure of quadrivalent human papillomavirus vaccine (HPV4, Gardasil) for use in males and guidance from the Advisory Committee on Immunization Practices (ACIP).
        MMWR. 2010; 59: 630-632
        • Reagan-Steiner S.
        • Yankey D.
        • Jeyarajah J.
        • et al.
        National, regional, state, and selected local area vaccination coverage among adolescents aged 13–17 Years — United States, 2015.
        MMWR. 2016; 65: 850-858
        • Allen C.
        • Harris J.
        • Hannon P.
        • et al.
        Opportunities for improving cancer prevention at federally qualified health centers.
        J Cancer Educ. 2013; 29: 30-37
        • Cowburn S.
        • Carlson M.
        • Lapidus J.
        • et al.
        Insurance continuity and human papillomavirus vaccine uptake in Oregon and California federally qualified health centers.
        Am J Public Health. 2014; 104: e71-e79
        • National Association of Community Health Centers
        About Our Health Centers. Available at.
        • Farmar A.
        • Love-Osborne K.
        • Chichester K.
        • et al.
        Achieving high adolescent HPV vaccination coverage.
        Pediatrics. 2016; 138: e20152653
        • Perkins R.
        • Zisblatt L.
        • Legler A.
        • et al.
        Effectiveness of a provider-focused intervention to improve HPV vaccination rates in boys and girls.
        Vaccine. 2015; 33: 1223-1229
        • Sussman A.
        • Helitzer D.
        • Bennett A.
        • et al.
        Catching up with the HPV vaccine: challenges and opportunities in primary care.
        Ann Fam Med. 2015; 13: 354-360
        • Colón-López V.
        • Quiñones V.
        • Del Toro-Mejías L.
        • et al.
        HPV awareness and vaccine willingness among Dominican immigrant parents attending a federal qualified health clinic in Puerto Rico.
        J Immigr Minor Health. 2014; 17: 1086-1090
        • Gerend M.
        • Zapata C.
        • Reyes E.
        Predictors of human papillomavirus vaccination among daughters of low-income Latina mothers: the role of acculturation.
        J Adolesc Health. 2013; 53: 623-629
        • Head K.
        • Vanderpool R.
        • Mills L.
        Health care providers’ perspectives on low HPV vaccine uptake and adherence in Appalachian Kentucky.
        Public Health Nurs. 2013; 30: 351-360
        • Roland K.
        • Benard V.
        • Greek A.
        • et al.
        Primary care providers human papillomavirus vaccine recommendations for the medically underserved: a pilot study in U.S. federally qualified health centers.
        Vaccine. 2014; 32: 5432-5435
        • Fu L.
        • Bonhomme L.
        • Cooper S.
        • et al.
        Educational interventions to increase HPV vaccination acceptance: a systematic review.
        Vaccine. 2014; 32: 1901-1920
        • Bernstein H.
        • Bocchini J.A.
        • Committee on Infectious Diseases
        Practical approaches to optimize adolescent immunization.
        Pediatrics. 2017; 139: e20164187
        • Fieldston E.S.
        • Hart J.
        Quality improvement in primary care for children: interest and desire, but lack of action.
        Acad Pediatr. 2016; 16: 712-713