Improving Vaccination for Young Children (IVY): A Stepped-Wedge Cluster Randomized Trial



      High childhood vaccination rates are critical for public health. We hypothesized that implementation of a vaccine education and quality improvement (QI) program, Improving Vaccinations for Young Children (IVY), would improve childhood vaccine rates.


      Eight pediatric practices (3 academic-based, 5 community-based) were randomized to implement IVY within a stepped-wedge cluster randomized trial (SWCRT) in middle Tennessee. Two educational modules on vaccines were developed using best practices in instructional design. Modules were provided electronically and were tailored to providers or office staff. Practices completed in-person QI coaching sessions and selected at least 2 vaccination-related QI changes. Data were collected monthly. The primary analysis examined intervention effect on the primary outcome of Combination 10 vaccination status for children who turned 2 in the previous month. Combination 10 status without influenza vaccine was a secondary outcome, and exploratory analyses assessed intervention effects after adjusting for time and practice type.


      Data from 4041 patients (1788 control; 2253 intervention) were collected. The intervention effect was not significant on the primary outcome (OR = 1.01; 95% CI [0.76, 1.34]; P > .9), however there were positive intervention effects in secondary and exploratory models analyzing Combination10 rates without flu, including models adjusting for variation over time (0.20; 95% CI [0.04,0.35]; P = .01) and practice type (higher vaccination rate in academic practices, 0.23; 95% CI [0.03,0.42]; P = .03).


      Combination 10 rates were not significantly improved with IVY, yet evidence of beneficial effect on rates without flu vaccine was found. Future studies could evaluate effects over a longer time period and within a larger practice sample.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Academic Pediatrics
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • McGovern ME
        • Canning D.
        Vaccination and all-cause child mortality from 1985 to 2011: global evidence from the Demographic and Health Surveys.
        Am J Epidemiol. 2015; 182: 791-798
      1. Office of Disease Prevention and Health Promotion. Healthy People 2020. Published 2018. Accessed December 9, 2018.

      2. TN Department of Health. Results of the 2016 Immunization Status Survey of 24 Month Old Children in Tennessee. Published 2016. Accessed May 9, 2018.

        • Edwards KM
        • Hackell JM
        Committee On Infectious Diseases. The Committee on Practice and Ambulatory Medicine. Countering vaccine hesitancy.
        Pediatrics. 2016; 138
        • Opel DJ
        • Heritage J
        • Taylor JA
        • et al.
        The architecture of provider-parent vaccine discussions at health supervision visits.
        Pediatrics. 2013; 132: 1037-1046
        • Williams SE
        • Swan R.
        Formal training in vaccine safety to address parental concerns not routinely conducted in U.S. pediatric residency programs.
        Vaccine. 2014; 32: 3175-3178
        • Kerneis S
        • Jacquet C
        • Bannay A
        • et al.
        Vaccine education of medical students: a nationwide cross-sectional survey.
        Am J Prev Med. 2017; 53: e97-e104
        • Paterson P
        • Meurice F
        • Stanberry LR
        • et al.
        Vaccine hesitancy and healthcare providers.
        Vaccine. 2016; 34: 6700-6706
        • Frew PM
        • Randall LA
        • Malik F
        • et al.
        Clinician perspectives on strategies to improve patient maternal immunization acceptability in obstetrics and gynecology practice settings.
        Hum Vaccines Immunotherapeut. 2018; 14: 1548-1557
        • Douville LE
        • Myers A
        • Jackson MA
        • et al.
        Health care worker knowledge, attitudes, and beliefs regarding mandatory influenza vaccination.
        Arch Pediatr Adolesc Med. 2010; 164: 33-37
        • Salmon DA
        • Pan WK
        • Omer SB
        • et al.
        Vaccine knowledge and practices of primary care providers of exempt vs. vaccinated children.
        Hum Vaccines. 2008; 4: 286-291
        • Glanz JM
        • McClure DL
        • Magid DJ
        • et al.
        Parental refusal of varicella vaccination and the associated risk of varicella infection in children.
        Arch Pediatr Adolesc Med. 2010; 164: 66-70
        • Glanz JM
        • Narwaney KJ
        • Newcomer SR
        • et al.
        Association between undervaccination with diphtheria, tetanus toxoids, and acellular pertussis (DTaP) vaccine and risk of pertussis infection in children 3 to 36 months of age.
        JAMA Pediatr. 2013; 167: 1060-1064
        • Frew PM
        • Lutz CS.
        Interventions to increase pediatric vaccine uptake: an overview of recent findings.
        Hum Vaccines Immunotherapeut. 2017; 13: 2503-2511
        • Szilagyi PG
        • Schaffer S
        • Shone L
        • et al.
        Reducing geographic, racial, and ethnic disparities in childhood immunization rates by using reminder/recall interventions in urban primary care practices.
        Pediatrics. 2002; 110: e58
        • Logue E
        • Dudley P
        • Imhoff T
        • et al.
        An opt-out influenza vaccination policy improves immunization rates in primary care.
        J Health Care Poor Underserved. 2011; 22: 232-242
      3. The Community Preventative Services Task Force. Vaccination Programs: Health Care System-Based Interventions Implemented in Combination. Published 2015. Accessed December 9, 2018.

        • Kelz RR
        • Schwartz TA
        • Haut ER.
        SQUIRE reporting guidelines for quality improvement studies.
        JAMA Surg. 2021; 156: 579-581
      4. American Academy of Pediatrics. Published 2020. Accessed 6/8/ 2020.

        • Opel DJ
        • Henrikson NB
        • Lepere K
        • et al.
        Progress in evidence-based communication about childhood vaccines.
        Acad Pediatr. 2020; 20: 729-730
        • Reno JE
        • ST O'Leary
        • Pyrzanowski J
        • et al.
        Evaluation of the implementation of a multicomponent intervention to improve health care provider communication about human papillomavirus vaccination.
        Acad Pediatr. 2018; 18: 882-888
        • Kolb DA.
        Experiential LEARNING: EXPERIENCE as the Source of Learning and Development.
        2nd ed. Pearson Education, Inc, New Jersery2015
        • Sweller J.
        Element interactivity and intrinsic, extraneous, and germane cognitive load.
        Educ Psychol Rev. 2010; 22: 123-138
        • Mayer RE.
        The science of learning: determining how multimedia learning works.
        Cambridge University Press, Cambridge2009
        • Zimmerman RK
        • Raviotta JM
        • Nowalk MP
        • et al.
        Using the 4 Pillars (TM) Practice Transformation Program to increase adolescent human papillomavirus, meningococcal, tetanus-diphtheria-pertussis and influenza vaccination.
        Vaccine. 2017; 35: 6180-6186
        • Nowalk MP
        • Zimmerman RK
        • Lin CJ
        • et al.
        Maintenance of increased childhood influenza vaccination rates 1 year after an intervention in primary care practices.
        Acad Pediatr. 2016; 16: 57-63
        • Zimmerman RK
        • Nowalk MP
        • Lin CJ
        • et al.
        Cluster randomized trial of a toolkit and early vaccine delivery to improve childhood influenza vaccination rates in primary care.
        Vaccine. 2014; 32: 3656-3663
        • Lin CJ
        • Nowalk MP
        • Zimmerman RK
        • et al.
        Reducing racial disparities in influenza vaccination among children with asthma.
        J Pediatr Health Car. 2016; 30: 208-215
      5. Institute for Healthcare Improvement. How to Improve. Published 2018.

        • Hemming K
        • Haines TP
        • Chilton PJ
        • et al.
        The stepped wedge cluster randomised trial: rationale, design, analysis, and reporting.
        BMJ. 2015; 350: h391
        • Hussey MA
        • Hughes JP.
        Design and analysis of stepped wedge cluster randomized trials.
        Contemp Clin Trials. 2007; 28: 182-191
        • Hemming K
        • Taljaard M
        • Forbes A.
        Analysis of cluster randomised stepped wedge trials with repeated cross-sectional samples.
        Trials. 2017; 18: 101
      6. WHO. Ten threats to global health in 2019. Accessed October 6, 2020.

      7. WHO. Measles, Global Situation. Accessed October 6, 2020.

      8. National Committee for Quality Assurance. NCQA Updates Quality Measures for HEDIS® 2019. Published 2020. Accessed December 10, 2020.

        • Chien AT
        • Li ZH
        • Rosenthal MB.
        Improving timely childhood immunizations through pay for performance in Medicaid-managed care.
        Health Serv Res. 2010; 45: 1934-1947
        • Santibanez TA
        • Srivastav A
        • Zhai Y
        • Singleton JA.
        Trends in childhood influenza vaccination coverage, United States, 2012-2019.
        Public Health Rep. 2020; 135: 640-649
      9. C.S. Mott Children's Hospital. Mott Poll report: flu vaccine for children in the time of COVID. Published 2020. Accessed December 10, 2020.