Centralized Reminder/Recall to Increase Immunization Rates in Young Children: How Much Bang for the Buck?

Published:November 29, 2016DOI:



      We compared the effectiveness and cost-effectiveness of: 1) centralized reminder/recall (C-R/R) using the Colorado Immunization Information System (CIIS) versus practice-based reminder/recall (PB-R/R) approaches to increase immunization rates; 2) different levels of C-R/R intensity; and 3) C-R/R with versus without the name of the child's provider.


      We conducted 3 sequential cluster-randomized trials involving children aged 19 to 25 months in 15 Colorado counties in March 2013 (trial 1), October 2013 (trial 2), and May 2014 (trial 3). In C-R/R counties, the intensity of the intervention decreased sequentially in trials 1 through 3, from 3 to 1 recall messages. In PB-R/R counties, practices were offered training using CIIS and financial support. The percentage of children with up-to-date (UTD) vaccinations was compared 6 months after recall. A mixed-effects model assessed the association between C-R/R versus PB-R/R and UTD rates.


      C-R/R was more effective in trials 1 to 3 (relative risk = 1.11; 95% confidence interval 1.01–1.20; P = .009). Effectiveness did not decrease with decreasing intervention intensity (P = .59). Costs decreased with decreasing intensity in the C-R/R arm, from $18.72 per child brought UTD in trial 1 to $10.11 in trial 3. Costs were higher and more variable in the PB-R/R arm, ranging from $20.63 to $237.81 per child brought UTD. C-R/R was significantly more effective if the child's practice name was included (P < .0001).


      C-R/R was more effective and cost-effective than PB-R/R for increasing UTD rates in young children and was most effective if messages included the child's provider name. Three reminders were not more effective than one, which may be explained by the increasing accuracy of contact information in CIIS over the course of the trials.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Academic Pediatrics
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Szilagyi P.G.
        • Bordley C.
        • Vann J.C.
        • et al.
        Effect of patient reminder/recall interventions on immunization rates: a review.
        JAMA. 2000; 284: 1820-1827
        • Szilagyi P.
        • Vann J.
        • Bordley C.
        • et al.
        Interventions aimed at improving immunization rates.
        Cochrane Database Syst Rev. 2002; : CD003941
      1. Centers for Disease Control and Prevention. IIS state/territory/city registry staff—main and technical contacts. Available at: Accessed September 16, 2015.

        • Centers for Disease Control and Prevention
        Immunization Information Systems annual reports.
        2014 (Available at:) (Accessed June 1, 2016)
        • Centers for Disease Control and Prevention
        Progress in Immunization Information Systems—United States, 2012.
        MMWR Morb Mortal Wkly Rep. 2013; 62: 1005-1008
        • Centers for Disease Control and Prevention
        Immunization Information System annual report.
        2012 (Available at:) (Accessed November 7, 2015)
      2. Community Preventive Services Task Force. Increasing appropriate vaccination: vaccination programs in schools and organized child care centers. Available at: Accessed June 4, 2015.

        • Saville A.W.
        • Albright K.
        • Nowels C.
        • et al.
        Getting under the hood: exploring issues that affect provider-based recall using an immunization information system.
        Acad Pediatr. 2011; 11: 44-49
        • Szilagyi P.G.
        • Roghmann K.J.
        • Campbell J.R.
        • et al.
        Immunization practices of primary care practitioners and their relation to immunization levels.
        Arch Pediatr Adolesc Med. 1994; 148: 158-166
        • Clark S.J.
        • Cowan A.E.
        • Bartlett D.L.
        Private provider participation in statewide immunization registries.
        BMC Public Health. 2006; 6: 33
        • Tierney C.D.
        • Yusuf H.
        • McMahon S.R.
        • et al.
        Adoption of reminder and recall messages for immunizations by pediatricians and public health clinics.
        Pediatrics. 2003; 112: 1076-1082
        • Dombkowski K.J.
        • Cowan A.E.
        • Harrington L.B.
        • et al.
        Feasibility of initiating and sustaining registry-based immunization recall in private practices.
        Acad Pediatr. 2012; 12: 104-109
        • Kempe A.
        • Hurley L.
        • Cardemil C.V.
        • et al.
        The use of immunization information systems (IISs) in primary care.
        Am J Prev Med. 2016; (in press)
        • Kempe A.
        • Saville A.W.
        • Dickinson L.M.
        • et al.
        Collaborative centralized reminder/recall notification to increase immunization rates among young children: a comparative effectiveness trial.
        JAMA Pediatr. 2015; 169: 365-373
        • Kempe A.
        • Saville A.
        • Dickinson L.M.
        • et al.
        Population-based versus practice-based recall for childhood immunizations: a randomized controlled comparative effectiveness trial.
        Am J Public Health. 2013; 103: 1116-1123
      3. Colorado Department of Public Health and Environment (CIIS). Colorado annual immunization progress report for CY2013, section IX—immunization information systems. 2013. Available at: Accessed March 3, 3017.

        • Dickinson L.M.
        • Beaty B.
        • Fox C.
        • et al.
        Pragmatic cluster randomized trials using covariate constrained randomization: a method for practice-based research networks (PBRNs).
        J Am Board Fam Med. 2015; 28: 663-672
        • Raab G.M.
        • Butcher I.
        Balance in cluster randomized trials.
        Stat Med. 2001; 20: 351-365
        • Glynn R.J.
        • Brookhart M.A.
        • Stedman M.
        • et al.
        Design of cluster-randomized trials of quality improvement interventions aimed at medical care providers.
        Med Care. 2007; 45: S38-S43
        • Ivers N.M.
        • Halperin I.J.
        • Barnsley J.
        • et al.
        Allocation techniques for balance at baseline in cluster randomized trials: a methodological review.
        Trials. 2012; 13: 120
        • Moulton L.H.
        Covariate-based constrained randomization of group-randomized trials.
        Clin Trials. 2004; 1: 297-305
        • Elam-Evans L.D.
        • Yankey D.
        • Singleton J.A.
        • et al.
        National, state, and selected local area vaccination coverage among children aged 19–35 months—United States, 2013.
        MMWR Morb Mortal Wkly Rep. 2014; 63: 741-748
        • Kleinman L.C.
        • Norton E.C.
        What's the risk? A simple approach for estimating adjusted risk measures from nonlinear models including logistic regression.
        Health Serv Res. 2009; 44: 288-302
        • Hill H.A.
        • Elam-Evans L.D.
        • Yankey D.
        • et al.
        National, state, and selected local area vaccination coverage among children aged 19–35 months—United States, 2014.
        MMWR Morb Mortal Wkly Rep. 2015; 64: 889-896
        • Saville A.W.
        • Beaty B.
        • Dickinson L.M.
        • et al.
        Novel immunization reminder/recall approaches: rural and urban differences in parent perceptions.
        Acad Pediatr. 2014; 14: 249-255
        • Saville A.W.
        • Gurfinkel D.
        • Sevick C.
        • et al.
        Provider preferences and experiences with a countywide centralized collaborative reminder/recall for childhood immunizations.
        Acad Pediatr. 2016; 16: 50-56