A Randomized Trial of the Effect of Centralized Reminder/Recall on Immunizations and Preventive Care Visits for Adolescents

Published:March 18, 2013DOI:



      To assess the impact of a managed care-based patient reminder/recall system on immunization rates and preventive care visits among low-income adolescents.


      We conducted a randomized controlled trial between December 2009 and December 2010 that assigned adolescents aged 11–17 years to one of three groups: mailed letter, telephone reminders, or control. Publicly insured youths (n = 4115) were identified in 37 participating primary care practices. The main outcome measures were immunization rates for routine vaccines (meningococcus, pertussis, HPV) and preventive visit rates at study end.


      Intervention and control groups were similar at baseline for demographics, immunization rates, and preventive visits. Among adolescents who were behind at the start, immunization rates at study end increased by 21% for mailed (P < .01 vs control), 17% for telephone (P < .05), and 13% for control groups. The proportion of adolescents with a preventive visit (within 12 months) was: mailed (65%; P < .01), telephone (63%; P < .05), and controls (59%). The number needed to treat for an additional fully vaccinated adolescent was 14 for mailed and 25 for telephone reminders; for an additional preventive visit, it was 17 and 29. The intervention cost $18.78 (mailed) or $16.68 (phone) per adolescent per year to deliver. The cost per additional adolescent fully vaccinated was $463.99 for mailed and $714.98 for telephone; the cost per additional adolescent receiving a preventive visit was $324.75 and $487.03.


      Managed care-based mail or telephone reminder/recall improved adolescent immunizations and preventive visits, with modest costs and modest impact.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Academic Pediatrics
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Markowitz L.E.
        • Dunne E.F.
        • Saraiya M.
        • et al.
        Quadrivalent human papillomavirus vaccine: recommendations of the Advisory Committee on Immunization Practices (ACIP).
        MMWR Recomm Rep. 2007; 56: 1-24
        • Broder K.R.
        • Cortese M.M.
        • Iskander J.K.
        • et al.
        Preventing tetanus, diphtheria, and pertussis among adolescents: use of tetanus toxoid, reduced diphtheria toxoid and acellular pertussis vaccines recommendations of the Advisory Committee on Immunization Practices (ACIP).
        MMWR Recomm Rep. 2006; 55: 1-34
      1. National and state vaccination coverage among adolescents aged 13 through 17 years—United States.
        MMWR Morb Mortal Wkly Rep. 2010; 2011: 1117-1123
      2. National, state, and local area vaccination coverage among adolescents aged 13–17 years—United States.
        MMWR Morb Mortal Wkly Rep. 2009; 2010: 1018-1023
        • Szilagyi P.G.
        • Rand C.M.
        • McLaurin J.
        • et al.
        Delivering adolescent vaccinations in the medical home: a new era?.
        Pediatrics. 2008; 121: S15-S24
        • Szilagyi P.
        • Vann J.
        • Bordley C.
        • et al.
        Interventions aimed at improving immunization rates.
        Cochrane Database Syst Rev. 2002; : CD003941
        • Szilagyi P.G.
        • Bordley C.
        • Vann J.C.
        • et al.
        Effect of patient reminder/recall interventions on immunization rates: a review.
        JAMA. 2000; 284: 1820-1827
        • Szilagyi P.G.
        • Adams W.G.
        Text messaging: a new tool for improving preventive services.
        JAMA. 2012; 307: 1748-1749
        • Briss P.A.
        • Rodewald L.E.
        • Hinman A.R.
        • et al.
        Reviews of evidence regarding interventions to improve vaccination coverage in children, adolescents, and adults. The Task Force on Community Preventive Services.
        Am J Prev Med. 2000; 18: 97-140
      3. Recommendations regarding interventions to improve vaccination coverage in children, adolescents, and adults. Task Force on Community Preventive Services.
        Am J Prev Med. 2000; 18: 92-96
      4. Task Force on Community Preventive Services. Guide to Community Preventive Services. Vaccinations to prevent diseases: universally recommended vaccinations. Available at: Accessed January 16, 2013.

        • Broder K.R.
        • Cohn A.C.
        • Schwartz B.
        • et al.
        Adolescent immunizations and other clinical preventive services: a needle and a hook?.
        Pediatrics. 2008; 121: S25-S34
        • Rand C.M.
        • Szilagyi P.G.
        • Albertin C.
        • et al.
        Additional health care visits needed among adolescents for human papillomavirus vaccine delivery within medical homes: a national study.
        Pediatrics. 2007; 120: 461-466
        • Rand C.M.
        • Szilagyi P.G.
        • Yoo B.K.
        • et al.
        Additional visit burden for universal influenza vaccination of US school-aged children and adolescents.
        Arch Pediatr Adolesc Med. 2008; 162: 1048-1055
        • Szilagyi P.G.
        • Schaffer S.
        • Barth R.
        • et al.
        Effect of telephone reminder/recall on adolescent immunization and preventive visits: results from a randomized clinical trial.
        Arch Pediatr Adolesc Med. 2006; 160: 157-163
        • Szilagyi P.G.
        • Humiston S.G.
        • Gallivan S.
        • et al.
        Effectiveness of a citywide patient immunization navigator program on improving adolescent immunizations and preventive care visit rates.
        Arch Pediatr Adolesc Med. 2011; 165: 547-553
        • Suh C.A.
        • Saville A.
        • Daley M.F.
        • et al.
        Effectiveness and net cost of reminder/recall for adolescent immunizations.
        Pediatrics. 2012; 129: e1437-e1445
        • LeBaron C.W.
        • Starnes D.M.
        • Rask K.J.
        The impact of reminder-recall interventions on low vaccination coverage in an inner-city population.
        Arch Pediatr Adolesc Med. 2004; 158: 255-261
        • Schaffer S.J.
        • Humiston S.G.
        • Shone L.P.
        • et al.
        Adolescent immunization practices: a national survey of US physicians.
        Arch Pediatr Adolesc Med. 2001; 155: 566-571
        • Tierney C.D.
        • Yusuf H.
        • McMahon S.R.
        • et al.
        Adoption of reminder and recall messages for immunizations by pediatricians and public health clinics.
        Pediatrics. 2003; 112: 1076-1082
        • Saville A.W.
        • Albright K.
        • Nowels C.
        • et al.
        Getting under the hood: exploring issues that affect provider-based recall using an immunization information system.
        Acad Pediatr. 2011; 11: 44-49
        • Kempe A.
        • Wortley P.
        • O'Leary S.
        • et al.
        Pediatricians' attitudes about collaborations with other community vaccinators in the delivery of seasonal influenza vaccine.
        Acad Pediatr. 2012; 1: 26-35
        • Kempe A.
        • Lowery N.E.
        • Pearson K.A.
        • et al.
        Immunization recall: effectiveness and barriers to success in an urban teaching clinic.
        J Pediatr. 2001; 139: 630-635
        • Daley M.F.
        • Steiner J.F.
        • Kempe A.
        • et al.
        Quality improvement in immunization delivery following an unsuccessful immunization recall.
        Ambul Pediatr. 2004; 4: 217-223
        • Davis M.M.
        • Szilagyi P.G.
        Can quality improvement reach into pockets of need for childhood immunizations?.
        Ambul Pediatr. 2004; 4: 224-225
        • Hambidge S.J.
        • Davidson A.J.
        • Phibbs S.L.
        • et al.
        Strategies to improve immunization rates and well-child care in a disadvantaged population: a cluster randomized controlled trial.
        Arch Pediatr Adolesc Med. 2004; 158: 162-169
        • Rodewald L.E.
        • Szilagyi P.G.
        • Humiston S.G.
        • et al.
        A randomized study of tracking with outreach and provider prompting to improve immunization coverage and primary care.
        Pediatrics. 1999; 103: 31-38
        • Hambidge S.J.
        • Phibbs S.L.
        • Chandramouli V.
        • et al.
        A stepped intervention increases well-child care and immunization rates in a disadvantaged population.
        Pediatrics. 2009; 124: 455-464
        • Szilagyi P.G.
        • Schaffer S.
        • Shone L.
        • et al.
        Reducing geographic, racial, and ethnic disparities in childhood immunization rates by using reminder/recall interventions in urban primary care practices.
        Pediatrics. 2002; 110: e58
        • Humiston S.G.
        • Bennett N.M.
        • Long C.
        • et al.
        Increasing inner-city adult influenza vaccination rates: a randomized controlled trial.
        Public Health Rep. 2011; 126: 39-47
        • Coker T.R.
        • Duplessis H.M.
        • Davoudpour R.
        • et al.
        Well-child care practice redesign for low-income children: the perspectives of health plans, medical groups, and state agencies.
        Acad Pediatr. 2012; 12: 43-52
        • Rosenbaum S.
        The patient protection and affordable care act and the future of child health policy.
        Acad Pediatr. 2012; 12: 363-364
      5. NCQA. HEDIS & Quality Measurement. 2012. Available at: Accessed January 16, 2013.

        • Fairbrother G.
        • Simpson L.A.
        Measuring and reporting quality of health care for children: CHIPRA and beyond.
        Acad Pediatr. 2011; 11: S77-S84
        • Dougherty D.
        • Clancy C.
        Transforming children's health care quality and outcomes–a not-so-random non-linear walk across the translational continuum.
        Acad Pediatr. 2011; 11: S91-S94
        • Mangione-Smith R.
        • Schiff J.
        • Dougherty D.
        Identifying children's health care quality measures for Medicaid and CHIP: an evidence-informed, publicly transparent expert process.
        Acad Pediatr. 2011; 11: S11-S21
        • Dougherty D.
        • Schiff J.
        • Mangione-Smith R.
        The Children's Health Insurance Program Reauthorization Act quality measures initiatives: moving forward to improve measurement, care, and child and adolescent outcomes.
        Acad Pediatr. 2011; 11: S1-S10
        • Glasgow R.E.
        • Vogt T.M.
        • Boles S.M.
        Evaluating the public health impact of health promotion interventions: the RE-AIM framework.
        Am J Public Health. 1999; 89: 1322-1327
        • Rodewald L.
        • Maes E.
        • Stevenson J.
        • et al.
        Immunization performance measurement in a changing immunization environment.
        Pediatrics. 1999; 103: 889-897
      6. U.S. Bureau of Labor Statistics. May 2011 Occupational Employment and Wage EstimatesF0BENational Cross-Industry estimates. Available at: Accessed November 28, 2012.

        • Williams R.L.
        A note on robust variance estimation for cluster-correlated data.
        Biometrics. 2000; 56: 645-646
        • Lagakos S.W.
        The challenge of subgroup analyses–reporting without distorting.
        N Engl J Med. 2006; 354: 1667-1669
        • Hoch J.S.
        • Blume J.D.
        Measuring and illustrating statistical evidence in a cost-effectiveness analysis.
        J Health Econ. 2008; 27: 476-495
      7. Task Force on Community Preventive Services. Guide to Community Preventive Services. Universally recommended vaccinations: immunization information systems.

        • Dombkowski K.J.
        • Cowan A.E.
        • Harrington L.B.
        • et al.
        Feasibility of initiating and sustaining registry-based immunization recall in private practices.
        Acad Pediatr. 2012; 12: 104-109
        • Stockwell M.S.
        • Kharbanda E.O.
        • Martinez R.A.
        • et al.
        Effect of a text messaging intervention on influenza vaccination in an urban, low-income pediatric and adolescent population: a randomized controlled trial.
        JAMA. 2012; 307: 1702-1708
        • Stockwell M.S.
        • Kharbanda E.O.
        • Martinez R.A.
        • et al.
        Text4Health: impact of text message reminder-recalls for pediatric and adolescent immunizations.
        Am J Public Health. 2012; 102: e15-e21
        • Eberly T.
        • Davidoff A.
        • Miller C.
        Managing the gap: evaluating the impact of Medicaid managed care on preventive care receipt by child and adolescent minority populations.
        J Health Care Poor Underserved. 2010; 21: 92-111
        • Chien A.T.
        • Li Z.
        • Rosenthal M.B.
        Improving timely childhood immunizations through pay for performance in Medicaid-managed care.
        Health Serv Res. 2010; 45: 1934-1947
        • Kempe A.
        • Barrow J.
        • Stokley S.
        • et al.
        Effectiveness and cost of immunization recall at school-based health centers.
        Pediatrics. 2012; 129: e1446-e1452
        • Franzini L.
        • Rosenthal J.
        • Spears W.
        • et al.
        Cost-effectiveness of childhood immunization reminder/recall systems in urban private practices.
        Pediatrics. 2000; 106: 177-183
        • Lieu T.A.
        • Black S.B.
        • Ray P.
        • et al.
        Computer-generated recall letters for underimmunized children: how cost-effective?.
        Pediatr Infect Dis J. 1997; 16: 28-33
        • Wood D.
        • Halfon N.
        • Donald-Sherbourne C.
        • et al.
        Increasing immunization rates among inner-city, African American children. A randomized trial of case management.
        JAMA. 1998; 279: 29-34
        • Clayton A.E.
        • McNutt L.A.
        • Homestead H.L.
        • et al.
        Public health in managed care: a randomized controlled trial of the effectiveness of postcard reminders.
        Am J Public Health. 1999; 89: 1235-1237
        • Kulasingam S.L.
        • Myers E.R.
        Potential health and economic impact of adding a human papillomavirus vaccine to screening programs.
        JAMA. 2003; 290: 781-789
        • Shepard C.W.
        • Ortega-Sanchez I.R.
        • Scott 2nd, R.D.
        • et al.
        Cost-effectiveness of conjugate meningococcal vaccination strategies in the United States.
        Pediatrics. 2005; 115: 1220-1232
        • Ortega-Sanchez I.R.
        • Lee G.M.
        • Jacobs R.J.
        • et al.
        Projected cost-effectiveness of new vaccines for adolescents in the United States.
        Pediatrics. 2008; 121: S63-S78
        • Ziv A.
        • Boulet J.R.
        • Slap G.B.
        Utilization of physician offices by adolescents in the United States.
        Pediatrics. 1999; 104: 35-42
        • Szilagyi P.G.
        Translational research and pediatrics.
        Acad Pediatr. 2009; 9: 71-80