Abstract
Reminders, alerting patients to the need for vaccines that will be due in the future, and recall messages, informing patients about vaccines that are overdue, have been shown to improve immunization rates for children and adolescents in numerous systematic reviews. Therefore, reminder and recall interventions (R/R) are recommended by the Task Force on Community Preventive Services for increasing immunization rates on the basis of strong evidence. R/R messages can be delivered by mail (letter or postcard), via personal or auto-dialer phone calls, by text or e-mail or via patient-portals and can simply be alerts to action or can include educational material with the aim of motivating patients to seek vaccination. R/R has also been shown to be a relatively low-cost intervention with high cost-effectiveness compared with other recommended strategies. However, although R/R as a strategy is consistently effective and cost-effective overall, there is wide variation in the impact of R/R by 1) modality of how it is delivered, 2) the targeted vaccine, 3) the age group, and 4) whether the R/R is conducted centrally by a health system or Immunization Information System or by individual practices. This narrative review summarizes the literature about effectiveness of R/R within each of these categories. We also discuss limitations of R/R, with a focus on the potential impact of parental vaccine hesitancy in blunting its effectiveness and problems with data integrity, on which R/R relies. We also discuss challenges to sustaining R/R efforts, including potential methods of funding for R/R efforts.
Keywords
Overview of Reminder/Recall
Reminder/recall (R/R) of patients is an evidence-based strategy that has been shown to improve immunization rates for children and adolescents.
1
Reminder messages alert patients to the need for vaccines that will be due in the future and recall messages inform patients about vaccines that are overdue; however, the terms are generally combined because efforts to remind and recall usually occur together. Important domains characterizing R/R efforts are summarized in Table 1. R/R can be delivered by mail (letter or postcard), via personal or auto-dialer phone calls, by text or e-mail or via patient-portals. Such messages can simply be alerts to action or can include educational material that aims to increase vaccine health literacy with the aim of motivating patients to seek vaccination. The number of R/R messages sent varies, but is often between one and 3 messages, optimally with removal of patients who have received vaccines within each R/R cycle. One of the most important variations in implementation of R/R efforts is the unit of delivery, most commonly either at the clinical practice level or via an entity using a centralized data system, such as the public health department using an immunization information system (IIS) or a health system using a centralized patient database.- Jacobson Vann JC
- Jacobson RM
- Coyne-Beasley T
- et al.
Patient reminder and recall interventions to improve immunization rates.
Cochrane Database Syst Rev. 2018; 1CD003941https://doi.org/10.1002/14651858.CD003941.pub3
Table 1Key Domains Characterizing Reminder/Recall (R/R) Efforts
| R/R Characteristic | Options |
|---|---|
| Delivery modality | Mail (letter or postcard), Phone (auto-dialer or personal call), text, e-mail, patient portals |
| Frequency | Usually 1, 2 or 3 reminders with responders removed between efforts |
| Target vaccine(s) | Often routine childhood, adolescent, or adult vaccines but may target an individual vaccine such as HPV or Influenza vaccines |
| Age group | Often 20-35 months, school-age, adolescent, all adults, >65 year old adults |
| Content | Simple cue to action or can include educational or motivational content |
| Unit of delivery | Often individual practices (practice-based R/R) or health care systems or public health departments using an Immunization Information System (centralized R/R) |
Table 2Effectiveness of Patient Reminder/Recall (R/R) Using Different Modalities Compared With No R/R for Receipt of Immunizations
| Intervention type | Relative Risk (95% Confidence Interval) | Certainty of Evidence (GRADE) |
|---|---|---|
| Patient postcard reminder or recall | RR 1.18 (1.08–1.30) | High |
| Patient text message reminder or recall | RR 1.29 (1.15–1.44) | High |
| Patient auto-dialer message reminder or recall | RR 1.17 (1.03–1.32) | High |
| Patient reminder or recall summary measure | RR 1.28 (1.23–1.35) | Moderate |
| Patent telephone reminder or recall | RR 1.75 (1.20–2.54) | Moderate |
| Patient letter reminder or recall | RR 1.29 (1.21–1.38) | Moderate |
| Combination of patient reminder or recall with outreach intervention | RR 1.22 (1.10–1.35) | High |
| Combination of patient mail and telephone reminder or recall | RR 1.28 (1.14–1.45) | Moderate |
| Combination of patient reminder or recall with provider reminder intervention | RR 2.91 (2.67–3.19) | Moderate |
Adapted from: Jacobson Vann et al.
1
- Jacobson Vann JC
- Jacobson RM
- Coyne-Beasley T
- et al.
Patient reminder and recall interventions to improve immunization rates.
Cochrane Database Syst Rev. 2018; 1CD003941https://doi.org/10.1002/14651858.CD003941.pub3
Table 3Effectiveness of Patient Reminder/Recall (R/R) for Different Age Groups and Vaccines Compared With No R/R for Receipt of Immunizations
| Type of Immunization | Relative Risk (95% Confidence Interval) | Certainty of Evidence (GRADE) |
|---|---|---|
| Childhood immunizations | RR 1.22 (1.15–1.29) | High |
| Adolescent immunizations | RR 1.29 (1.17–1.42) | High |
| Childhood influenza immunizations | RR 1.51 (1.14–1.99) | Moderate |
| Adult immunizations—Other than influenza or travel (‘Other adult’) | RR 2.08 (0.91–4.78) | Low |
| Adult influenza immunizations | RR 1.29 (1.17–1.43) | Moderate |
Adapted from: Jacobson Vann.
1
- Jacobson Vann JC
- Jacobson RM
- Coyne-Beasley T
- et al.
Patient reminder and recall interventions to improve immunization rates.
Cochrane Database Syst Rev. 2018; 1CD003941https://doi.org/10.1002/14651858.CD003941.pub3
Effectiveness of R/R
The effectiveness of R/R as a general strategy has been well-documented in numerous studies summarized in multiple Cochrane reviews.
1
, - Jacobson Vann JC
- Jacobson RM
- Coyne-Beasley T
- et al.
Patient reminder and recall interventions to improve immunization rates.
Cochrane Database Syst Rev. 2018; 1CD003941https://doi.org/10.1002/14651858.CD003941.pub3
2
, - Jacobson Vann JC
- Szilagyi P
Patient reminder and patient recall systems to improve immunization rates.
Cochrane Database Syst Rev. 2005; 2005Cd003941https://doi.org/10.1002/14651858.CD003941.pub2
3
, 4
It is recommended by the Task Force on Community Preventive Services as a method for increasing immunization rates- Szilagyi P
- Vann J
- Bordley C
- et al.
Interventions aimed at improving immunization rates.
Cochrane Database Syst Rev. 2002; Cd003941https://doi.org/10.1002/14651858.cd003941
5
on the basis of strong evidence. The 2018 Cochrane review,Community Preventive Services Task Force. The Community Guide: Vaccination Programs: Client Reminder and Recall Systems. Centers for Disease Control and Prevention. Available at:https://www.thecommunityguide.org/findings/vaccination-programs-client-reminder-and-recall-systems. Accessed October 21, 2019.
1
included 75 studies conducted in 10 countries, with the majority of studies being in the US. On average, R/R improved the proportion of patients who received immunizations by roughly 8 percentage points (ranging from 5-20 percentage points) with a risk ratio (RR) of 1.28 (95% confidence intervals [CI] 1.23–1.35). All the Cochrane reviews have been fairly consistent in their overall findings.- Jacobson Vann JC
- Jacobson RM
- Coyne-Beasley T
- et al.
Patient reminder and recall interventions to improve immunization rates.
Cochrane Database Syst Rev. 2018; 1CD003941https://doi.org/10.1002/14651858.CD003941.pub3
1
, - Jacobson Vann JC
- Jacobson RM
- Coyne-Beasley T
- et al.
Patient reminder and recall interventions to improve immunization rates.
Cochrane Database Syst Rev. 2018; 1CD003941https://doi.org/10.1002/14651858.CD003941.pub3
2
, - Jacobson Vann JC
- Szilagyi P
Patient reminder and patient recall systems to improve immunization rates.
Cochrane Database Syst Rev. 2005; 2005Cd003941https://doi.org/10.1002/14651858.CD003941.pub2
3
, 4
However, although R/R as a strategy is consistently effective overall, there is wide variation in the impact of R/R by 1) modality of how it is delivered, 2) focus of R/R (by age or specific vaccine) and 3) whether the R/R is conducted centrally by a system or IIS or by individual practices. Therefore, we will discuss the impact of R/R within each of these categories.- Szilagyi P
- Vann J
- Bordley C
- et al.
Interventions aimed at improving immunization rates.
Cochrane Database Syst Rev. 2002; Cd003941https://doi.org/10.1002/14651858.cd003941
Effectiveness of R/R by Delivery Modality
All modes of delivery were found by the Cochrane review to be effective, although there was variability in the quality of the studies supporting the evidence. Text messaging showed the highest level of effectiveness for a single reminder method based on a high certainty of evidence, while patient telephone R/R showed the largest effect based on moderate certainty of evidence. R/R combining different modalities generally showed similar levels of effectiveness as single methods. However, there was substantial variation in effectiveness of different modalities by study, reflecting variation in the number of messages used in different protocols, how different modalities were combined, and the populations being targeted. As an example, although the literature shows substantial effectiveness of text messaging among many patient populations,
6
, 7
, 8
, 9
it has been less effective in other studies involving populations where texting from a health provider is novel, where mobile phones are not ubiquitous, or among young adult populations.10
, 11
, 12
, 13
, 14
, 15
, 16
Studies that combined patient R/R with other interventions (such as prompts to healthcare providers at the time of a visit, or outreach to patients using outreach workers) appear to have a greater impact on vaccination rates.
17
, 18
, 19
However, the systematic review could not assess many of these multi-component studies quantitatively because of their heterogeneity in modalities and their inability to distinguish the impact of patient R/R from the impact of other interventions.1
The review did evaluate the combination of R/R with outreach (generally targeting low-income populations), and noted improved vaccination rates with high certainty of evidence.- Jacobson Vann JC
- Jacobson RM
- Coyne-Beasley T
- et al.
Patient reminder and recall interventions to improve immunization rates.
Cochrane Database Syst Rev. 2018; 1CD003941https://doi.org/10.1002/14651858.CD003941.pub3
Variation in the modalities of R/R that patients or parents prefer to receive may play a role in their responsiveness to the R/R intervention. Previous survey data demonstrated different preferences for R/R modalities between urban versus rural populations, within different socioeconomic strata and by type of health care delivery facility, especially regarding text messaging.
20
, 21
, 22
, 23
However, given the changing landscape of digital technology and health care delivery in general, it is likely that patient preferences will be a moving target. There is also some evidence that allowing patients to choose the method of R/R, “preference-based R/R” can be more effective than strategies in which the method is dictated. For example, one study using preference-based R/R showed substantially higher effect sizes than seen in most previous R/R trials for any vaccine, with an absolute increase of 25 percentage points for HPV completion (63% vs 38%) compared with no R/R.24
- Kempe A
- O'Leary ST
- Shoup JA
- et al.
Parental choice of recall method for HPV vaccination: a pragmatic trial.
Pediatrics. 2016; 137e20152857https://doi.org/10.1542/peds.2015-2857
Effectiveness of R/R Within Different Age Groups or for Different Vaccines
R/R has been shown to be effective for increasing vaccination rates across the age spectrum, although there is variability in effectiveness by type of vaccine and age group. In the 2018 Cochrane review, risk differences averaged 8 absolute percentage points between R/R and control (23 studies) for routine childhood vaccinations and 7 percentage points (10 studies) for adolescent vaccinations. The data regarding effectiveness for childhood influenza suggested relatively large effect sizes, 17-26 absolute percentage points higher than controls (5 studies); however 4 of the 5 studies were among children with high-risk medical conditions. The one study that focused on healthy children
25
found an increase of 4.4 absolute percentage points (62.4% vs 58.0%) among children 6-23 months of age. Similarly, another more recent trial not included in the Cochrane review focusing on R/R for healthy children of all age groups and using text reminders, found an absolute difference of 3.7% (43.6% vs 39.9%).26
Therefore, the effect size for R/R for influenza vaccine for healthy children may be lower than the effect for children with chronic conditions and also lower compared with other childhood vaccines.Effectiveness of Centralized vs Practice-Based R/R
There are 2 important limitations to the literature on patient R/R for vaccinations. First, many studies have been done in single practices or small numbers of practices. Second, the majority of studies included in Cochrane reviews of the effectiveness of R/R were based on research studies done with the assistance of an outside research study team and few measured the sustainability of R/R after such a trial. Previous national data suggested that only 16% of providers were doing any type of R/R
27
on a routine basis and it is unknown how many are using an IIS to do so. Identified barriers to practice-based R/R cited by providers have included insufficient staff time and competing demands of primary care, staff turnover, costs related to mailings or telephone call reminders and the lack of computerized systems to readily identify patients who need immunizations.27
,28
However, a number of studies have shown that practice-based R/R is rarely implemented or sustained even when practices are offered training, have access to an IIS, and are provided with technical assistance and funding to cover their costs.28
, 29
, 30
To address the challenges of cost, sustainability, and scalability, centralized R/R has been studied as an alternative to practice-based R/R and as a mechanism to scale-up patient R/R. Most often, this has taken the form of state or county health departments conducting R/R on a population level using a state or regional IIS. Centralized R/R not only has the advantage of unburdening practices, but also has the potential to reach children who are in the IIS but not currently affiliated with a practice site. A recent survey of 57 of the 62 existing IIS programs across the country demonstrated that 61% had conducted centralized R/R for counties or geographic areas, 84% focused on the childhood series, 70% on HPV, 61% on other adolescent vaccines, and 35% for childhood influenza.
31
There have been several large trials comparing centralized IIS-based R/R by public health departments to practice-based approaches for increasing routine childhood vaccines among young children demonstrating significantly higher rates at lower cost achieved with the centralized approach. Relative risks estimates for the centralized compared with the practice-based R/R approach among 19-35 month children in one study were 1.23 (95% CI 1.10, 1.37) for becoming up-to-date (UTD) and 1.26 (95% CI 1.115–1.38) for receipt of any needed vaccine.- Saville AW
- Gurfinkel D
- Beaty BL
- et al.
The potential for centralized reminder/recall to increase immunization rates: a national survey of immunization information systems (IIS) managers.
Prev Med Rep. 2021; 21101296https://doi.org/10.1016/j.pmedr.2020.101296
29
In a different study adjusted odd ratios were 1.34 (95% CI 1.09–1.64) for becoming UTD and 1.31 (95% CI 1.16–1.48) for receipt of a needed vaccine.30
Particularly notable were the substantially lower cost of centralized versus practice-based approaches, $17 versus $62 per child brought UTD in the first study29
and $24.72 versus $124.45 per child brought UTD in the second.30
Other studies of centralized R/R conducted within systems of care that have centralized databases, including Medicaid HMOs,32
,33
county Medicaid programs,34
networks of community health centers,6
or Kaiser HMOs35
have shown similar success with centralized R/R for a variety of vaccines.Surprisingly, the effects of centralized R/R approaches for increasing rates of influenza for children and HPV vaccines have been more modest. Two recent large trials examined the effectiveness of IIS-based R/R by State Public Health Departments in both New York State and Colorado compared with usual care.
36
, 37
, 38
, - Szilagyi P
- Albertin C
- Gurfinkel D
- et al.
Effect of state immunization information system centralized reminder and recall on HPV vaccination rates.
Pediatrics. 2020; 145https://doi.org/10.1542/peds.2019-2689
39
Effects of IIS-based centralized R/R for increasing influenza rates among children in 2 studies were modest, with maximum differences in receipt of influenza vaccine of 1.7 percentage points (25.3% vs 23.6%) between R/R and control and adjusted risk ratios varying from 1.05 to 1.06 across the 2 states. Similarly, IIS-based centralized R/R for increasing HPV vaccine initiation showed a maximum differences of 1.3 absolute percentages points (33.5% vs 31.2%) in intervention versus controls in Colorado (maximum adjusted risk ratio between intervention and control arms of 1.07) and no significant effect in New York. The methods used in these IIS-based centralized R/R trials for influenza and HPV vaccines were very similar to the methods used by IIS-based centralized R/R trials focused on childhood vaccines; all trials tested multiple reminders and included some educational information within the R/R messages based upon components of the Health Belief Model. However, the trials focusing on childhood vaccines29
,30
noted far larger impact than did the trials focused on influenza or HPV vaccine.36
, 37
, 38
, - Szilagyi P
- Albertin C
- Gurfinkel D
- et al.
Effect of state immunization information system centralized reminder and recall on HPV vaccination rates.
Pediatrics. 2020; 145https://doi.org/10.1542/peds.2019-2689
39
These findings suggest that other factors that may not be as responsive to R/R efforts are playing a dominant role.Limitations of R/R
R/R in the Face of Vaccine Hesitancy
An important factor that may be responsible for the findings of the 2 IIS-based centralized R/R trials discussed above is whether vaccine hesitancy regarding influenza and HPV vaccines is blunting the effectiveness of R/R. Two recent national surveys of parents,
40
,- Kempe A
- Saville AW
- Albertin C
- et al.
Parental hesitancy about routine childhood and influenza vaccinations: a national survey.
Pediatrics. 2020; 146https://doi.org/10.1542/peds.2019-3852
41
one focused on childhood vaccines and childhood influenza and the other on HPV examined the level of hesitancy toward these 3 types of vaccines using modified versions of the Vaccine Hesitancy Scale, developed by the World Health Organization. The prevalence of hesitancy as defined by this scale was 6% for routine childhood vaccination, 26% for childhood influenza vaccines and 23% for HPV vaccines. In the face of much higher levels of hesitancy, simply reminding or recalling parents for a vaccination may have minimal impact when parents have concerns about that vaccine. Therefore, hesitancy may explain the higher level of effectiveness of centralized R/R for routine childhood vaccines, in contrast to both HPV and influenza vaccines, due to much higher levels of hesitancy for HPV and influenza vaccines than for childhood vaccines.In the face of high vaccine hesitancy, R/R may need to do more than simply cue caregivers or patients about the need for a vaccine. As one example in a group of parents whose children were not yet vaccinated against influenza by November, interactive text messaging in which a parent could request further information via the text message was effective versus usual care while conventional text message reminders were not.
42
Message effectiveness may be further enhanced by including principles of behavioral economics such as message framing or utilizing social norms.43
However a better understanding of what principles work best for which vaccines may be needed since not all studies suggest these methods are effective for increasing vaccination.44
The role of hesitancy is suggested by comparing the impact of reminders for initial versus second influenza vaccination among children eligible for 2 doses. Two studies noted relatively large impact for the second dose
44
,45
while several contemporary studies (and one using the same population) did not note impact for the first dose, presumably because of high parental hesitancy. These findings suggest parental/patient hesitancy plays a major role in modulating the impact of R/R.Potential Impact of R/R If Scaled Up
It is important to note, however, that even small increases in vaccination rates such as those seen for centralized R/R for HPV or influenza may have substantial benefit at the population level. For example, a recent modeling study demonstrated that a 5% increase in influenza vaccine coverage among children could result in 282,000 fewer influenza infections and 1,440 fewer hospitalizations during a severe influenza season.
46
Modeling for smaller effect sizes has not been done, but across populations even differences of 1-2% may be very important. Nonetheless, interventions to increase rates for vaccines associated with higher rates of hesitancy will, ideally require a multi-dimensional intervention rather than R/R alone.R/R and Data Quality
There are other limitations of R/R that may limit its impact. Principal among these are the fact that R/R is only as good as the data driving it. Immunization records may be fragmented at any one site of care, especially in low-income populations or when immunizations are given in retail pharmacies, potentially leading to erroneous R/R based on incomplete records.
47
, 48
, 49
If an IIS is used for R/R, immunization data may be limited by underreporting of vaccines, as IIS reporting is not mandatory in many states. IISs often suffer from incorrect contact information, because such data are not necessarily updated on a regular basis. Despite the fact that real-time electronic interfaces between practice data and IISs (HL7) have recently increased,31
contact information is frequently not uploaded automatically and may require administration of a vaccine to trigger updating. Data also suggest that parents may be more responsive to R/R that appears to come from their child's provider- Saville AW
- Gurfinkel D
- Beaty BL
- et al.
The potential for centralized reminder/recall to increase immunization rates: a national survey of immunization information systems (IIS) managers.
Prev Med Rep. 2021; 21101296https://doi.org/10.1016/j.pmedr.2020.101296
30
,50
and they usually prefer personalized messages,22
therefore R/R messages should be personalized if at all possible.Cost and Cost-Effectiveness of R/R
One of the most compelling aspects of R/R as an approach for increasing vaccination rates is its relative low cost and high-cost-effectiveness compared with other recommended strategies. A recent economic systematic review of the twelve interventions recommended by the Community Preventive Services Task Force to increase vaccination coverage
51
compared the reach, cost and cost effectiveness of each strategy. Patient reminder systems were the lowest cost interventions to implement and demonstrated the lowest cost per additional person vaccinated of any intervention. The number of patients reached was highest for interventions that were conducted within systems rather than those conducted by individual practices. These findings additionally support the use of a centralized R/R approach using data from a larger system to increase sustainment and efficiency.Sustainability of R/R Efforts
Despite all the data demonstrating effectiveness, cost-effectiveness and efficiency of R/R, a central dilemma remains. Whose responsibility is it to fund R/R efforts to assure they are sustainable? Even when the most efficient and least costly methods are used, such as IIS-based centralized R/R, there are some costs involved, primarily personnel costs. The current pandemic has highlighted the need for and critical importance of a strong public health infrastructure and since immunizations are one of the ten most important public health advances,
52
in the ideal world R/R would largely be funded by public health, even if delivered within health systems or by practices. Public health departments do report receiving funding for centralized R/R from grants from the Centers for Disease Control and Prevention (CDC), contractual agreements with the CDC, or as part of the CDC's Sentinel Sites Program.53
They sometimes receive supplies for R/R or other assistance from pharmaceutical companies or regional and state coalitions promoting immunizations and, infrequently, receive funds directly from the state.53
However, these opportunities are variable and there is rarely sustained, dependable funding for public health departments to conduct centralized R/R activities. Of note, New York City's IIS, in addition to other state IISs, has taken on a more hybrid approach that allows health care providers to identify children from their practices who are in the registry who need vaccination, and use the registry to create recall lists, letters or even to send a text message from the IIS.54
Citywide Immunization Registry. New Text Messaging via the Online Registry to Recall Patients Due/Overdue Immunizations. New York City Department of Health and Mental Hygiene. Available at: https://www1.nyc.gov/assets/doh/downloads/pdf/cir/txt-messaging-guide.pdf. Accessed February 1, 2021.
There is little information about sources of funding for R/R conducted by health delivery systems, although pay for performance measures by insurers or HEDIS performance measures may make R/R efforts worth the investment if up-to-date rates for relevant vaccines are key performance indicators.
53
Individual clinical practices may also benefit from such measures, but the lack of sustainment of practice-based R/R suggests this does not provide sufficient support for practices to maintain R/R.It seems logical that sustaining R/R efforts would optimally be supported via partnerships between the public and private entities that have a stake in raising immunization rates including health departments, insurers, health care systems and practices. Such collaborations would be ideal examples of the types of multi-sector initiatives that have been strongly supported by multiple reports by the Institutes of Medicine, now the National Academy of Medicine,
55
, 56
, 57
the Robert Wood Johnson Foundation58
and other organizations advocating for collaborative models. They would have the potential to generate a larger collective impact and population-level impact than any one organization independently and, because costs would be shared, would be less of a burden on any single entity.Conclusions
Despite some challenges posed by parental hesitancy, data integrity issues, and problems with sustainability, issues that are common to most strategies used to increase immunizations, evidence suggests that R/R is one of the most effective and cost-effective modalities and should continue to play a major role in efforts to increase immunization rates for many vaccines. Multi-level interventions, combining R/R with other interventions at the provider, practice, and system levels, will likely be needed to increase rates for those vaccines that are associated with higher levels of vaccine hesitancy. In addition, enhancement of R/R with educational or motivational messages and increasing the interactivity of messaging are innovations that may increase effectiveness in the face of vaccine hesitancy. Increased interoperability of electronic medical records and IISs over time with the potential for automatic updates of addresses from administrative data is likely to increase effectiveness and efficiency of centralized R/R using either IIS or other health system level data. Sustainment of R/R could optimally be supported via partnerships between public and private entities that have a stake in raising immunization rates including businesses, health departments, insurers, health care systems and practices. There are many examples of such cross-sector collaborations in community efforts to improve health which could serve as examples for R/R efforts
59
for an entire community. Such a collaborative approach would decrease the burden on each individual entity and increase the opportunity for a truly population-based approach to increasing immunization rates.Acknowledgments
Financial disclosure: This article was published as part of a supplement sponsored by the Centers for Disease Control and Prevention.
Disclaimer: The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.
References
- Patient reminder and recall interventions to improve immunization rates.Cochrane Database Syst Rev. 2018; 1CD003941https://doi.org/10.1002/14651858.CD003941.pub3
- Patient reminder and patient recall systems to improve immunization rates.Cochrane Database Syst Rev. 2005; 2005Cd003941https://doi.org/10.1002/14651858.CD003941.pub2
- Effect of patient reminder/recall interventions on immunization rates: a review.JAMA. 2000; 284: 1820-1827https://doi.org/10.1001/jama.284.14.1820
- Interventions aimed at improving immunization rates.Cochrane Database Syst Rev. 2002; Cd003941https://doi.org/10.1002/14651858.cd003941
Community Preventive Services Task Force. The Community Guide: Vaccination Programs: Client Reminder and Recall Systems. Centers for Disease Control and Prevention. Available at:https://www.thecommunityguide.org/findings/vaccination-programs-client-reminder-and-recall-systems. Accessed October 21, 2019.
- Text4Health: impact of text message reminder–recalls for pediatric and adolescent immunizations.Am J Public Health. 2012; 102: e15-e21https://doi.org/10.2105/ajph.2011.300331
- Text message reminders for timely routine MMR vaccination: a randomized controlled trial.Vaccine. 2015; 33: 5741-5746https://doi.org/10.1016/j.vaccine.2015.09.042
- Effectiveness and cost of bidirectional text messaging for adolescent vaccines and well care.Pediatrics. 2015; 136: e1220-e1227https://doi.org/10.1542/peds.2015-1089
- Effect of mobile text message reminders on routine childhood vaccination: a systematic review and meta-analysis.Syst Rev. 2019; 8: 154https://doi.org/10.1186/s13643-019-1054-0
- A randomized intervention study to evaluate whether electronic messaging can increase human papillomavirus vaccine completion and knowledge among college students.J Am Coll Health. 2016; 64: 269-278https://doi.org/10.1080/07448481.2015.1117466
- Staying on track: a cluster randomized controlled trial of automated reminders aimed at increasing human papillomavirus vaccine completion.Vaccine. 2014; 32: 2428-2433https://doi.org/10.1016/j.vaccine.2014.02.095
- A human papillomavirus vaccination program for low-income postpartum women.Am J Obstet Gynecol. 2016; 215: 318.e1-318.e9https://doi.org/10.1016/j.ajog.2016.02.032
- The impact of reminder-recall interventions on low vaccination coverage in an inner-city population.Arch Pediatr Adolesc Med. 2004; 158: 255-261https://doi.org/10.1001/archpedi.158.3.255
- Strategies to improve immunization rates and well-child care in a disadvantaged population: a cluster randomized controlled trial.Arch Pediatr Adolesc Med. 2004; 158: 162-169https://doi.org/10.1001/archpedi.158.2.162
- Immunization recall: effectiveness and barriers to success in an urban teaching clinic.J Pediatr. 2001; 139: 630-635https://doi.org/10.1067/mpd.2001.117069
- Challenges and successes of immunization registry reminders at inner-city practices.Ambul Pediatr. 2006; 6: 100-104https://doi.org/10.1016/j.ambp.2005.10.006
- A randomized study of tracking with outreach and provider prompting to improve immunization coverage and primary care.Pediatrics. 1999; 103: 31-38https://doi.org/10.1542/peds.103.1.31
- Effectiveness of a citywide patient immunization navigator program on improving adolescent immunizations and preventive care visit rates.Arch Pediatr Adolesc Med. 2011; 165: 547-553https://doi.org/10.1001/archpediatrics.2011.73
- A stepped intervention increases well-child care and immunization rates in a disadvantaged population.Pediatrics. 2009; 124: 455-464https://doi.org/10.1542/peds.2008-0446
- Novel immunization reminder/recall approaches: rural and urban differences in parent perceptions.Acad Pediatr. 2014; 14: 249-255https://doi.org/10.1016/j.acap.2014.02.003
- Attitudes about adult vaccines and reminder/recall in a safety net population.Vaccine. 2017; 35: 7292-7296https://doi.org/10.1016/j.vaccine.2017.11.001
- Parental and provider preferences and concerns regarding text message reminder/recall for early childhood vaccinations.Prev Med. Aug 2013; 57: 75-80https://doi.org/10.1016/j.ypmed.2013.04.007
- Parents' experiences with and preferences for immunization reminder/recall technologies.Pediatrics. 2011; 128: e1100-e1105https://doi.org/10.1542/peds.2011-0270
- Parental choice of recall method for HPV vaccination: a pragmatic trial.Pediatrics. 2016; 137e20152857https://doi.org/10.1542/peds.2015-2857
- Implementation of universal influenza immunization recommendations for healthy young children: results of a randomized, controlled trial with registry-based recall.Pediatrics. 2005; 115: 146-154https://doi.org/10.1542/peds.2004-1804
- Effect of a text messaging intervention on influenza vaccination in an urban, low-income pediatric and adolescent population: a randomized controlled trial.JAMA. 2012; 307: 1702-1708
- Adoption of reminder and recall messages for immunizations by pediatricians and public health clinics.Pediatrics. 2003; 112: 1076-1082https://doi.org/10.1542/peds.112.5.1076
- Getting under the hood: exploring issues that affect provider-based recall using an immunization information system.Acad Pediatr. 2011; 11: 44-49
- Population-based versus practice-based recall for childhood immunizations: a randomized controlled comparative effectiveness trial.Am J Public Health. 2013; 103: 1116-1123https://doi.org/10.2105/AJPH.2012.301035
- Collaborative centralized reminder/recall notification to increase immunization rates among young children: a comparative effectiveness trial.JAMA Pediatr. 2015; 169: 365-373https://doi.org/10.1001/jamapediatrics.2014.3670
- The potential for centralized reminder/recall to increase immunization rates: a national survey of immunization information systems (IIS) managers.Prev Med Rep. 2021; 21101296https://doi.org/10.1016/j.pmedr.2020.101296
- A randomized trial of the effect of centralized reminder/recall on immunizations and preventive care visits for adolescents.Acad Pediatr. 2013; 13: 204-213https://doi.org/10.1016/j.acap.2013.01.002
- Effectiveness of centralized text message reminders on human papillomavirus immunization coverage for publicly insured adolescents.J Adolesc Health. 2015; 56: S17-S20https://doi.org/10.1016/j.jadohealth.2014.10.273
- Seasonal influenza vaccination reminders for children with high-risk conditions: a registry-based randomized trial.Am J Prev Med. 2012; 42: 71-75
- A randomized intervention of reminder letter for human papillomavirus vaccine series completion.J Adolesc Health. 2015; 56: 85-90https://doi.org/10.1016/j.jadohealth.2014.08.014
- Centralized reminder/recall to increase influenza vaccination rates: a two-state pragmatic randomized trial.Acad Pediatr. 2020; 20: 374-383https://doi.org/10.1016/j.acap.2019.10.015
- Effect of state immunization information system based reminder/recall for influenza vaccinations: a randomized trial of autodialer, text, and mailed messages.J Pediatr. 2020; 221: 123-131.e4https://doi.org/10.1016/j.jpeds.2020.02.020
- Effect of state immunization information system centralized reminder and recall on HPV vaccination rates.Pediatrics. 2020; 145https://doi.org/10.1542/peds.2019-2689
Gurfinkel D, Kempe A, Albertin C, et al. Centralized reminder/recall of HPV vaccination: findings from two states: a randomized clinical trial. J Adolesc Health. In press.
- Parental hesitancy about routine childhood and influenza vaccinations: a national survey.Pediatrics. 2020; 146https://doi.org/10.1542/peds.2019-3852
- Prevalence and characteristics of HPV vaccine hesitancy among parents of adolescents across the US.Vaccine. 2020; 38: 6027-6037https://doi.org/10.1016/j.vaccine.2020.06.074
- Impacting delayed pediatric influenza vaccination: a randomized controlled trial of text message reminders.Am J Prev Med. 2015; 48: 392-401https://doi.org/10.1016/j.amepre.2014.10.023
- Using behavioral economics to encourage parent behavior change: opportunities to improve clinical effectiveness.Acad Pediatr. 2019; 19: 4-10https://doi.org/10.1016/j.acap.2018.08.010
Lerner C, Albertin C, Casillas A, et al. Patient portal reminders for pediatric influenza vaccinations: a randomized clinical trial. Pediatrics. In press.
- Text message reminders for second dose of influenza vaccine: a randomized controlled trial.Pediatrics. 2015; 135: e83-e91https://doi.org/10.1542/peds.2014-2475
- Projected population benefit of increased effectiveness and coverage of influenza vaccination on influenza burden in the United States.Clin Infect Dis. 2020; 70: 2496-2502https://doi.org/10.1093/cid/ciz676
- Fragmentation of immunization history among providers and parents of children in selected underserved areas.Am J Prev Med. 2002; 23: 106-112https://doi.org/10.1016/s0749-3797(02)00463-4
- How complete are immunization registries? The Philadelphia story.Ambul Pediatr. 2006; 6: 21-24https://doi.org/10.1016/j.ambp.2005.08.006
- Factors associated with underimmunization at 3 months of age in four medically underserved areas.Public Health Rep. 2004; 119: 479-485https://doi.org/10.1016/j.phr.2004.07.005
- Centralized reminder/recall to increase immunization rates in young children: how much bang for the buck?.Acad Pediatr. 2017; 17: 330-338https://doi.org/10.1016/j.acap.2016.11.016
- Increasing coverage of appropriate vaccinations: a community guide systematic economic review.Am J Prev Med. 2016; 50: 797-808https://doi.org/10.1016/j.amepre.2015.11.003
- Ten great public health achievements–United States, 2001-2010.MMWR Morb Mortal Wkly Rep. 2011; 60: 619-623
- Supporting and sustaining centralized reminder/recall for immunizations: qualitative insights from stakeholders.Vaccine. 2019; 37: 6601-6608https://doi.org/10.1016/j.vaccine.2019.09.055
Citywide Immunization Registry. New Text Messaging via the Online Registry to Recall Patients Due/Overdue Immunizations. New York City Department of Health and Mental Hygiene. Available at: https://www1.nyc.gov/assets/doh/downloads/pdf/cir/txt-messaging-guide.pdf. Accessed February 1, 2021.
- The Future of the Public's Health in the 21st Century.National Academies Press (US), 2002
- Primary Care and Public Health: Exploring Integration to Improve Population Health.National Academies Press (US), 2012
- Collaboration Between Health Care and Public Health: Workshop Summary.National Academies Press (US), 2016
- Collaboration to Build Healthier Communities: A Report for the Robert Wood Johnson Foundation Commission to Build a Healthier America.2013: 24
- Roundtable on Population Health Improvement.National Academies of Sciences, 2021 (Available at:) (Accessed February 24, 2021)
Article Info
Publication History
Accepted:
February 24,
2021
Received:
February 6,
2021
Footnotes
The authors have no conflicts of interest to disclose.
Identification
Copyright
© 2021 Published by Elsevier Inc. on behalf of Academic Pediatric Association
