Using Social Networks, Incentives, and Reminders to Boost Vaccine Demand

Overview and Policy Issue

Immunization is a highly cost-effective way of improving child survival. However, over two million children around the world die each year from vaccine-preventable diseases. Governments across low- and middle-income countries have made immunizations free and focused on strengthening vaccine delivery through mobile camps and frequent immunization drives.

Despite large investments to increase access, full immunization coverage for BCG, polio, DPT, and measles remains low in several countries, including India. As of 2016, only 62 percent of Indian children were fully immunized against these diseases. A large fraction of children receive the first vaccine, but do not complete the full schedule, reflecting high initial motivation but difficulty completing later vaccination visits.

Previous research suggests that offering incentives,^1,2 sending reminders,^3,4 and using influential individuals in the community^5,6 can build parental demand for vaccination and increase child immunization. Yet these programs have rarely been evaluated to examine impact at scale, compare different variants of each strategy, or test these policies in combination with one another.

Given limited resources, it is important to determine which combinations of these policy tools are most effective, leading to the largest increase in immunization, and those which are most cost-effective, leading to the largest increase in immunization per dollar spent. It is also important to know whether a particular policy would be more effective in some villages compared to others.

To test how different policy combinations affect immunization rates in an at-scale experiment and to determine in what type of villages they would be most effective, J-PAL affiliates and other researchers randomized the provision of incentives, targeted reminders, and local immunization ambassadors in the Indian state of Haryana. They combined the experimental results with machine learning techniques to ascertain the most effective and cost-effective packages, as well as to predict what village characteristics make policy packages the most effective.

Context: Immunization in Haryana

Despite large investments in vaccine delivery, immunization rates are particularly low in some districts of Haryana, India. In the study population, about 86 percent of children aged 12-23 months received at least three vaccines at baseline. According to parents’ reports, however, only 39 percent were fully immunized and less than 20 percent had received the measles vaccine before 15 months (it should be received between 10-12 months for on-time immunization).

As part of the Universal Immunization Program (UIP), Auxiliary Nurse Midwives (ANMs) run free monthly immunization camps in villages. Local health workers, Accredited Social Health Activists (ASHAs), help inform and motivate parents to attend.

Details of the Intervention

To complement UIP, the Government of Haryana was interested in innovative interventions to increase parental demand for immunization. The government collaborated with researchers and J-PAL South Asia to conduct a randomized evaluation in seven districts with especially low vaccine coverage. The study involved 140 primary health centers (PHC) and 755 subcenters and focused on children under 12 months of age receiving five basic immunizations (BCG, Penta-1, Penta-2, Penta-3, and Measles-1).

From 2016 to 2018, researchers evaluated the effect of three policy tools on increasing demand for immunization: small incentives, targeted reminders, and local immunization ambassadors.

First, the project team partnered with SANA, a software development group at MIT, to design a cheap, simple, user-friendly e-health platform to register all children attending immunization camps. Nurses widely adopted the software, which served as the data infrastructure for both the implementation of the project and its evaluation. Registrations included parents’ mobile numbers and tracked children’s immunizations using a unique ID for the project. The software also helped calculate incentives and deliver SMS reminders.

Over the course of the project, approximately 295,000 children were recorded on the platform.

Small incentives

Caregivers at 70 randomly selected PHCs received a small mobile phone credit each time they brought their children to get immunized. ANMs put up posters with information on incentives, and both ASHAs and ANMs informed caregivers about the incentives. Within PHCs, researchers randomly selected subcenters to receive varying levels of incentives. Caregivers received one of four incentive structures:

1. High incentive, flat payment: 90 rupees (US$1.25) per immunization (450 rupees total)
2. High incentive, increasing payment: 50 rupees (US$0.70) for the first three immunizations, 100 for the fourth, 200 for the fifth (450 rupees total)
3. Low incentive, flat payment: 50 rupees per immunization (250 rupees total)
4. Low incentive, increasing payment. 10 rupees (US$0.14) for the first three immunizations, 60 for the fourth, 160 for the fifth (250 rupees total)

Targeted reminders

Caregivers were randomly assigned to receive text and voice call messages reminding them that their child was due to receive a specific vaccine. Researchers randomly varied the fraction of caregivers receiving reminders in the catchment area of each subcenter: either zero, 33, or 66 percent.

Local immunization ambassadors

Researchers leveraged social networks to test an ambassador program in a subset of 912 villages. The ambassadors received one text message and one voice call every month asking them to remind their friends, family, and other community members of the value of immunization and, in villages with incentives, remind them about the incentives.

Within each village, 17 randomly selected individuals were asked to identify people with certain characteristics. The six people nominated most frequently in each village were recruited as ambassadors for the program. Villages were randomly assigned to one of five different ambassador selection strategies:

1. Information hubs: respondents were asked to identify who was good at relaying information.
2. Trusted individuals: respondents were asked to identify who was generally trusted to provide good advice on health or agricultural questions
3. Trusted information hubs: respondents were asked to identify who was both trusted, and good at transmitting information
4. Random selection: six ambassadors selected randomly
5. Comparison group with no ambassadors

The researchers cross-randomized these three policy interventions, resulting in a total of 75 different policy bundles. Given the many policies being evaluated, researchers developed an innovative machine learning technique to determine which combinations were most effective and cost-effective.

To measure immunization, researchers used administrative data that ANMs recorded on the SANA health platform. J-PAL verified data quality through a sample of household visits. Researchers used receipt of the measles vaccine, the last vaccine in the immunization schedule, as an indicator for full immunization. The main measure of performance, at the village level, is the number of measles vaccines delivered every month in each village, corresponding to the number of vaccines delivered in one particular immunization session.

Results

Information hubs improved full immunization rates, while SMS reminders alone had no impact. On average across incentive and non-incentive villages, in communities where information hubs (individuals identified as being good at spreading information) acted as immunization ambassadors, 1.89 more children in the village completed the full immunization schedule per month relative to 7.32 in the comparison group (26 percent increase). In contrast, SMS reminders sent directly to a much larger set of parents had no impact on their own, compared to villages with no SMS reminder.

The most effective policy option was a full package that combined local immunization ambassadors selected by the community, incentives that increased in amount across the immunization schedule, and SMS reminders to caregivers about the next scheduled vaccine. This combination increased full immunization rates by 55 percent (4.02 measles vaccines per village per month) relative to the comparison group.

The results suggest that local immunization ambassadors amplified the effect of the other interventions, possibly by diffusing information about the incentives widely and explaining the content of personalized reminders. In this combination, high and low incentives are equally effective as long as they are combined with other interventions. In contrast, incentives that are flat across the immunization schedule had no impact in any combination.

There was, however, substantial heterogeneity in the impact of this combined intervention. The package was most effective in poorer areas with low initial immunization levels. In the villages where it was the most effective, it increased the immunization rate six-fold (approximately 500 percent) relative to the comparison group. The estimates suggest that in the most affected villages, the rate of on-time immunization for measles increased from 13 percent to 61 percent. However, in some villages with higher baseline immunization, the effect of the full package was actually negative.

In the areas where it was most effective, the full package does not increase cost per immunization compared to the status quo immunization program. On average, the combined intervention increased costs per immunization compared to current status quo. However, because this package had larger impacts in areas with low initial immunization levels, it was as cost-effective as the status quo for each immunization received: even though it increased marginal costs by paying an incentive to parents for each immunization, it spread the fixed costs of running an immunization program over a much larger number of immunizations delivered.

The most cost-effective policy combined information hubs or trusted information hubs with SMS reminders. This policy increased the number of immunizations per dollar by 9.1 percent relative to the comparison group receiving standard UIP services because the combination of the two interventions was both effective and inexpensive. In contrast, policies involving any amount of incentives were less cost-effective than the status quo on average.

Policy Lessons

A combination of inexpensive and effective interventions can both increase immunization and reduce the cost per immunization compared to the status quo.

Combining SMS reminders and community-nominated ambassadors increased immunization and reduced the cost per immunization. A more expensive policy (adding incentives to the package) does not increase cost per immunization if it is implemented only where it is predicted to be the most effective (areas with low initial immunization levels).

Policymakers can benefit from leveraging social networks to accelerate information diffusion and increase the effectiveness of other policies.

Simply asking a few villagers who are good people to spread information was an easy, inexpensive, and reliable way to identify well-connected information hubs. The results of this evaluation are consistent with other studies on the importance of social networks for diffusing information about vaccination in Indonesia,⁷ microfinance in India,⁸ and more.⁹

Before widespread adoption of a policy, it is important to test at scale what combinations of interventions and what specific version of the policy might be most effective.

On their own, SMS reminders for immunization, promoted widely in India and elsewhere,^10,11 had no impact in Haryana on their own, but were a useful complement to other interventions. Incentives were most effective when combined with SMS reminders and community-nominated ambassadors, and they never worked when they were flat. However, the “dosage” was less important: both low incentives and a lower fraction of caregivers receiving SMS reminders were effective.

Policy Influence

The Government of Haryana is working with researchers and J-PAL South Asia to scale the most cost-effective package, information hubs and targeted reminders, across the state with support from J-PAL’s Innovation in Government Initiative.

In areas with particularly low immunization coverage, J-PAL is recommending policymakers supplement the package with the smaller, increasing incentives given its even larger improvements in immunization and its cost-effectiveness.

References

1. Banerjee, A. V., E. Duflo, R. Glennerster, and D. Kothari. 2010. “Improving Immunisation Coverage in Rural India: Clustered Randomised Controlled Evaluation of Immunisation Campaigns with and without Incentives”. BMJ 340, c2220.

2. Domek, Gretchen J., Ingrid L. Contreras-Roldan, Sean T. O’Leary, Sheana Bull, Anna Furniss, Allison Kempe, and Edwin J. Asturias. 2016. “SMS Text Message Reminders To Improve Infant Vaccination Coverage In Guatemala: A Pilot Randomized Controlled Trial”. Vaccine 34 (21): 2437-2443.

3. Uddin, Md. Jasim, Md. Shamsuzzaman, Lily Horng, Alain Labrique, Lavanya Vasudevan, Kelsey Zeller, Mridul Chowdhury, Charles P. Larson, David Bishai, and Nurul Alam. 2016. “Use Of Mobile Phones For Improving Vaccination Coverage Among Children Living In Rural Hard-To- Reach Areas And Urban Streets Of Bangladesh”. Vaccine 34 (2): 276-283.

4. Regan, Annette K., Lauren Bloomfield, Ian Peters, and Paul V. Effler. 2017. “Randomized Controlled Trial Of Text Message Reminders For Increasing Influenza Vaccination”. The Annals Of Family Medicine 15 (6): 507-514.

5. Alatas, Vivi, Arun G. Chandrasekhar, Markus Mobius, Benjamin A. Olken, and Cindy Paladines. “When Celebrities Speak: A Nationwide Twitter Experiment Promoting Vaccination in Indonesia”. NBER Working Paper No. 25589, February 2019.

6. Banerjee, Abhijit, Arun G Chandrasekhar, Esther Duflo, and Matthew O Jackson. 2019. “Using Gossips To Spread Information: Theory And Evidence From Two Randomized Controlled Trial”. The Review Of Economic Studies 86 (6): 2453-2490.

7. Alatas, Vivi, Arun G. Chandrasekhar, Markus Mobius, Benjamin A. Olken, and Cindy Paladines. “When Celebrities Speak: A Nationwide Twitter Experiment Promoting Vaccination in Indonesia”. NBER Working Paper No. 25589, February 2019.

8. Banerjee, A., A. G. Chandrasekhar, E. Duflo, and M. O. Jackson. 2013. “The Diffusion Of Microfinance”. Science 341 (6144): 1236498-1236498. doi:10.1126/science.1236498.

9. Beaman, Lori A., Ariel BenYishay, Jeremy Magruder, and Ahmed Mushfiq Mobarak. 2018. “Can Network Theory-Based Targeting Increase Technology Adoption?”. SSRN Electronic Journal. doi:10.2139/ssrn.3225815.

10. Uddin, Md. Jasim, Md. Shamsuzzaman, Lily Horng, Alain Labrique, Lavanya Vasudevan, Kelsey Zeller, Mridul Chowdhury, Charles P. Larson, David Bishai, and Nurul Alam. 2016. “Use Of Mobile Phones For Improving Vaccination Coverage Among Children Living In Rural Hard-To- Reach Areas And Urban Streets Of Bangladesh”. Vaccine 34 (2): 276-283.

11. Regan, Annette K., Lauren Bloomfield, Ian Peters, and Paul V. Effler. 2017. “Randomized Controlled Trial Of Text Message Reminders For Increasing Influenza Vaccination”. The Annals Of Family Medicine 15 (6): 507-514.

This research was made possible by

Fieldwork: J-PAL South Asia
Partners: Government of Haryana, SANA Health
Funders: 3ie, Givewell, J-PAL Government Partnerships Initiative, USAID Development Innovation Ventures