Limits of technological solutions to provider monitoring

Increasing health care worker attendance is a critical challenge in improving health care delivery. An evaluation by J-PAL Executive Director Iqbal Dhaliwal and affiliated researcher Rema Hanna (Harvard University) of a biometric monitoring system for recording doctors’ attendance in primary health centers in Karnataka, India revealed that despite the availability of real-time, accurate attendance data, the government was unable to enforce the planned incentives and penalties, and overall attendance of health workers did not increase. The government decided to not scale up the program as planned, saving millions of dollars in costs and countless hours of staff time needed to run it.

The Problem

How can governments encourage public health workers to show up to work?

The government is the primary health care provider in many developing countries. Despite large budgetary allocations and an extensive network of public health centers, health outcomes remain poor. High rates of staff absenteeism, enabled by poor systems of accountability, contribute to low levels of care.

India has a network of over 24,000 public Primary Health Centers (PHCs) that provide basic services, such as immunization and antenatal care, to the poor and are the primary provider in most rural areas. In Karnataka, as in many Indian states, this system is plagued by absenteeism. In 2010, doctors were present at PHCs in the state of Karnataka less than 40 percent of the time. As a result, many citizens that visit PHCs to see health staff leave without receiving medical care.

Previously tested interventions in India targeting health worker absenteeism were unsuccessful because supervisors maintained discretion in recording attendance and issuing penalties. A biometric monitoring system that accurately records attendance and reports it to supervisors in real time, combined with a system of incentives and penalties for unauthorized absences, offered promise as a way to improve staff attendance and patient health.

The Research

A randomized evaluation in India revealed the political and logistical challenges faced by the government in enforcing penalties for health worker absences.

The National Rural Health Mission (NRHM) of Karnataka—the lead department for the delivery of health services in the state—wanted to improve attendance in PHCs in rural areas across the state. In particular, improving doctor attendance was a key policy goal for NRHM because these clinics do not open without a doctor on site. From 2011–2012, NRHM collaborated with Dhaliwal and Hanna to evaluate the impact on attendance of the Integrated Medical Information and Disease Surveillance System (IMIDSS).

NRHM equipped randomly selected PHCs in five districts of the state with an IMIDSS device, consisting of a fingerprint reader and a multipurpose mobile phone. Staff recorded their attendance on the device via thumb impression at the beginning and end of each day. Attendance data was transferred wirelessly using the cellular network to the state health department headquarters so supervisors could track attendance in near real time. Headquarters staff analyzed this data and then communicated it back to the districts.

Using this data, the government planned to issue both positive and negative incentives. Staff with good attendance records would earn cash rewards and non-cash recognition. Staff with unauthorized absences would receive an escalating set of penalties, including reprimand letters, deduction of earned leave, and finally docking of pay.

For more detail, see the evaluation summary.

From Research to Action

The results of the randomized evaluation informed the government’s decision to end the attendance monitoring program, saving taxpayers money and avoiding an added burden and wasted time on the part of the government.

Equipping health centers with IMIDSS slightly improved attendance of nurses, lab technicians, and pharmacists, but did not improve doctor attendance—NRHM’s policy priority. During random checks after the IMIDSS was rolled out, doctors were present only 31 percent of the time. Throughout the roll out of IMIDSS, it became clear that the government found it hard to implement the reform as planned.

Why weren’t the reforms implemented as designed? First, existing civil service rules made the incentive system difficult to implement, particularly for doctors. Enforcing penalties is both time- and effort-intensive for sub-district health staff, who are charged with monitoring attendance. Levying penalties for non-attendance on doctors in particular requires coordination among multiple state government officials and other departments (the process is less complex for nurses and other health workers). As a result, very few doctors received formal notice that they were noncompliant with the attendance policy (the first step in imposing penalties); even fewer saw a resulting reduction in pay. Conversely, on the positive incentives, the department was unable to finalize an objective criterion for awarding doctors and PHCs with the best attendance record.

Second, NRHM was reluctant to actually enforce the penalties, because it feared losing doctors to the private health sector. Private clinics offer more attractive work schedules and leave entitlements, while doctors in the public health system earn less and are expected to work longer hours. The relaxed accountability around attendance before the IMIDSS was implemented may have acted as an implicit perk of work in the public system; the government feared that attendance penalties for doctors could ultimately harm health outcomes if they drove more doctors away. Doctors were already difficult to recruit and retain: at the time of the evaluation, more than 20 percent of PHCs had at least one vacant doctor position. The decision to discontinue the IMIDSS reflected these concerns.

Altogether, the evaluation revealed to the government the underlying limitations of using a purely technical solution to improve doctors’ attendance. While the government may have felt that just putting in the attendance monitoring system would act as a deterrent to truant doctors and increase attendance, in practice, without the ability or willingness to enforce penalties, the monitoring system and the data it generated had limited effect and provided little value to NRHM.

Following a discussion between the researchers and the NRHM of the results and mechanisms, the government decided to cancel the planned scale-up of IMIDSS. This decision determined that scarce resources would not be spent on a program that failed to address the NRHM’s policy priorities, despite a strong theory of change, sound monitoring technology, and substantial effort put into the program’s implementation.

The choice not to scale IMIDSS saved taxpayers US$604,000 in the initial equipment costs alone of providing all the PHCs with the devices. Adding the cost of running the data center, hiring additional monitoring staff, maintaining and replacing thousands of devices, monthly data charges, and the daily burden on thousands of staff and supervisors of entering and reviewing their attendance on the devices, the avoided cost of not scaling up this program amounted to millions of dollars that could instead be allocated to other policy priorities. The Karnataka government and NRHM displayed considerable foresight and patience in piloting and evaluating the program rather than rolling it out in the entire state. Additionally, this choice exhibited incredible courage in admitting that its own pilot program did not work and in deciding not to scale it up, despite the obvious government pressure to “show action” by implementing new programs.

References

Dhaliwal, Iqbal, and Rema Hanna. 2017. "The Devil is in the Details: The Successes and Limitations of Bureaucratic Reform in India." Journal of Development Economics 124 (January): 1-21. https://doi.org/10.1016/j.jdeveco.2016.08.008.